datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
atomic-metrics-six-task-preferences
Six-task benchmark inputs
Seed 17. No demographic conditioning. Each task has shared train100.jsonl and test500.jsonl for Atomic Metrics, five judge variants, and learned baselines. Pair plans cover all 100 training rows once. Atomic Metrics extraction and BT/LR fitting use train100. Judges use the same test500. RM and WIMHF in the matched-data comparison use train100; rm_train_full is an explicitly separate expanded-data setting and must not be described as train100.… See the full description on the dataset page: https://huggingface.co/datasets/tintin1027/atomic-metrics-six-task-preferences.pairs_Movies_and_TVopenai-tldr-summarisation-preferences
Human feedback data
This is the version of the dataset used in https://arxiv.org/abs/2310.06452.
If starting a new project we would recommend using https://huggingface.co/datasets/openai/summarize_from_feedback.
See https://github.com/openai/summarize-from-feedback for original details of the dataset.
Here the data is formatted to enable huggingface transformers sequence classification models to be trained as reward functions.
pairs_Grocery_and_Gourmet_Foodifeval-obf-rl-preferences
IFEval Obfuscation — Full Preference Pairs (2023 constitution)
Preference pairs over responses from a Wood-Labs eval-aware 49B organism (nemotron-nas / DeciLM),
judged under the 2023 Claude constitution, for training reward models / DPO on verbalized
evaluation-awareness (VEA). These are the FULL files the RMs actually trained on — not the
earlier filtered subset.
Files (DPO-ready)
prefs_2023_leak_full.jsonl — 14,074 pairs. Judge saw the CoT + answer ("leak"… See the full description on the dataset page: https://huggingface.co/datasets/rlundqvist/ifeval-obf-rl-preferences.adaption-clinical-triage-preferences
This dataset is a remastered version prepared using Adaption's Adaptive Data platform.
adaption-clinical_triage_preferences
Multi-turn conversational preference dataset designed for fine-grained safety and tone calibration in emergency first aid and symptom triage. Each sample pairs a user prompt with chosen and rejected AI responses, contrasting concise, grounded clinical guidance against subtly misleading or overly verbose advice. It supports reward modeling and preference… See the full description on the dataset page: https://huggingface.co/datasets/gimmy256/adaption-clinical-triage-preferences.amadablam-dpo-preferences
Ama Dablam DPO Preference Data
Preference pairs used to DPO-tune Ama Dablam,
a 322M trilingual (Nepali/Maithili/Bhojpuri) language model, across all three languages
and three writing systems (Devanagari, IAST, phonetic romanization). See the
technical report §9 for full
methodology.
Splits
split
rows
purpose
train
14,152
DPO Stage 2 preference-optimization training
validation
744
preference-accuracy / forgetting evaluation
warmup
3,203
Stage 1… See the full description on the dataset page: https://huggingface.co/datasets/spandyie/amadablam-dpo-preferences.Human-Preferences-Alignment-KTO-Dataset-AI-Services-Genuine-User-Reviews
Human Preferences Alignment KTO Dataset of AI Service User Reviews of ChatGPT Gemini Claude Perplexity
Introduction to Human Preferences Alignment
There are many methods of applying Human Preference Alignment techniques to help model align in the supervised finetuning stage, including RLHF Reinforcement Learning from Human Feedback(paper), PPO Proximal policy optimization(paper/equation), DPO Direct Preference Optimization (paper/equation), KTO Kahneman-Tversky… See the full description on the dataset page: https://huggingface.co/datasets/DeepNLP/Human-Preferences-Alignment-KTO-Dataset-AI-Services-Genuine-User-Reviews.PACIFIC-big-five-trait-preferencesDataset For Paper "Can LLMs Discern the Traits Influencing Your Preferences? Evaluating Personality-Driven Preference Alignment in LLMs"
PACIFIC (Preference Alignment for Choices Inference via Five-factor Identity Characterization) is a psychometrics-grounded dataset for studying whether Large Language Models can use stable personality traits — rather than exhaustive preference logs — as a latent signal for inferring user preferences on unseen queries.
It contains 1,200 preference–query pairs… See the full description on the dataset page: https://huggingface.co/datasets/TylerZ0931/PACIFIC-big-five-trait-preferences.PKU-Alignment_PKU-SafeRLHF-Safer-PreferenceShareGPTultrafeedback-binarized-preferences-cleaned-deGerman translation from Mixtral (not the best one, and might contain comments etc, despite it prompted not to, but this is mostly for testing purposes atm) of a first part of the dataset as provided by argilla.
CyberNative_Code_Vulnerability_Security_DPO-PreferenceShareGPTHalluGuard-Preferences-76k
🛡️ HalluGuard-Prefs: A 76K Synthetic Preference Dataset for Document-Grounded Hallucination Detection
🌍 Overview
This dataset was used to fine-tune HalluGuard-Qwen3-4B via Odds Ratio Preference Optimization (ORPO).
It consists of 76,708 high-quality preference tuples designed to teach the model how to reason and justify its hallucination detection.
📖 Publication
This dataset was introduced in our paper at the 64th Annual Meeting of the Association… See the full description on the dataset page: https://huggingface.co/datasets/lrsbrgrn/HalluGuard-Preferences-76k.vicgalle_configurable-system-prompt-multitask-PreferenceShareGPTLLM-Preferences-HelpSteer2
LLM-Preferences-HelpSteer2
Author: Min Li
Blog: https://rlhflow.github.io/posts/2025-01-22-decision-tree-reward-model/
Dataset Description
This dataset contains pairwise preference judgments from 34 modern LLMs on response pairs from the HelpSteer2 dataset.
Key Features
Contains 9,125 response pairs from HelpSteer2-Preference
Includes preferences from 9 closed-source and 25 open-source LLMs
Documents position bias analysis and preference consistency metrics… See the full description on the dataset page: https://huggingface.co/datasets/RLHFlow/LLM-Preferences-HelpSteer2.Curriculum_DPO_preferences
Curriculum DPO Preference Pairs
This repository provides the curriculum DPO preference pairs used in the paper Curri-DPO, which explores enhancing model alignment through curriculum learning and ranked preferences.
Datasets
Ultrafeedback
The Ultrafeedback dataset contains 64K preference pairs. We randomly sample 5K pairs and rank responses for each prompt, organizing them into three difficulty levels: easy, medium, and hard, based on response scores.… See the full description on the dataset page: https://huggingface.co/datasets/ServiceNow-AI/Curriculum_DPO_preferences.M4-ai_prm_dpo_pairs_cleaned-PreferenceShareGPTwinglian_no_robots_rlhf-PreferenceShareGPTPKU-Alignment_PKU-SafeRLHF-Better-PreferenceShareGPTChaiverse-Leaderboard-PreferenceShareGPTadaption-python-code-preferences
This dataset is a remastered version prepared using Adaption's Adaptive Data platform.
adaption-python_code_preferences
This dataset consists of paired code completions designed for preference alignment of language models. Each entry contains a coding prompt alongside a chosen and a rejected response focused on Python, Django, and Flask topics. The preference pairs facilitate training models to output better implementation logic and framework usage.
Dataset size… See the full description on the dataset page: https://huggingface.co/datasets/himanshunakrani9/adaption-python-code-preferences.argilla_ultrafeedback-multi-binarized-quality-preferences-cleaned-PreferenceShareGPTjondurbin_contextual-dpo-v0.1-PreferenceShareGPTstudents-subject-preferences
Students' Subject Preferences
A small survey-style dataset recording which school subjects five students like and dislike.
Each row is one student: their ID, the subjects they named as favorites, and the subjects they
named as least favorites. Subject names are in Mongolian Cyrillic.
Files
File
Rows
Description
data/train.jsonl
5
One JSON object per student
Schema
Column
Type
Description
student_id
int
Student identifier… See the full description on the dataset page: https://huggingface.co/datasets/sumya123/students-subject-preferences.Intel_orca_dpo_pairs-PreferenceShareGPTantiven0m_catboros-3.2-dpo-PreferenceShareGPTtatsu-lab_alpaca_farm_human_preference-PreferenceShareGPTefederici_alpaca-vs-alpaca-orpo-dpo-PreferenceShareGPTargilla_distilabel-math-preference-dpo-PreferenceShareGPTsummarization-preferences
Summarization Preferences Dataset
This is a processed subset of the openai/summarize_from_feedback comparisons subset, including training and validation splits.
Description
The original dataset consists of paired human comparisons between summary candidates for given source texts. This processed version aggregates all comparisons per unique text to determine the overall best (chosen) and worst (rejected) summaries using the Bradley-Terry model.
Fields
text:… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/summarization-preferences.