datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
hh-rlhf
Dataset Card for HH-RLHF
Dataset Summary
This repository provides access to two different kinds of data:
Human preference data about helpfulness and harmlessness from Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. These data are meant to train preference (or reward) models for subsequent RLHF training. These data are not meant for supervised training of dialogue agents. Training dialogue agents on these data is likely… See the full description on the dataset page: https://huggingface.co/datasets/Anthropic/hh-rlhf.hh-rlhf-49k-ja-single-turnThis dataset was created by automatically translating part of "Anthropic/hh-rlhf" into Japanese, and selected for single turn conversations.You can use this dataset for RLHF and DPO.
hh-rlhf repository
https://github.com/anthropics/hh-rlhf
Anthropic/hh-rlhf
https://huggingface.co/datasets/Anthropic/hh-rlhf
Nemotron-RLHF-GenRM-v1
Dataset Description:
This dataset is designed to train Generative Reward Models (GenRMs). It leverages reinforcement learning at scale to train accurate and robust GenRMs that generalize better than traditional Bradley-Terry models and reduce the risk of reward hacking.
The dataset is composed of:
Preference data focused on diverse domains
A synthetic safety blend
The data follows a "meta-prompt" structure where the model is instructed to act as an expert evaluation judge. For… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-RLHF-GenRM-v1.social-reasoning-rlhf
Dataset Summary
This repository provides access to a social reasoning dataset that aims to provide signal to how humans navigate social situations, how they reason about them and how they understand each other. It contains questions probing people's thinking and understanding of various social situations.
This dataset was created by collating a set of questions within the following social reasoning tasks:
understanding of emotions
intent recognition
social norms
social… See the full description on the dataset page: https://huggingface.co/datasets/ProlificAI/social-reasoning-rlhf.hh_rlhf_cn
hh-rlhf中文翻译版本
基于Anthropic论文Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback 开源的helpful 和harmless数据，使用翻译工具进行了翻译。hh_rlhf_train.jsonl 合并中英文训练集数据 清洗过后17万条hh_rlhf_test.jsonl 合并中英文测试集数据 清洗过后9千条harmless_base_cn_train.jsonl 42394条harmless_base_cn_test.jsonl 2304条helpful_base_cn_train.jsonl 43722条helpful_base_cn_test.jsonl 2346条
实验报告
相关rlhf实验报告:https://zhuanlan.zhihu.com/p/652044120
hh-rlhf-12k-ja
hh-rlhf-12k-ja
This repository provides a human preference dataset developed by LLM-jp, a collaborative project launched in Japan.
This dataset is a Japanese translation of a subset of hh-rlhf using DeepL.
This dataset consists of 12,000 entries randomly sampled from hh-rlhf. Specifically, it includes a random selection of 3,000 entries from the training splits of the four groups: harmless-base, helpful-base, helpful-online, and helpful-rejection-sampled. For more information on… See the full description on the dataset page: https://huggingface.co/datasets/llm-jp/hh-rlhf-12k-ja.Deepseek-PRM-DataSee https://github.com/RLHFlow/RLHF-Reward-Modeling/tree/main/math-rm for more data information.
anthropic_hh_rlhf_japanesehttps://huggingface.co/datasets/Anthropic/hh-rlhf
Japanese Translation
hh-rlhf-strength-cleaned
Dataset Card for hh-rlhf-strength-cleaned
Other Language Versions: English, 中文.
Dataset Description
In the paper titled "Secrets of RLHF in Large Language Models Part II: Reward Modeling" we measured the preference strength of each preference pair in the hh-rlhf dataset through model ensemble and annotated the valid set with GPT-4. In this repository, we provide:
Metadata of preference strength for both the training and valid sets.
GPT-4 annotations on the valid set.
We… See the full description on the dataset page: https://huggingface.co/datasets/OpenMOSS-Team/hh-rlhf-strength-cleaned.formatted-hh-rlhfhh-rlhf
Dataset Card for HH-RLHF
Dataset Summary
This repository provides access to two different kinds of data:
Human preference data about helpfulness and harmlessness from Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. These data are meant to train preference (or reward) models for subsequent RLHF training. These data are not meant for supervised training of dialogue agents. Training dialogue agents on these data is likely to lead… See the full description on the dataset page: https://huggingface.co/datasets/polinaeterna/hh-rlhf.french_hh_rlhf
Dataset Card for french_hh_rlhf
This dataset offers a french translation of the famous Anthropic/hh-rlhf dataset in order to improve the allignement work/research in the french NLP community.
Dataset Card Contact
ntnq
rlhf_helpful_evalhh-rlhf-49k-jaThis dataset was created by automatically translating part of "Anthropic/hh-rlhf" into Japanese.This dataset is also included in "mosaicml/dolly_hhrlhf".
The "ng_translation" flag indicates that the translation was not successful, and "1" means that the translation failed.
Therefore, for data with "1", "instruction" and "instruction_en" contain the same text.
以下の通りに読み込むことで"ng_translation"が"1"（翻訳誤り）のものを除外して使用できます。
pip install datasets
from datasets import Dataset, load_dataset
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/kunishou/hh-rlhf-49k-ja.HH-RLHF-MetaAligner-Data
Dataset Card for HH-RLHF-MetaAligner-Data
Dataset Summary
This repository provides access to 2 subsets of the MetaAligner data built from HH-RLHF:
The equal-preference subset is included in: equal_train.json and equal_val.json.
The preference subset is included in: preference_train.json, preference_val.json, and preference_test.json.
Disclaimer: As the dialogue data comes from HH-RLHF, the data contain content that may be offensive or upsetting. Topics include, but are… See the full description on the dataset page: https://huggingface.co/datasets/MetaAligner/HH-RLHF-MetaAligner-Data.korean_rlhf_dataset성균관대학교 산학협력프로젝트 과정에서 한국어 llm 모델 SFT 학습을 위해 구축한 데이터셋 입니다.2023-09-25오픈 어시스턴트 data에서 오픈 어시스턴트를 포함하는 데이터 삭제-> 답변에 오픈 어시스턴트라고 하는 경우가 나오기 때문또한 스탠포드 대학 번역 데이터에서 번역 과정 오류로 input에 입력없음 과 같이 추가된 부분 삭제그리고 <unk> 등으로 gpt 상에서 번역 오류가 난 것들을 삭제
자연스러움을 위해 stanford alpaca data, oig_chip2를 ChatGPT3.5 turbo 16k를 이용하여 새롭게 전처리 과정을 거쳤습니다.https://github.com/JoJo0217/rlhf_korean_dataset/tree/main여기에서 자세한 설명을 볼 수 있으며데이터의 구성은 다음과 같습니다.
데이터 구성
데이터 종류
개수
url
koalpaca v1.1
21155… See the full description on the dataset page: https://huggingface.co/datasets/jojo0217/korean_rlhf_dataset.cvalues_rlhfConverted from: https://modelscope.cn/datasets/damo/CValues-Comparison/summary. We obtained harmless set by selecting pos_type="拒绝为主" and neg_type="风险回复". We obtained helpful set by selecting pos_type="拒绝&正向建议" and neg_type="拒绝为主".
hh-rlhf-ko
Original Dataset: Anthropic/hh-rlhf
Translation by using maywell/Synatra-7B-v0.3-Translation
Translating in progress...
processed-hh-rlhf
Dataset Card for Processed-Hh-RLHF
This is a dataset that processes hh-rlhf into an easy-to-use conversational and human-preference form.
no_robots_rlhfhh-rlhf
Dataset Card for HH-RLHF
Dataset Summary
This repository provides access to two different kinds of data:
Human preference data about helpfulness and harmlessness from Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. These data are meant to train preference (or reward) models for subsequent RLHF training. These data are not meant for supervised training of dialogue agents. Training dialogue agents on these data is likely… See the full description on the dataset page: https://huggingface.co/datasets/kwakuobeng/hh-rlhf.mtmc-rlhf
mtmc-rlhf
The mtmc-rlhf (multi-task multi-capability rlhf) dataset is in Chinese and consists primarily of text prompts submitted to a large language model API, enriched by a small portion of prompts crafted by our annotators.
Each sample in the dataset represents a multi-turn session between a user and the language model with a category label. The final query within the session has several distinct responses as well as their corresponding preference rank sorted by annotators.
The… See the full description on the dataset page: https://huggingface.co/datasets/quanshr/mtmc-rlhf.korean_rlhf_content_filtered
Korean RLHF Content Filtered
Dataset Summary
This dataset is a cleaned, content-only derivative of:
Source dataset: jojo0217/korean_rlhf_dataset
Source URL: https://huggingface.co/datasets/jojo0217/korean_rlhf_dataset
Each row has a single content field suitable for LM pretraining/SFT-style text modeling.
Construction
Content construction rule
For each source row:
If input is empty: content = instruction + "\n" + output
If input is not empty:… See the full description on the dataset page: https://huggingface.co/datasets/alwaysgood/korean_rlhf_content_filtered.LLM-Preferences-HelpSteer2
LLM-Preferences-HelpSteer2
Author: Min Li
Blog: https://rlhflow.github.io/posts/2025-01-22-decision-tree-reward-model/
Dataset Description
This dataset contains pairwise preference judgments from 34 modern LLMs on response pairs from the HelpSteer2 dataset.
Key Features
Contains 9,125 response pairs from HelpSteer2-Preference
Includes preferences from 9 closed-source and 25 open-source LLMs
Documents position bias analysis and preference consistency metrics… See the full description on the dataset page: https://huggingface.co/datasets/RLHFlow/LLM-Preferences-HelpSteer2.hh-rlhf-sycophantic
hh-rlhf-sycophantic
A re-judged subset of Anthropic/hh-rlhf
(helpful-online config) where each preference pair has a syc_score ∈ [-4, +4]
indicating how much MORE sycophantic the chosen response is than the rejected
one, per an LLM judge with positional-bias debiasing.
DPO-training on syc_score > threshold pairs is a knob to amplify sycophancy
in a target model (reproducing Sharma et al. 2023, "Towards Understanding Sycophancy
in Language Models" §3.3 finding that PM
training data… See the full description on the dataset page: https://huggingface.co/datasets/wassname/hh-rlhf-sycophantic.hh-rlhf
Dataset Card for HH-RLHF Better Uncensored
Better Uncensored Summary
This is the Better Uncensored version of the famous Anthropic preference dataset Anthropic/hh-rlhf
Only the train files were processed with the rlhf uncensor script in this manner:
find ../hh-rlhf/ -type f -name 'train.jsonl' | xargs -I {} python uncensor_rlhf.py --in-file {}
This should work as a drop in replacement of the original dataset for training uncensored models. About 10% to 25% of the dataset… See the full description on the dataset page: https://huggingface.co/datasets/betteruncensored/hh-rlhf.Vietnamese-beyond-rlhf-reward-single-round-gg-translatedHH-RLHF-Cleaned-GRPO
HH-RLHF-Cleaned-GRPO
Deep-cleaned for GRPO/RL training | 1,847 examples | 187 bugs fixed
📋 Dataset Description
Safety and helpfulness prompts extracted from Anthropic's HH-RLHF dataset. This cleaned version removes 100 duplicate prompts and 87 whitespace issues that would waste training compute on redundant data.
Original source: Anthropic/hh-rlhf by Anthropic
📊 Cleaning Statistics
Metric
Value
Original examples
1,947
Clean… See the full description on the dataset page: https://huggingface.co/datasets/Eyght/HH-RLHF-Cleaned-GRPO.Arabic-preference-data-RLHFhh-rlhf-harmlessInternal copy of https://huggingface.co/datasets/Anthropic/hh-rlhf.