datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
hh-rlhf
Dataset Card for HH-RLHF
Dataset Summary
This repository provides access to two different kinds of data:
Human preference data about helpfulness and harmlessness from Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. These data are meant to train preference (or reward) models for subsequent RLHF training. These data are not meant for supervised training of dialogue agents. Training dialogue agents on these data is likely… See the full description on the dataset page: https://huggingface.co/datasets/Anthropic/hh-rlhf.prm800k_rlhf
Dataset Card for "prm800k_rlhf"
More Information needed
hh-rlhf-49k-ja-single-turnThis dataset was created by automatically translating part of "Anthropic/hh-rlhf" into Japanese, and selected for single turn conversations.You can use this dataset for RLHF and DPO.
hh-rlhf repository
https://github.com/anthropics/hh-rlhf
Anthropic/hh-rlhf
https://huggingface.co/datasets/Anthropic/hh-rlhf
rlhf-gemma3-indfood-1kmllm-rlhf-testingfull-hh-rlhf
Dataset Card for "full-hh-rlhf"
Anthropic's HH dataset reformatted into prompt, chosen, rejected samples.
wiki-linguarlhflow_mixture_with_math_del_systemrlhflow_mix_w_aimo_mathtldr-thematicHH-RLHF-Helpful-standardWe process the helpful subset of Anthropic-HH into the standard format. The filtering script is as follows.
def filter_example(example):
if len(example['chosen']) != len(example['rejected']):
return False
if len(example['chosen']) % 2 != 0:
return False
n_rounds = len(example['chosen'])
for i in range(len(example['chosen'])):
if example['chosen'][i]['role'] != ['user', 'assistant'][i % 2]:
return False
if… See the full description on the dataset page: https://huggingface.co/datasets/RLHFlow/HH-RLHF-Helpful-standard.RLHF-V-Dataset
Dataset Card for RLHF-V-Dataset
Project Page | Paper | GitHub
Updates
[2024.05.28] 📃 Our RLAIF-V paper is accesible at arxiv now!
[2024.05.20] 🎉 We release a new feedback dataset, RLAIF-V-Dataset, which is a large-scale diverse-task multimodal feedback dataset constructed using open-source models. You can download the corresponding dataset and models (7B, 12B) now!
[2024.04.11] 🔥 Our data is used in MiniCPM-V 2.0, an end-side multimodal large language model that… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/RLHF-V-Dataset.UltraFeedback-preference-standardWe include all the possible comparisons following the Instruct-GPT. We use the fine-grained_score.
import os
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from datasets import load_dataset, DatasetDict
from transformers import AutoTokenizer
from tqdm import tqdm
from transformers import AutoTokenizer
ds = load_dataset("openbmb/UltraFeedback", split="train")
import itertools
data = []
for example in ds:
prompt = example['instruction']
responses = {}… See the full description on the dataset page: https://huggingface.co/datasets/RLHFlow/UltraFeedback-preference-standard.hh-rlhf-helpful-base-trl-style
TRL's Anthropic HH Dataset
We preprocess the dataset using our standard prompt, chosen, rejected format.
Reproduce this dataset
Download the anthropic_hh.py from the https://huggingface.co/datasets/trl-internal-testing/hh-rlhf-helpful-base-trl-style/tree/0.1.0.
Run python examples/datasets/anthropic_hh.py --push_to_hub --hf_entity trl-internal-testing
rlhflow_mix_w_dart_mathrlhflow_mix_del_system_and_empty_round2MM-RLHF
[📖 arXiv Paper]
[📊 Training Code]
[📝 Homepage]
[🏆 Reward Model]
[🔮 MM-RewardBench]
[🔮 MM-SafetyBench]
[📈 Evaluation Suite]
The Next Step Forward in Multimodal LLM Alignment
[2025/02/10] 🔥 We are proud to open-source MM-RLHF, a comprehensive project for aligning Multimodal Large Language Models (MLLMs) with human preferences. This release includes:
A high-quality MLLM alignment dataset.
A strong Critique-Based MLLM reward model and its training algorithm.
A… See the full description on the dataset page: https://huggingface.co/datasets/yifanzhang114/MM-RLHF.hh_rlhf_cn_rm来源：https://huggingface.co/datasets/dikw/hh_rlhf_cn
把所有context字段的第一个对话与prompt中有重复的都过滤掉了，不过比原作者提到的2w多过滤了不少。
rlhflow_mix_del_system_and_empty_roundbilingual-rlhf-financial-semantics
🧠 RLHF Semantic Contribution Dataset: Financial & Policy Bilingual Corpus
Dataset name: sunwang4gptplus/bilingual-rlhf-financial-semanticsCreated by: Sun WangLanguages: English, Mandarin ChineseLicense: MITTags: RLHF, bilingual, semantic_contribution, ux_tone, human_feedback, financial_policy, openai_auditTasks: text-classification, text-generation, reinforcement-learning
📦 Dataset Overview
This dataset was created by Sun Wang as part of a multi-day Reinforcement… See the full description on the dataset page: https://huggingface.co/datasets/sunwang4gptplus/bilingual-rlhf-financial-semantics.social-reasoning-rlhf-ULTRAFEEDBACK-honesty
Dataset Card for "social-reasoning-rlhf-ULTRAFEEDBACK-honesty"
More Information needed
hh-rlhfThis dataset is part of the Anthropic's HH data used to train their RLHF Assistant https://github.com/anthropics/hh-rlhf.
The data contains the first utterance from human to the dialog agent and the number of words in that utterance. The sampled version is a random sample of size 200.
anthropic-hh-rlhf
Dataset Card for HH-RLHF
Dataset Summary
This repository provides access to two different kinds of data:
Human preference data about helpfulness and harmlessness from Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. These data are meant to train preference (or reward) models for subsequent RLHF training. These data are not meant for supervised training of dialogue agents. Training dialogue agents on these data is likely to lead… See the full description on the dataset page: https://huggingface.co/datasets/giovannioliveira/anthropic-hh-rlhf.eli5_rlhfELI5 paired
This is a processed version of the eli5 dataset. The dataset was created following very closely the steps in the stack-exchange-paired dataset. The following steps were applied:
Create pairs (response_j, response_k) where j was rated better than k
Sample at most 10 pairs per question
Shuffle the dataset globally
This dataset is designed to be used for preference learning using techniques such as Reinforcement Learning from Human Feedback. The processing notebook is in the… See the full description on the dataset page: https://huggingface.co/datasets/vincentmin/eli5_rlhf.hh-rlhf-helpful-base
HH-RLHF-Helpful-Base Dataset
Summary
The HH-RLHF-Helpful-Base dataset is a processed version of Anthropic's HH-RLHF dataset, specifically curated to train models using the TRL library for preference learning and alignment tasks. It contains pairs of text samples, each labeled as either "chosen" or "rejected," based on human preferences regarding the helpfulness of the responses. This dataset enables models to learn human preferences in generating helpful responses… See the full description on the dataset page: https://huggingface.co/datasets/trl-lib/hh-rlhf-helpful-base.oasst1_pairwise_rlhf_reward
Dataset Card for "oasst1_pairwise_rlhf_reward"
OASST1 dataset preprocessed for reward modeling:
import pandas as pd
from datasets import load_dataset,concatenate_datasets, Dataset, DatasetDict
import numpy as np
dataset = load_dataset("OpenAssistant/oasst1")
df=concatenate_datasets(list(dataset.values())).to_pandas()
m2t=df.set_index("message_id")['text'].to_dict()
m2r=df.set_index("message_id")['role'].to_dict()
m2p=df.set_index('message_id')['parent_id'].to_dict()… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/oasst1_pairwise_rlhf_reward.Mistral-PRM-DataSee https://github.com/RLHFlow/RLHF-Reward-Modeling/tree/main/math-rm for more data information.
anthropic-hh-rlhfNemotron-RLHF-GenRM-v1
Dataset Description:
This dataset is designed to train Generative Reward Models (GenRMs). It leverages reinforcement learning at scale to train accurate and robust GenRMs that generalize better than traditional Bradley-Terry models and reduce the risk of reward hacking.
The dataset is composed of:
Preference data focused on diverse domains
A synthetic safety blend
The data follows a "meta-prompt" structure where the model is instructed to act as an expert evaluation judge. For… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-RLHF-GenRM-v1.social-reasoning-rlhf
Dataset Summary
This repository provides access to a social reasoning dataset that aims to provide signal to how humans navigate social situations, how they reason about them and how they understand each other. It contains questions probing people's thinking and understanding of various social situations.
This dataset was created by collating a set of questions within the following social reasoning tasks:
understanding of emotions
intent recognition
social norms
social… See the full description on the dataset page: https://huggingface.co/datasets/ProlificAI/social-reasoning-rlhf.