datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
helpful-instructions
Dataset Card for Helpful Instructions
Dataset Summary
Helpful Instructions is a dataset of (instruction, demonstration) pairs that are derived from public datasets. As the name suggests, it focuses on instructions that are "helpful", i.e. the kind of questions or tasks a human user might instruct an AI assistant to perform. You can load the dataset as follows:
from datasets import load_dataset
# Load all subsets
helpful_instructions =… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceH4/helpful-instructions.HH-RLHF-Helpful-standardWe process the helpful subset of Anthropic-HH into the standard format. The filtering script is as follows.
def filter_example(example):
if len(example['chosen']) != len(example['rejected']):
return False
if len(example['chosen']) % 2 != 0:
return False
n_rounds = len(example['chosen'])
for i in range(len(example['chosen'])):
if example['chosen'][i]['role'] != ['user', 'assistant'][i % 2]:
return False
if… See the full description on the dataset page: https://huggingface.co/datasets/RLHFlow/HH-RLHF-Helpful-standard.hh-rlhf-helpful-base-trl-style
TRL's Anthropic HH Dataset
We preprocess the dataset using our standard prompt, chosen, rejected format.
Reproduce this dataset
Download the anthropic_hh.py from the https://huggingface.co/datasets/trl-internal-testing/hh-rlhf-helpful-base-trl-style/tree/0.1.0.
Run python examples/datasets/anthropic_hh.py --push_to_hub --hf_entity trl-internal-testing
hh-rlhf-helpful-base
HH-RLHF-Helpful-Base Dataset
Summary
The HH-RLHF-Helpful-Base dataset is a processed version of Anthropic's HH-RLHF dataset, specifically curated to train models using the TRL library for preference learning and alignment tasks. It contains pairs of text samples, each labeled as either "chosen" or "rejected," based on human preferences regarding the helpfulness of the responses. This dataset enables models to learn human preferences in generating helpful responses… See the full description on the dataset page: https://huggingface.co/datasets/trl-lib/hh-rlhf-helpful-base.helpsteer2_dpo_nonverboseHelperSteer 2, formatted in DPO format (prompt, chosen, rejected).
in main branch there is a custom scoring correct > helpful > -verbosity
in each branch we have preference pairs for only correct, helpful, verbosity, coherence, complexity
Please note that only correct and helpful has strong inter-rater agreement in the HelpSteer2 paper
This is the notebook used to produce the dataset… See the full description on the dataset page: https://huggingface.co/datasets/wassname/helpsteer2_dpo_nonverbose.Helpsteer-preference-standardhelpsteer3-codehelpsteer3-multilingualhelpful_instructions_splitsThis splits the original helpful_instructions dataset into train and test splits.
chat-v2-anthropic-helpfulnessdemo3_frames_grab3This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "sam_two",
"total_episodes": 1,
"total_frames": 464,
"total_tasks": 1,
"total_videos": 3,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:1"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/helper2424/demo3_frames_grab3.helpful-anthropic-raw
Dataset Card for "helpful-raw-anthropic"
This is a dataset derived from Anthropic's HH-RLHF data of instructions and model-generated demonstrations. We combined training splits from the following two subsets:
helpful-base
helpful-online
To convert the multi-turn dialogues into (instruction, demonstration) pairs, just the first response from the Assistant was included. This heuristic captures the most obvious answers, but overlooks more complex questions where multiple turns were… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceH4/helpful-anthropic-raw.train_data_SFT_Helpful
HH-RLHF-Helpful-Base Dataset
Summary
The HH-RLHF-Helpful-Base dataset is a processed version of Anthropic's HH-RLHF dataset, specifically curated to train models using the TRL library for preference learning and alignment tasks. It contains pairs of text samples, each labeled as either "chosen" or "rejected," based on human preferences regarding the helpfulness of the responses. This dataset enables models to learn human preferences in generating helpful responses… See the full description on the dataset page: https://huggingface.co/datasets/Kyleyee/train_data_SFT_Helpful.helpsteer_hh_combined_pref
Dataset Card for "helpsteer_hh_combined_pref"
More Information needed
HelpSteer3-DPO-Llama-3.2-3Brm_hh_helpful_only
Dataset Card for "rm_hh_helpful_only"
More Information needed
anthropic-helpful-harmless-rlhfhh-rlhf-helpful-base-trl-style
TRL's Anthropic HH Dataset
We preprocess the dataset using our standard prompt, chosen, rejected format.
Reproduce this dataset
Download the anthropic_hh.py from the https://huggingface.co/datasets/qgallouedec/hh-rlhf-helpful-base-trl-style/tree/0.1.0.
Run python examples/datasets/anthropic_hh.py --push_to_hub --hf_entity qgallouedec
HelpSteer2-trl-stylehelpsteer2-categorized-prompts
HelpSteer2 Categorized Prompts
Dataset Summary
A curated collection of 540 instruction prompts derived from nvidia/HelpSteer2 and several complementary open datasets, enriched with category labels for use in instruction-tuning, benchmark evaluation, and prompt engineering research.
Prompts are clean plain text, ready for direct use in fine-tuning pipelines, benchmarks, and prompt engineering workflows.
Categories
Category
Count
Description
BASIC… See the full description on the dataset page: https://huggingface.co/datasets/atekrugis/helpsteer2-categorized-prompts.rm_instruct_helpful_preferences
Dataset Card for "rm_instruct_helpful_preferences"
More Information needed
h4-anthropic-hh-rlhf-helpful-base-genbackdoored_helpful_only_completions_probe_type_linear_threshold_0_4backdoored_helpful_only_completions_probe_type_linear_threshold_0_65hh-rlhf-helpful-base-rollouts-gpt-oss-20b-diverse-openrouterhelpsteer2-preference-openai-native
HelpSteer2 Preference — OpenAI Native Format
A deterministic, training-ready repackaging of the preference split of
nvidia/HelpSteer2.
Why use this
What it is for. Preference optimisation — DPO, ORPO, SimPO, KTO — and reward
modelling, on 7,051 pairs that come from paid human annotators, not from an LLM
judge. Each pair carries a graded strength from 1 to 3 rather than a bare
binary label, so you can weight the loss by how strongly humans actually
disagreed, or… See the full description on the dataset page: https://huggingface.co/datasets/Archangel-system/helpsteer2-preference-openai-native.ultrafeedback-gpt-3.5-turbo-helpfulness
UltraFeedback GPT-3.5-Turbo Helpfulness Dataset
Summary
The UltraFeedback GPT-3.5-Turbo Helpfulness dataset contains processed user-assistant interactions filtered for helpfulness, derived from the openbmb/UltraFeedback dataset. It is designed for fine-tuning and evaluating models in alignment tasks.
Data Structure
Format: Conversational
Type: Unpaired preference
Column:
"pompt": The input question or instruction provided to the model.
"completion": The… See the full description on the dataset page: https://huggingface.co/datasets/trl-lib/ultrafeedback-gpt-3.5-turbo-helpfulness.RiC_harmless_helpfulThe hhrlhf dataset for RiC (https://huggingface.co/papers/2402.10207) training with harmless (R1) and helpful (R2) rewards.
The 'input_ids' are obtained from Llama2 tokenizer. If you want to use other base models, replace it using other tokenizers.
Note: the rewards are already normalized accroding to their corresponding mean and std. The mean and std data for R1 and R2 are saved into all_reward_stat_harmhelp_Rlarge.npy.
The mean and std for R1 and R2 is (-0.94732502, 1.92034349)… See the full description on the dataset page: https://huggingface.co/datasets/Ray2333/RiC_harmless_helpful.HelpSteer-AIF
HelpSteer: Helpfulness SteerLM Dataset
HelpSteer is an open-source Helpfulness Dataset (CC-BY-4.0) that supports aligning models to become more helpful, factually correct and coherent, while being adjustable in terms of the complexity and verbosity of its responses.
HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM
Disclaimer
This is only a subset created with distilabel to evaluate the first 1000 rows using AI Feedback (AIF) coming from GPT-4, only created for… See the full description on the dataset page: https://huggingface.co/datasets/alvarobartt/HelpSteer-AIF.HelpSteer2A reformatted version of nvidia/HelpSteer2 into both a multiturn config conversation and completion config config.
A v4 UUID doc_id is shared across the same document in each config, source, conversation, and completion.