datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
helpful-instructions
Dataset Card for Helpful Instructions
Dataset Summary
Helpful Instructions is a dataset of (instruction, demonstration) pairs that are derived from public datasets. As the name suggests, it focuses on instructions that are "helpful", i.e. the kind of questions or tasks a human user might instruct an AI assistant to perform. You can load the dataset as follows:
from datasets import load_dataset
# Load all subsets
helpful_instructions =… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceH4/helpful-instructions.HH-RLHF-Helpful-standardWe process the helpful subset of Anthropic-HH into the standard format. The filtering script is as follows.
def filter_example(example):
if len(example['chosen']) != len(example['rejected']):
return False
if len(example['chosen']) % 2 != 0:
return False
n_rounds = len(example['chosen'])
for i in range(len(example['chosen'])):
if example['chosen'][i]['role'] != ['user', 'assistant'][i % 2]:
return False
if… See the full description on the dataset page: https://huggingface.co/datasets/RLHFlow/HH-RLHF-Helpful-standard.hh-rlhf-helpful-base-trl-style
TRL's Anthropic HH Dataset
We preprocess the dataset using our standard prompt, chosen, rejected format.
Reproduce this dataset
Download the anthropic_hh.py from the https://huggingface.co/datasets/trl-internal-testing/hh-rlhf-helpful-base-trl-style/tree/0.1.0.
Run python examples/datasets/anthropic_hh.py --push_to_hub --hf_entity trl-internal-testing
hh-rlhf-helpful-base
HH-RLHF-Helpful-Base Dataset
Summary
The HH-RLHF-Helpful-Base dataset is a processed version of Anthropic's HH-RLHF dataset, specifically curated to train models using the TRL library for preference learning and alignment tasks. It contains pairs of text samples, each labeled as either "chosen" or "rejected," based on human preferences regarding the helpfulness of the responses. This dataset enables models to learn human preferences in generating helpful responses… See the full description on the dataset page: https://huggingface.co/datasets/trl-lib/hh-rlhf-helpful-base.helpful_instructions_splitsThis splits the original helpful_instructions dataset into train and test splits.
chat-v2-anthropic-helpfulnesshelpful-anthropic-raw
Dataset Card for "helpful-raw-anthropic"
This is a dataset derived from Anthropic's HH-RLHF data of instructions and model-generated demonstrations. We combined training splits from the following two subsets:
helpful-base
helpful-online
To convert the multi-turn dialogues into (instruction, demonstration) pairs, just the first response from the Assistant was included. This heuristic captures the most obvious answers, but overlooks more complex questions where multiple turns were… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceH4/helpful-anthropic-raw.train_data_SFT_Helpful
HH-RLHF-Helpful-Base Dataset
Summary
The HH-RLHF-Helpful-Base dataset is a processed version of Anthropic's HH-RLHF dataset, specifically curated to train models using the TRL library for preference learning and alignment tasks. It contains pairs of text samples, each labeled as either "chosen" or "rejected," based on human preferences regarding the helpfulness of the responses. This dataset enables models to learn human preferences in generating helpful responses… See the full description on the dataset page: https://huggingface.co/datasets/Kyleyee/train_data_SFT_Helpful.rm_hh_helpful_only
Dataset Card for "rm_hh_helpful_only"
More Information needed
anthropic-helpful-harmless-rlhfhh-rlhf-helpful-base-trl-style
TRL's Anthropic HH Dataset
We preprocess the dataset using our standard prompt, chosen, rejected format.
Reproduce this dataset
Download the anthropic_hh.py from the https://huggingface.co/datasets/qgallouedec/hh-rlhf-helpful-base-trl-style/tree/0.1.0.
Run python examples/datasets/anthropic_hh.py --push_to_hub --hf_entity qgallouedec
rm_instruct_helpful_preferences
Dataset Card for "rm_instruct_helpful_preferences"
More Information needed
h4-anthropic-hh-rlhf-helpful-base-genbackdoored_helpful_only_completions_probe_type_linear_threshold_0_4backdoored_helpful_only_completions_probe_type_linear_threshold_0_65hh-rlhf-helpful-base-rollouts-gpt-oss-20b-diverse-openrouterultrafeedback-gpt-3.5-turbo-helpfulness
UltraFeedback GPT-3.5-Turbo Helpfulness Dataset
Summary
The UltraFeedback GPT-3.5-Turbo Helpfulness dataset contains processed user-assistant interactions filtered for helpfulness, derived from the openbmb/UltraFeedback dataset. It is designed for fine-tuning and evaluating models in alignment tasks.
Data Structure
Format: Conversational
Type: Unpaired preference
Column:
"pompt": The input question or instruction provided to the model.
"completion": The… See the full description on the dataset page: https://huggingface.co/datasets/trl-lib/ultrafeedback-gpt-3.5-turbo-helpfulness.RiC_harmless_helpfulThe hhrlhf dataset for RiC (https://huggingface.co/papers/2402.10207) training with harmless (R1) and helpful (R2) rewards.
The 'input_ids' are obtained from Llama2 tokenizer. If you want to use other base models, replace it using other tokenizers.
Note: the rewards are already normalized accroding to their corresponding mean and std. The mean and std data for R1 and R2 are saved into all_reward_stat_harmhelp_Rlarge.npy.
The mean and std for R1 and R2 is (-0.94732502, 1.92034349)… See the full description on the dataset page: https://huggingface.co/datasets/Ray2333/RiC_harmless_helpful.train_data_Helpful_implicit_prompt
HH-RLHF-Helpful-Base Dataset
Summary
The HH-RLHF-Helpful-Base dataset is a processed version of Anthropic's HH-RLHF dataset, specifically curated to train models using the TRL library for preference learning and alignment tasks. It contains pairs of text samples, each labeled as either "chosen" or "rejected," based on human preferences regarding the helpfulness of the responses. This dataset enables models to learn human preferences in generating helpful responses… See the full description on the dataset page: https://huggingface.co/datasets/Kyleyee/train_data_Helpful_implicit_prompt.backdoored_helpful_only_completions_probe_type_linear_threshold_0_45Anthropic-helpful-basetrain_data_Helpful_drdpo_7b_sft_1e
HH-RLHF-Helpful-Base Dataset
Summary
The HH-RLHF-Helpful-Base dataset is a processed version of Anthropic's HH-RLHF dataset, specifically curated to train models using the TRL library for preference learning and alignment tasks. It contains pairs of text samples, each labeled as either "chosen" or "rejected," based on human preferences regarding the helpfulness of the responses. This dataset enables models to learn human preferences in generating helpful responses… See the full description on the dataset page: https://huggingface.co/datasets/Kyleyee/train_data_Helpful_drdpo_7b_sft_1e.helpful-anthropic-raw
Dataset Card for "helpful-anthropic-raw"
More Information needed
backdoored_helpful_only_completions_probe_type_linear_threshold_0_7hh-rlhf-helpful-dpo-10k
HH-RLHF Helpful DPO Preference Pairs · 10k
10,000 real human preference pairs for teaching a tiny language model (≤50M params)
what a good assistant sounds like — more helpful, more natural, less evasive.
Why this dataset exists
This is the preference-tuning stage of an end-to-end tiny-model training pipeline:
Pretraining ──► SFT ──► DPO (this dataset) ──► Tiny Edge Assistant
After SFT teaches the model how to speak, this dataset… See the full description on the dataset page: https://huggingface.co/datasets/salisai/hh-rlhf-helpful-dpo-10k.helpful-self-instruct-raw
Dataset Card for "helpful-self-instruct-raw"
This dataset is derived from the finetuning subset of Self-Instruct, with some light formatting to remove trailing spaces and <|endoftext|> tokens.
Safe_dpo_helpfultrain_data_Helpful_drdpo_7b
HH-RLHF-Helpful-Base Dataset
Summary
The HH-RLHF-Helpful-Base dataset is a processed version of Anthropic's HH-RLHF dataset, specifically curated to train models using the TRL library for preference learning and alignment tasks. It contains pairs of text samples, each labeled as either "chosen" or "rejected," based on human preferences regarding the helpfulness of the responses. This dataset enables models to learn human preferences in generating helpful responses… See the full description on the dataset page: https://huggingface.co/datasets/Kyleyee/train_data_Helpful_drdpo_7b.ultrafeedback_helpful_baseHelpful