datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
helpful-instructions
Dataset Card for Helpful Instructions
Dataset Summary
Helpful Instructions is a dataset of (instruction, demonstration) pairs that are derived from public datasets. As the name suggests, it focuses on instructions that are "helpful", i.e. the kind of questions or tasks a human user might instruct an AI assistant to perform. You can load the dataset as follows:
from datasets import load_dataset
# Load all subsets
helpful_instructions =… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceH4/helpful-instructions.HH-RLHF-Helpful-standardWe process the helpful subset of Anthropic-HH into the standard format. The filtering script is as follows.
def filter_example(example):
if len(example['chosen']) != len(example['rejected']):
return False
if len(example['chosen']) % 2 != 0:
return False
n_rounds = len(example['chosen'])
for i in range(len(example['chosen'])):
if example['chosen'][i]['role'] != ['user', 'assistant'][i % 2]:
return False
if… See the full description on the dataset page: https://huggingface.co/datasets/RLHFlow/HH-RLHF-Helpful-standard.hh-rlhf-helpful-base-trl-style
TRL's Anthropic HH Dataset
We preprocess the dataset using our standard prompt, chosen, rejected format.
Reproduce this dataset
Download the anthropic_hh.py from the https://huggingface.co/datasets/trl-internal-testing/hh-rlhf-helpful-base-trl-style/tree/0.1.0.
Run python examples/datasets/anthropic_hh.py --push_to_hub --hf_entity trl-internal-testing
helpful_instructionsHelpful Instructions is a dataset of (prompt, completion) pairs that are derived from a variety of public datasets. As the name suggests, it focuses on instructions that are "helpful", i.e. the kind of questions or tasks a human user might instruct an AI assistant to perform.hh-rlhf-helpful-base
HH-RLHF-Helpful-Base Dataset
Summary
The HH-RLHF-Helpful-Base dataset is a processed version of Anthropic's HH-RLHF dataset, specifically curated to train models using the TRL library for preference learning and alignment tasks. It contains pairs of text samples, each labeled as either "chosen" or "rejected," based on human preferences regarding the helpfulness of the responses. This dataset enables models to learn human preferences in generating helpful responses… See the full description on the dataset page: https://huggingface.co/datasets/trl-lib/hh-rlhf-helpful-base.helpful_instructions_splitsThis splits the original helpful_instructions dataset into train and test splits.
rlhf_helpful_evalchat-v2-anthropic-helpfulnesshelpful-anthropic-raw
Dataset Card for "helpful-raw-anthropic"
This is a dataset derived from Anthropic's HH-RLHF data of instructions and model-generated demonstrations. We combined training splits from the following two subsets:
helpful-base
helpful-online
To convert the multi-turn dialogues into (instruction, demonstration) pairs, just the first response from the Assistant was included. This heuristic captures the most obvious answers, but overlooks more complex questions where multiple turns were… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceH4/helpful-anthropic-raw.review_helpfulness_prediction
Dataset Card for Review Helpfulness Prediction (RHP) Dataset
Dataset Summary
The success of e-commerce services is largely dependent on helpful reviews that aid customers in making informed purchasing decisions. However, some reviews may be spammy or biased, making it challenging to identify which ones are helpful. Current methods for identifying helpful reviews only focus on the review text, ignoring the importance of who posted the review and when it was posted.… See the full description on the dataset page: https://huggingface.co/datasets/tafseer-nayeem/review_helpfulness_prediction.train_data_SFT_Helpful
HH-RLHF-Helpful-Base Dataset
Summary
The HH-RLHF-Helpful-Base dataset is a processed version of Anthropic's HH-RLHF dataset, specifically curated to train models using the TRL library for preference learning and alignment tasks. It contains pairs of text samples, each labeled as either "chosen" or "rejected," based on human preferences regarding the helpfulness of the responses. This dataset enables models to learn human preferences in generating helpful responses… See the full description on the dataset page: https://huggingface.co/datasets/Kyleyee/train_data_SFT_Helpful.anthropic-helpful-harmless-rlhfrm_hh_helpful_only
Dataset Card for "rm_hh_helpful_only"
More Information needed
hh-rlhf-helpful-base-trl-style
TRL's Anthropic HH Dataset
We preprocess the dataset using our standard prompt, chosen, rejected format.
Reproduce this dataset
Download the anthropic_hh.py from the https://huggingface.co/datasets/qgallouedec/hh-rlhf-helpful-base-trl-style/tree/0.1.0.
Run python examples/datasets/anthropic_hh.py --push_to_hub --hf_entity qgallouedec
Meta-Llama-3-8B-Instruct_ultrafeedback-annotate-judge-mtbench_cot_helpsteer_helpfulnessrm_instruct_helpful_preferences
Dataset Card for "rm_instruct_helpful_preferences"
More Information needed
h4-anthropic-hh-rlhf-helpful-base-genChinese_Preference_Safe_and_Helpful
数据集
简介
我们参考微调LLama2的方式构建中文数据集。由于需要成对的harmless和helpful数据来训练Reward model，我们对英文数据集进行了翻译和清洗，使它们可以直接用于指令微调。
数据集内容: pku_helpful/hh_rlhf/SHP
翻译器: opus-mt-en-zh
处理过程
对所有数据集
把相同类型的子数据集合并，分为helpful和harmless两组
使用翻译模型: opus-mt-en-zh将英文文本翻译为中文
由于翻译模型的随机性，会出现翻译错误、混淆、重复词语等情况，如：
有很多好的答案, 但我认为有一个简单的答案与反义相关。 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之, 反之,...
将这类重复出现词语的情况进行清洗，得到：… See the full description on the dataset page: https://huggingface.co/datasets/DirectLLM/Chinese_Preference_Safe_and_Helpful.backdoored_helpful_only_completions_probe_type_linear_threshold_0_4backdoored_helpful_only_completions_probe_type_linear_threshold_0_65hh-rlhf-helpful-base-rollouts-gpt-oss-20b-diverse-openrouterRiC_harmless_helpfulThe hhrlhf dataset for RiC (https://huggingface.co/papers/2402.10207) training with harmless (R1) and helpful (R2) rewards.
The 'input_ids' are obtained from Llama2 tokenizer. If you want to use other base models, replace it using other tokenizers.
Note: the rewards are already normalized accroding to their corresponding mean and std. The mean and std data for R1 and R2 are saved into all_reward_stat_harmhelp_Rlarge.npy.
The mean and std for R1 and R2 is (-0.94732502, 1.92034349)… See the full description on the dataset page: https://huggingface.co/datasets/Ray2333/RiC_harmless_helpful.ultrafeedback-gpt-3.5-turbo-helpfulness
UltraFeedback GPT-3.5-Turbo Helpfulness Dataset
Summary
The UltraFeedback GPT-3.5-Turbo Helpfulness dataset contains processed user-assistant interactions filtered for helpfulness, derived from the openbmb/UltraFeedback dataset. It is designed for fine-tuning and evaluating models in alignment tasks.
Data Structure
Format: Conversational
Type: Unpaired preference
Column:
"pompt": The input question or instruction provided to the model.
"completion": The… See the full description on the dataset page: https://huggingface.co/datasets/trl-lib/ultrafeedback-gpt-3.5-turbo-helpfulness.train_data_Helpful_implicit_prompt
HH-RLHF-Helpful-Base Dataset
Summary
The HH-RLHF-Helpful-Base dataset is a processed version of Anthropic's HH-RLHF dataset, specifically curated to train models using the TRL library for preference learning and alignment tasks. It contains pairs of text samples, each labeled as either "chosen" or "rejected," based on human preferences regarding the helpfulness of the responses. This dataset enables models to learn human preferences in generating helpful responses… See the full description on the dataset page: https://huggingface.co/datasets/Kyleyee/train_data_Helpful_implicit_prompt.backdoored_helpful_only_completions_probe_type_linear_threshold_0_45Anthropic-helpful-basetrain_data_Helpful_drdpo_7b_sft_1e
HH-RLHF-Helpful-Base Dataset
Summary
The HH-RLHF-Helpful-Base dataset is a processed version of Anthropic's HH-RLHF dataset, specifically curated to train models using the TRL library for preference learning and alignment tasks. It contains pairs of text samples, each labeled as either "chosen" or "rejected," based on human preferences regarding the helpfulness of the responses. This dataset enables models to learn human preferences in generating helpful responses… See the full description on the dataset page: https://huggingface.co/datasets/Kyleyee/train_data_Helpful_drdpo_7b_sft_1e.hh-rlhf-helpful-processedhh-rlhf-helpful-dpo-10k
HH-RLHF Helpful DPO Preference Pairs · 10k
10,000 real human preference pairs for teaching a tiny language model (≤50M params)
what a good assistant sounds like — more helpful, more natural, less evasive.
Why this dataset exists
This is the preference-tuning stage of an end-to-end tiny-model training pipeline:
Pretraining ──► SFT ──► DPO (this dataset) ──► Tiny Edge Assistant
After SFT teaches the model how to speak, this dataset… See the full description on the dataset page: https://huggingface.co/datasets/salisai/hh-rlhf-helpful-dpo-10k.backdoored_helpful_only_completions_probe_type_linear_threshold_0_7