datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
HelpSteer2
HelpSteer2: Open-source dataset for training top-performing reward models
HelpSteer2 is an open-source Helpfulness Dataset (CC-BY-4.0) that supports aligning models to become more helpful, factually correct and coherent, while being adjustable in terms of the complexity and verbosity of its responses.
This dataset has been created in partnership with Scale AI.
When used to tune a Llama 3.1 70B Instruct Model, we achieve 94.1% on RewardBench, which makes it the best Reward Model as… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/HelpSteer2.HelpSteer3
HelpSteer3
HelpSteer3 is an open-source dataset (CC-BY-4.0) that supports aligning models to become more helpful in responding to user prompts.
HelpSteer3-Preference can be used to train Llama 3.3 Nemotron Super 49B v1 (for Generative RMs) and Llama 3.3 70B Instruct Models (for Bradley-Terry RMs) to produce Reward Models that score as high as 85.5% on RM-Bench and 78.6% on JudgeBench, which substantially surpass existing Reward Models on these benchmarks.
HelpSteer3-Feedback and… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/HelpSteer3.HelpSteer
HelpSteer: Helpfulness SteerLM Dataset
HelpSteer is an open-source Helpfulness Dataset (CC-BY-4.0) that supports aligning models to become more helpful, factually correct and coherent, while being adjustable in terms of the complexity and verbosity of its responses.
Leveraging this dataset and SteerLM, we train a Llama 2 70B to reach 7.54 on MT Bench, the highest among models trained on open-source datasets based on MT Bench Leaderboard as of 15 Nov 2023.
This model is available on… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/HelpSteer.helpsteer2_dpo_nonverboseHelperSteer 2, formatted in DPO format (prompt, chosen, rejected).
in main branch there is a custom scoring correct > helpful > -verbosity
in each branch we have preference pairs for only correct, helpful, verbosity, coherence, complexity
Please note that only correct and helpful has strong inter-rater agreement in the HelpSteer2 paper
This is the notebook used to produce the dataset… See the full description on the dataset page: https://huggingface.co/datasets/wassname/helpsteer2_dpo_nonverbose.Helpsteer-preference-standardhelpsteer3-codehelpsteer3-multilingualhelpsteer_hh_combined_pref
Dataset Card for "helpsteer_hh_combined_pref"
More Information needed
HelpSteer3-DPO-Llama-3.2-3BHelpSteer2-trl-stylehelpsteer2-categorized-prompts
HelpSteer2 Categorized Prompts
Dataset Summary
A curated collection of 540 instruction prompts derived from nvidia/HelpSteer2 and several complementary open datasets, enriched with category labels for use in instruction-tuning, benchmark evaluation, and prompt engineering research.
Prompts are clean plain text, ready for direct use in fine-tuning pipelines, benchmarks, and prompt engineering workflows.
Categories
Category
Count
Description
BASIC… See the full description on the dataset page: https://huggingface.co/datasets/atekrugis/helpsteer2-categorized-prompts.HelpSteer2-Preference-WarmStarthelpsteer2-preference-openai-native
HelpSteer2 Preference — OpenAI Native Format
A deterministic, training-ready repackaging of the preference split of
nvidia/HelpSteer2.
Why use this
What it is for. Preference optimisation — DPO, ORPO, SimPO, KTO — and reward
modelling, on 7,051 pairs that come from paid human annotators, not from an LLM
judge. Each pair carries a graded strength from 1 to 3 rather than a bare
binary label, so you can weight the loss by how strongly humans actually
disagreed, or… See the full description on the dataset page: https://huggingface.co/datasets/Archangel-system/helpsteer2-preference-openai-native.HelpSteer-AIF
HelpSteer: Helpfulness SteerLM Dataset
HelpSteer is an open-source Helpfulness Dataset (CC-BY-4.0) that supports aligning models to become more helpful, factually correct and coherent, while being adjustable in terms of the complexity and verbosity of its responses.
HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM
Disclaimer
This is only a subset created with distilabel to evaluate the first 1000 rows using AI Feedback (AIF) coming from GPT-4, only created for… See the full description on the dataset page: https://huggingface.co/datasets/alvarobartt/HelpSteer-AIF.HelpSteer2A reformatted version of nvidia/HelpSteer2 into both a multiturn config conversation and completion config config.
A v4 UUID doc_id is shared across the same document in each config, source, conversation, and completion.
Meta-Llama-3-8B-Instruct_ultrafeedback-annotate-judge-mtbench_cot_helpsteer_coherenceWeyaxi_HelpSteer-filtered-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
Weyaxi_HelpSteer-filtered-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT
Weyaxi/HelpSteer-filtered with responses regenerated with gemini-2.0-flash-thinking-exp-1219.
Generation Details
If BlockedPromptException, StopCandidateException, or InvalidArgument was returned, the sample was skipped.
If ["candidates"][0]["safety_ratings"] == "SAFETY" the sample was skipped.
If ["candidates"][0]["finish_reason"] != 1 the sample was skipped.
model = genai.GenerativeModel(… See the full description on the dataset page: https://huggingface.co/datasets/PJMixers-Dev/Weyaxi_HelpSteer-filtered-gemini-2.0-flash-thinking-exp-1219-CustomShareGPT.helpsteer3-qwen35_annotated_humanHelpSteer_prompt_per_row
Dataset Card for "HelpSteer_prompt_per_row"
More Information needed
HelpSteer-filtered
HelpSteer-filtered
This dataset is a highly filtered version of the nvidia/HelpSteer dataset.
❓ How this dataset was filtered:
I calculated the sum of the columns ["helpfulness," "correctness," "coherence," "complexity," "verbosity"] and created a new column named sum.
I changed some column names and added a empty column to match the Alpaca format.
The dataset was then filtered to include only those entries with a sum greater than or equal to 16.
🧐 More… See the full description on the dataset page: https://huggingface.co/datasets/Weyaxi/HelpSteer-filtered.ro_dpo_helpsteer2
Dataset Description
HelpSteer2 dataset contains 10k human-annotated preferences entries.
Here we provide the Romanian translation of the HelpSteer2 dataset, translated with GPT-4o mini.
This dataset is a next step of the alignment protocol for Romanian LLMs proposed in "Vorbeşti Româneşte?" A Recipe to Train Powerful Romanian LLMs with English Instructions (Masala et al., 2024).
Citation
@misc{wang2024helpsteer2preferencecomplementingratingspreferences… See the full description on the dataset page: https://huggingface.co/datasets/OpenLLM-Ro/ro_dpo_helpsteer2.helpsteer_combined_pref
Dataset Card for "helpsteer_combined_pref"
More Information needed
HelpSteer-AIF-raw
HelpSteer: Helpfulness SteerLM Dataset
HelpSteer is an open-source Helpfulness Dataset (CC-BY-4.0) that supports aligning models to become more helpful, factually correct and coherent, while being adjustable in terms of the complexity and verbosity of its responses.
HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM
Disclaimer
This is only a subset created with distilabel to evaluate the first 1000 rows using AI Feedback (AIF) coming from GPT-4, only created for… See the full description on the dataset page: https://huggingface.co/datasets/alvarobartt/HelpSteer-AIF-raw.HelpSteer2-preference-pairsLLM-Preferences-HelpSteer2
LLM-Preferences-HelpSteer2
Author: Min Li
Blog: https://rlhflow.github.io/posts/2025-01-22-decision-tree-reward-model/
Dataset Description
This dataset contains pairwise preference judgments from 34 modern LLMs on response pairs from the HelpSteer2 dataset.
Key Features
Contains 9,125 response pairs from HelpSteer2-Preference
Includes preferences from 9 closed-source and 25 open-source LLMs
Documents position bias analysis and preference consistency metrics… See the full description on the dataset page: https://huggingface.co/datasets/RLHFlow/LLM-Preferences-HelpSteer2.HelpSteer2_labeled_task
Dataset Card for HelpSteer2_labeled_task
This dataset has been created with distilabel.
Dataset Summary
This dataset contains a pipeline.yaml which can be used to reproduce the pipeline that generated it in distilabel using the distilabel CLI:
distilabel pipeline run --config "https://huggingface.co/datasets/CharlieJi/HelpSteer2_labeled_task/raw/main/pipeline.yaml"
or explore the configuration:
distilabel pipeline info --config… See the full description on the dataset page: https://huggingface.co/datasets/CharlieJi/HelpSteer2_labeled_task.HelpSteer3-en_prompt_en_thinking-filtered_correcthelpsteer2_alpaca-format_pml256helpsteer-coherence
Helpsteer-coherence
This dataset is derived from NVIDIA's HelpSteer dataset, processed specifically for preference learning on the coherence dimension.
- Train split: 22876 examples
- Test split: 1131 examples
## Format
Each example contains the following fields:
- `prompt`: Question with "Human:" prefix and "Assistant:" suffix
- `chosen`: The response with higher coherence score
- `rejected`: The response with lower coherence score
-… See the full description on the dataset page: https://huggingface.co/datasets/cheryyunl/helpsteer-coherence.helpsteer3-train-en_prompt_en_thinking