datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
ultrafeedback-binarized-preferences-cleaned
UltraFeedback - Binarized using the Average of Preference Ratings (Cleaned)
This dataset represents a new iteration on top of argilla/ultrafeedback-binarized-preferences,
and is the recommended and preferred dataset by Argilla to use from now on when fine-tuning on UltraFeedback.
Read more about Argilla's approach towards UltraFeedback binarization at argilla/ultrafeedback-binarized-preferences/README.md.
Differences with argilla/ultrafeedback-binarized-preferences… See the full description on the dataset page: https://huggingface.co/datasets/argilla/ultrafeedback-binarized-preferences-cleaned.ultrafeedback_binarized
Dataset Card for UltraFeedback Binarized
Dataset Description
This is a pre-processed version of the UltraFeedback dataset and was used to train Zephyr-7Β-β, a state of the art chat model at the 7B parameter scale.
The original UltraFeedback dataset consists of 64k prompts, where each prompt is accompanied with four model completions from a wide variety of open and proprietary models. GPT-4 is then used to assign a score to each completion, along criteria like… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized.distilabel-capybara-dpo-7k-binarized
Capybara-DPO 7K binarized
A DPO dataset built with distilabel atop the awesome LDJnr/Capybara
This is a preview version to collect feedback from the community. v2 will include the full base dataset and responses from more powerful models.
Why?
Multi-turn dialogue data is key to fine-tune capable chat models. Multi-turn preference data has been used by the most relevant RLHF works (Anthropic, Meta Llama2, etc.). Unfortunately, there are very few… See the full description on the dataset page: https://huggingface.co/datasets/argilla/distilabel-capybara-dpo-7k-binarized.ultrafeedback-binarized-preferences-cleaned-kto
UltraFeedback - Binarized using the Average of Preference Ratings (Cleaned) KTO
A KTO signal transformed version of the highly loved UltraFeedback Binarized Preferences Cleaned, the preferred dataset by Argilla to use from now on when fine-tuning on UltraFeedback
This dataset represents a new iteration on top of argilla/ultrafeedback-binarized-preferences,
and is the recommended and preferred dataset by Argilla to use from now on when fine-tuning on UltraFeedback.
Read more about… See the full description on the dataset page: https://huggingface.co/datasets/argilla/ultrafeedback-binarized-preferences-cleaned-kto.ultrafeedback-multi-binarized-preferences-cleaned
UltraFeedback - Multi-Binarized using the Average of Preference Ratings (Cleaned)
This dataset represents a new iteration on top of argilla/ultrafeedback-binarized-preferences-cleaned,
and has been created to explore whether DPO fine-tuning with more than one rejection per chosen response helps the model perform better in the
AlpacaEval, MT-Bench, and LM Eval Harness benchmarks.
Read more about Argilla's approach towards UltraFeedback binarization at… See the full description on the dataset page: https://huggingface.co/datasets/argilla/ultrafeedback-multi-binarized-preferences-cleaned.llama3.2-3b-ultrafeedback-armorm-binarizedThis repository is associated with the paper Pre-DPO: Improving Data Utilization in Direct Preference Optimization Using a Guiding Reference Model.
Code: https://github.com/DtYXs/Pre-DPO
CodeUltraFeedback_binarizedInstructions coming soon
ultrafeedback_binarized_serbian
Dataset Card for UltraFeedback Binarized Serbian
Dataset Description
This dataset is a Serbian-translated version of the UltraFeedback dataset, utilized for training Zephyr-7Β-β. The original dataset comprises 64k English-language prompts, each paired with four completions from various models. In this Serbian version, the prompts and completions have been translated into Serbian. The dataset creation process remains the same: selecting the completion with the highest… See the full description on the dataset page: https://huggingface.co/datasets/datatab/ultrafeedback_binarized_serbian.distilabel-capybara-kto-15k-binarized
Capybara-KTO 15K binarized
A KTO signal transformed version of the highly loved Capybara-DPO 7K binarized, A DPO dataset built with distilabel atop the awesome LDJnr/Capybara
This is a preview version to collect feedback from the community. v2 will include the full base dataset and responses from more powerful models.
Why KTO?
The KTO paper states:
KTO matches or exceeds DPO performance at scales from 1B to 30B parameters.1 That is, taking a… See the full description on the dataset page: https://huggingface.co/datasets/argilla/distilabel-capybara-kto-15k-binarized.argilla-ultrafeedback-binarized-preferences-cleaned
UltraFeedback (Cleaned)
This dataset combines the train split of argilla/ultrafeedback-binarized-preferences-cleaned,
and test split of HuggingFaceH4/ultrafeedback_binarized.
ChatML-distilabel-capybara-dpo-7k-binarizedargilla/distilabel-capybara-dpo-7k-binarized in ChatML format, ready to use in HuggingFace TRL's DPO Trainer.
Python code used for conversion:
from datasets import load_dataset
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Felladrin/Llama-160M-Chat-v1")
dataset = load_dataset("argilla/distilabel-capybara-dpo-7k-binarized", split="train")
def format(columns):
return {
"prompt": tokenizer.apply_chat_template(columns["chosen"][:-1]… See the full description on the dataset page: https://huggingface.co/datasets/Felladrin/ChatML-distilabel-capybara-dpo-7k-binarized.ultrafeedback_binarized
Dataset Card for UltraFeedback Binarized
Dataset Description
This is a pre-processed version of the UltraFeedback dataset and was used to train Zephyr-7Β-β, a state of the art chat model at the 7B parameter scale.
The original UltraFeedback dataset consists of 64k prompts, where is prompt is accompanied with four model completions from a wide variety of open and proprietary models. GPT-4 is then used to assign a score to each completion, along criteria like helpfulness… See the full description on the dataset page: https://huggingface.co/datasets/zhengr/ultrafeedback_binarized.ultrafeedback-binarized-preferences-cleaned
UltraFeedback - Binarized using the Average of Preference Ratings (Cleaned)
This dataset represents a new iteration on top of argilla/ultrafeedback-binarized-preferences,
and is the recommended and preferred dataset by Argilla to use from now on when fine-tuning on UltraFeedback.
Read more about Argilla's approach towards UltraFeedback binarization at argilla/ultrafeedback-binarized-preferences/README.md.
Differences with argilla/ultrafeedback-binarized-preferences… See the full description on the dataset page: https://huggingface.co/datasets/pharaouk/ultrafeedback-binarized-preferences-cleaned.dpo-merged-binarizedultrafeedback_binarized
Dataset Card for UltraFeedback Binarized
Dataset Description
This is a pre-processed version of the UltraFeedback dataset and was used to train Zephyr-7Β-β, a state of the art chat model at the 7B parameter scale.
The original UltraFeedback dataset consists of 64k prompts, where each prompt is accompanied with four model completions from a wide variety of open and proprietary models. GPT-4 is then used to assign a score to each completion, along criteria like helpfulness… See the full description on the dataset page: https://huggingface.co/datasets/heqianwan/ultrafeedback_binarized.aya_dutch_dpo_binarized
Dataset Card for aya_dutch_dpo
This dataset has been created with distilabel.
This dataset was created as part of the Data is Better Together project, in particular as part of an ongoing effort to help foster the creation of DPO/ORPO datasets for more languages.
The dataset was constructed using the following steps:
starting with the aya_dataset and filtering for Dutch examples
using the Meta-Llama-3-70B-Instruct model to generate new examples for each promptUsing… See the full description on the dataset page: https://huggingface.co/datasets/CultriX/aya_dutch_dpo_binarized.qwen2.5-7b-ultrafeedback-armorm-binarizedThis repository contains the data for the paper Pre-DPO: Improving Data Utilization in Direct Preference Optimization Using a Guiding Reference Model.
Github: https://github.com/DtYXs/Pre-DPO
ultrafeedback-binarized-preferences-cleaned-no-refusals
UltraFeedback Binarized Preferences Cleaned No Refusals
A Minos-cleaned version of argilla/ultrafeedback-binarized-preferences-cleaned for use as a neutral helpfulness DPO anchor. Rows are removed when either the chosen or rejected assistant response is classified as a refusal by NousResearch/Minos-v1.
Cleaning version: minos-only-v1-2026-06-23
See manifest.json in the repository files for counts and endpoint metadata.
dpo-chatml-mix-binarizedultrafeedback-binarized-preferences-cleaned-multilingual
Dataset Card for ultrafeedback-binarized-preferences-cleaned-multilingual
本資料集是 argilla/ultrafeedback-binarized-preferences-cleaned 的多語言（含繁體中文 zh-tw）版本，每筆樣本保留原始來源、語言、對話與 chosen / rejected 回應，可作為繁中模型的 DPO 對齊資料。
Dataset Details
Dataset Description
原始 UltraFeedback 是一個英文偏好資料集，本資料集將其翻譯／增廣為多語版本，並對譯文做品質清理，以利非英文（特別是繁中）DPO 訓練。
每筆樣本包含：
source：原始來源（如 evol_instruct、flan_v2_p3 等）。
lang：語言代碼（如 zh-tw、en）。
conversations：human／gpt 對話結構，內容已翻譯為對應語言。
（以及對應的 chosen / rejected… See the full description on the dataset page: https://huggingface.co/datasets/lianghsun/ultrafeedback-binarized-preferences-cleaned-multilingual.mirror-CodeUltraFeedback_binarizedInstructions coming soon
vietnamese_ultrafeedback_binarized