datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
agent-reward-bench
AgentRewardBench
💾Code
📄Paper
🌐Website
🤗Dataset
💻Demo
🏆Leaderboard
AgentRewardBench: Evaluating Automatic Evaluations of Web Agent TrajectoriesXing Han Lù, Amirhossein Kazemnejad*, Nicholas Meade, Arkil Patel, Dongchan Shin, Alejandra Zambrano, Karolina Stańczak, Peter Shaw, Christopher J. Pal, Siva Reddy*Core Contributor
Loading dataset
You can use the huggingface_hub library to load the dataset. The dataset is available on Huggingface Hub at… See the full description on the dataset page: https://huggingface.co/datasets/McGill-NLP/agent-reward-bench.reward-bench
Code | Leaderboard | Prior Preference Sets | Results | Paper
Reward Bench Evaluation Dataset Card
The RewardBench evaluation dataset evaluates capabilities of reward models over the following categories:
Chat: Includes the easy chat subsets (alpacaeval-easy, alpacaeval-length, alpacaeval-hard, mt-bench-easy, mt-bench-medium)
Chat Hard: Includes the hard chat subsets (mt-bench-hard, llmbar-natural, llmbar-adver-neighbor, llmbar-adver-GPTInst, llmbar-adver-GPTOut… See the full description on the dataset page: https://huggingface.co/datasets/allenai/reward-bench.rlvr-reward-hacking-scale-no-conftest-20260909-completion
Matched no-conftest RLVR study 20260909-completion
Lossless research records, grouped by model and trajectory type. Only the listed
configurations have published records. Canary diagnostics are excluded from study
estimates; run status in provenance distinguishes retired diagnostics from active
or completed training. Valid failures, refusals and truncations are retained.
The train split name is a dataset-loader convention; record_type identifies
whether a record is training… See the full description on the dataset page: https://huggingface.co/datasets/lucabaroni/rlvr-reward-hacking-scale-no-conftest-20260909-completion.reward-bench-2Code | Leaderboard | Results | Paper
RewardBench 2 Evaluation Dataset Card
The RewardBench 2 evaluation dataset is the new version of RewardBench that is based on unseen human data and designed to be substantially more difficult! RewardBench 2 evaluates capabilities of reward models over the following categories:
Factuality (NEW!): Tests the ability of RMs to detect hallucinations and other basic errors in completions.
Precise Instruction Following (NEW!): Tests the ability of RMs… See the full description on the dataset page: https://huggingface.co/datasets/allenai/reward-bench-2.rewardbenchSkywork-Reward-Preference-80K-v0.2
Skywork Reward Preference 80K
IMPORTANT:
This dataset is the decontaminated version of Skywork-Reward-Preference-80K-v0.1. We removed 4,957 pairs from the magpie-ultra-v0.1 subset that have a significant n-gram overlap with the evaluation prompts in RewardBench. You can find the set of removed pairs here. For more information, see this GitHub gist.
If your task involves evaluation on RewardBench, we strongly encourage you to use v0.2 instead of v0.1 of the dataset.
We will soon… See the full description on the dataset page: https://huggingface.co/datasets/Skywork/Skywork-Reward-Preference-80K-v0.2.sparse-reward-long-tasks
Sparse Reward Long Tasks
Rights & intended use: legacy public research corpus / portfolio
artifact. Hosted frontier-model outputs are research-only inputs under
project policy (synthetic-factory#161):
intended_use: research_only, project_training_policy: blocked. Not
training data for any model-weight update. Machine-readable record:
rights.json.
Release status: The raw, uncurated payload is now published under
data/raw/. It is available for inspection and reproducibility… See the full description on the dataset page: https://huggingface.co/datasets/rmems/sparse-reward-long-tasks.multilingual-reward-bench
Multilingual Reward Bench (v1.0)
Reward models (RMs) have driven the development of state-of-the-art LLMs today, with unprecedented impact across the globe. However, their performance in multilingual settings still remains understudied.
In order to probe reward model behavior on multilingual data, we present M-RewardBench, a benchmark for 23 typologically diverse languages.
M-RewardBench contains prompt-chosen-rejected preference triples obtained by curating and translating chat… See the full description on the dataset page: https://huggingface.co/datasets/CohereLabsCommunity/multilingual-reward-bench.school-of-reward-hacksThis repository contains the dataset for School of Reward Hacks: Hacking Harmless Tasks Generalizes to Misaligned Behavior in LLMs. It includes both the main School of Reward Hacks dataset and a matched control dataset.
Field Descriptions:
user: The user message, which introduces the task and evaluation method.
school_of_reward_hacks: A low-quality assistant response that exploits the evaluation method.
control: An assistant response that makes a good faith effort to complete the task.… See the full description on the dataset page: https://huggingface.co/datasets/longtermrisk/school-of-reward-hacks.aeslides-reward-bench
AeSlides-Reward-Bench
This dataset is part of the work presented in the paper AeSlides: Incentivizing Aesthetic Layout in LLM-Based Slide Generation via Verifiable Rewards.
AeSlides is a reinforcement learning framework with verifiable rewards for aesthetic layout supervision in slide generation. This benchmark focuses on quantifying slide layout quality through verifiable metrics like aspect ratio compliance, whitespace reduction, and visual balance.
GitHub Repository:… See the full description on the dataset page: https://huggingface.co/datasets/ympan/aeslides-reward-bench.rlvr-reward-hacking-scale-no-conftest-20260909
Matched no-conftest RLVR study 20260909
Complete immutable training, monitoring and comparison trajectories for six models.
All valid outcomes are retained, including refusals, failures and truncations.
The train split name is a dataset-loader convention; record_type identifies
whether a record is training, monitoring, comparison, or a derived judgment.
import json
from datasets import load_dataset
rows = load_dataset("lucabaroni/rlvr-reward-hacking-scale-no-conftest-20260909"… See the full description on the dataset page: https://huggingface.co/datasets/lucabaroni/rlvr-reward-hacking-scale-no-conftest-20260909.agent-reward-bench
AgentRewardBench
💾Code
📄Paper
🌐Website
🤗Dataset
💻Demo
🏆Leaderboard
AgentRewardBench: Evaluating Automatic Evaluations of Web Agent TrajectoriesXing Han Lù, Amirhossein Kazemnejad*, Nicholas Meade, Arkil Patel, Dongchan Shin, Alejandra Zambrano, Karolina Stańczak, Peter Shaw, Christopher J. Pal, Siva Reddy*Core Contributor
Loading dataset
You can use the huggingface_hub library to load the dataset. The dataset is available on Huggingface Hub at… See the full description on the dataset page: https://huggingface.co/datasets/yuanyyaa/agent-reward-bench.Pose_Reward_DPOVideoGen-RewardBench
🏆 [VideoGen-RewardBench Leaderboard]
Introduction
VideoGen-RewardBench is a comprehensive benchmark designed to evaluate the performance of video reward models on modern text-to-video (T2V) systems. Derived from the third-party VideoGen-Eval (Zeng et.al, 2024), we constructing 26.5k (prompt, Video A, Video B) triplets and employing expert annotators to provide pairwise preference labels.
These annotations are based on key evaluation dimensions—Visual Quality (VQ), Motion… See the full description on the dataset page: https://huggingface.co/datasets/KlingTeam/VideoGen-RewardBench.AceMath-RewardBenchwebsite | paper
AceMath-RewardBench Evaluation Dataset Card
The AceMath-RewardBench evaluation dataset evaluates capabilities of a math reward model using the best-of-N (N=8) setting for 7 datasets:
GSM8K: 1319 questions
Math500: 500 questions
Minerva Math: 272 questions
Gaokao 2023 en: 385 questions
OlympiadBench: 675 questions
College Math: 2818 questions
MMLU STEM: 3018 questions
Each example in the dataset contains:
A mathematical question
64 solution attempts with varying… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/AceMath-RewardBench.VL-RewardBench
Dataset Card for VLRewardBench
Project Page:
https://vl-rewardbench.github.io
Dataset Summary
VLRewardBench is a comprehensive benchmark designed to evaluate vision-language generative reward models (VL-GenRMs) across visual perception, hallucination detection, and reasoning tasks. The benchmark contains 1,250 high-quality examples specifically curated to probe model limitations.
Dataset Structure
Each instance consists of multimodal queries spanning three key… See the full description on the dataset page: https://huggingface.co/datasets/MMInstruction/VL-RewardBench.MultiGen_Reward_DPO_condRAG-RewardBenchThis repository contains the data presented in RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment.
Code: https://github.com/jinzhuoran/RAG-RewardBench/
EditScore-Reward-Data
Introduction
Training data for EditScore.
Usage
# meta file: reward.json
# images:
cat images_part_* > images.tar.gz && tar -xzvf images.tar.gz
Citation
@article{luo2025editscore,
title={EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling},
author={Xin Luo and Jiahao Wang and Chenyuan Wu and Shitao Xiao and Xiyan Jiang and Defu Lian and Jiajun Zhang and Dong Liu and Zheng Liu},
journal={arXiv… See the full description on the dataset page: https://huggingface.co/datasets/EditScore/EditScore-Reward-Data.Pose_Reward_DPO_condverified-sql-rewards
Verified SQL Rewards
A text-to-SQL corpus where every reward carries a machine-checkable proof
that it is correct.
Questions, all independently verified
109,306
Databases
1,400 across 7 schema families
Tables / data rows
4,400 / ~19.6 million
Unique (question, answer) pairs
102,764
Candidates refused and published
12,150
Verification pass rate
90.00%
Trivial baseline (always answer 0)
1.83%
Each item is a natural-language question, a gold SQL query… See the full description on the dataset page: https://huggingface.co/datasets/rasinmuhammed/verified-sql-rewards.reward-hacking-olmo3.1-32b-kl0.02-seed2-rollouts
Reward-Hacking Training Rollouts — OLMo-3.1-32B (β=0.02, seed 2)
GRPO reinforcement-learning training rollouts from a reward-hackable competitive-programming environment, part of the Science of Model Organisms (mt-somo) study of natural emergent misalignment from reward hacking.
Companion to the checkpoint repo ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.02-seed2. With a small KL penalty (β=0.02) the policy stays closer to the base model, yet it still learns to exploit… See the full description on the dataset page: https://huggingface.co/datasets/ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.02-seed2-rollouts.processed_vision_rewardoasst1_pairwise_rlhf_reward
Dataset Card for "oasst1_pairwise_rlhf_reward"
OASST1 dataset preprocessed for reward modeling:
import pandas as pd
from datasets import load_dataset,concatenate_datasets, Dataset, DatasetDict
import numpy as np
dataset = load_dataset("OpenAssistant/oasst1")
df=concatenate_datasets(list(dataset.values())).to_pandas()
m2t=df.set_index("message_id")['text'].to_dict()
m2r=df.set_index("message_id")['role'].to_dict()
m2p=df.set_index('message_id')['parent_id'].to_dict()… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/oasst1_pairwise_rlhf_reward.reward_data_v1reward-hacking-olmo3.1-32b-kl0.0-seed2-rollouts
Reward-Hacking Training Rollouts — OLMo-3.1-32B (β=0.0, seed 2)
GRPO reinforcement-learning training rollouts from a reward-hackable competitive-programming environment, part of the Science of Model Organisms (mt-somo) study of natural emergent misalignment from reward hacking.
Companion to the checkpoint repo ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.0-seed2. With no KL penalty (β=0) the policy drifts freely from the base model and reliably discovers and exploits the… See the full description on the dataset page: https://huggingface.co/datasets/ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.0-seed2-rollouts.fc-reward-bench
fc-reward-bench (HF papers) (arxiv)
fc-reward-bench is a benchmark designed to evaluate reward model performance in function-calling tasks. It features 1,500 unique user inputs derived from the single-turn splits of the BFCL-v3 dataset. Each input is paired with both correct and incorrect function calls. Correct calls are sourced directly from BFCL, while incorrect calls are generated by 25 permissively licensed models.
Performance of ToolRM, top reward models from… See the full description on the dataset page: https://huggingface.co/datasets/ibm-research/fc-reward-bench.red_teaming_reward_modeling_pairwise
Dataset Card for "red_teaming_reward_modeling_pairwise"
More Information needed
multimodal-rewardbench-2Paper: https://arxiv.org/abs/2512.16899
Multimodal RewardBench 2 (MMRB2). Processed from https://github.com/facebookresearch/MMRB2 .
If you find this useful, please cite with following bibtex:
@article{hu2025multimodalrewardbench2,
title={Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image},
author={Hu, Yushi and Askari-Hemmat, Reyhane and Hall, Melissa and Dinan, Emily and Zettlemoyer, Luke and Ghazvininejad, Marjan},
journal={arXiv preprint… See the full description on the dataset page: https://huggingface.co/datasets/rl-research/multimodal-rewardbench-2.MM-RLHF-RewardBench
[📖 arXiv Paper]
[📊 MM-RLHF Data]
[📝 Homepage]
[🏆 Reward Model]
[🔮 MM-RewardBench]
[🔮 MM-SafetyBench]
[📈 Evaluation Suite]
The Next Step Forward in Multimodal LLM Alignment
[2025/02/10] 🔥 We are proud to open-source MM-RLHF, a comprehensive project for aligning Multimodal Large Language Models (MLLMs) with human preferences. This release includes:
A high-quality MLLM alignment dataset.
A strong Critique-Based MLLM reward model and its training algorithm.
A novel… See the full description on the dataset page: https://huggingface.co/datasets/yifanzhang114/MM-RLHF-RewardBench.