datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
R-PRM
📘 R-PRM Dataset (SFT + DPO)
This dataset is developed for training Reasoning-Driven Process Reward Models (R-PRM), proposed in our ACL 2025 paper. It consists of two stages:
SFT (Supervised Fine-Tuning): collected from strong LLMs prompted with limited annotated examples, enabling reasoning-style evaluation.
DPO (Direct Preference Optimization): constructed by sampling multiple reasoning trajectories and forming preference pairs without additional labels.
These datasets are used… See the full description on the dataset page: https://huggingface.co/datasets/kevinpro/R-PRM.PRO-STEP-PRM-Data
PRO-STEP: PRM Training Annotations
Paper: PRO-STEP: Step-level Process Reward Optimization for Retrieval-Augmented GenerationCode: https://github.com/keemminnke/PRO-Step
Step-level annotations used to train the PRO-STEP PRM.
Total step annotations: ~109K across 31,728 trajectories
Source questions: 2,000 (HotpotQA + MuSiQue training splits)
Generation: 16 sampled trajectories per question with Qwen2.5-7B-Instruct
Annotator: QwQ-32B (open-source reasoning model), prompted with… See the full description on the dataset page: https://huggingface.co/datasets/MinKeonKim/PRO-STEP-PRM-Data.DianJin-Fin-PRM-Data
DianJin-Fin-PRM Dataset
Overview
DianJin-Fin-PRM is a Chinese financial domain Process Reward Model (PRM) training dataset. It contains 4,969 samples of financial exam questions with step-by-step reasoning traces and multi-dimensional quality annotations.
Dataset Structure
Field
Type
Description
名称
string
Exam name (e.g., 初级经济师)
科目
string
Subject (e.g., 金融实务)
章节
string
Chapter
task
string
Question type (e.g., 单项选择题)
question
string
Question… See the full description on the dataset page: https://huggingface.co/datasets/DianJin/DianJin-Fin-PRM-Data.math-prm-800k-deGerman version of prm800k. Translated using DeepL (informal style).
lang
#chars
en
11_479_654
de
12_516_903
prm_dpo_pairs_cleaned
prm_dpo_pairs_cleaned
Description
Taken M4-ai/prm_dpo_pairs and extensively cleaned it.
Filtering technique
We filtered out any incorrect completions in the chosen field
We extensively deduplicated around 3,000 examples after filtering out incorrect completions.
The resulting dataset should be able to train a very strong math language model
step_prm
数据集名称
是否有step
可用于PRM训练
标签形式
Title
备注
GSM8K
✅
❌
答案
Training Verifiers to Solve Math Word Problems
MATH
❌
❌
答案
Measuring Mathematical Problem Solving With the MATH Dataset
Non-Step
PRM800K
✅
✅
正确类别
Let's Verify Step by Step
prompt deduplication
Math-Shepherd
✅
✅
正确类别
Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
Not used
ProcessBench
✅
✅
首个错误步骤
ProcessBench: Identifying Process Errors in Mathematical Reasoning
only label -1
prm_dpo_pairs
prm_dpo_pairs
Description
prm_dpo_pairs is a curated version of the PRM800K dataset designed for ease of use when fine-tuning a language model using the DPO (Direct Preference Optimization) technique. The dataset contains pairs of prompts and completions, with labels indicating which completion was preferred by the original language model.
Dataset Structure
The dataset consists of the following features:
prompt: The input prompt or question posed to the… See the full description on the dataset page: https://huggingface.co/datasets/M4-ai/prm_dpo_pairs.PRM_1541i
Not the paper's DPO data. This is an earlier set of 1,541 preference pairs used for development DPO sweeps. The released critic Qwen3-4B-Critic-SFT-DPO from Steer, Don't Solve was trained on a different set of 1,409 pairs built with the same procedure. See the organization page for the artifacts behind the paper.
PRM_1541i
1541 preference pairs for training a critic over coding-agent trajectories. Each example is a
multi-turn agent transcript paired with two candidate… See the full description on the dataset page: https://huggingface.co/datasets/code-critic-model/PRM_1541i.prm800k_scThis repository contains the dataset for Self-Correction Bench: Revealing and Addressing the Self-Correction Blind Spot in LLMs.
Self-Correction Bench introduces a systematic framework to measure the "Self-Correction Blind Spot" in Large Language Models (LLMs) through controlled error injection at three complexity levels. The dataset allows for studying why LLMs, despite being able to identify errors in user input, often fail to correct identical errors in their own outputs. This work… See the full description on the dataset page: https://huggingface.co/datasets/kenhktsui/prm800k_sc.AI502-DeepMath-15K-PRM
Original Dataset : random sampled 15K rows from the link (https://huggingface.co/datasets/zwhe99/DeepMath-103K)
Process Reward Model : Qwen2.5-MATH-PRM-7B
\n\n으로 구분 후, PRM 결과가 0.9이하인 step을 으로 정의