datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
SWE-Bench-Verified-O1-reasoning-high-results
SWE-Bench Verified O1 Dataset
Executive Summary
This repository contains verified reasoning traces from the O1 model evaluating software engineering tasks. Using OpenHands + CodeAct v2.2, we tested O1's bug-fixing capabilities on the SWE-Bench Verified dataset, achieving a 28.8% success rate across 500 test instances.
Overview
This dataset was generated using the CodeAct framework, which aims to improve code generation through enhanced action-based reasoning.… See the full description on the dataset page: https://huggingface.co/datasets/AlexCuadron/SWE-Bench-Verified-O1-reasoning-high-results.Big-Math-RL-Verified
Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models
Big-Math is the largest open-source dataset of high-quality mathematical problems, curated specifically for reinforcement learning (RL) training in language models. With over 250,000 rigorously filtered and verified problems, Big-Math bridges the gap between quality and quantity, establishing a robust foundation for advancing reasoning in LLMs.
Request Early Access to Private… See the full description on the dataset page: https://huggingface.co/datasets/SynthLabsAI/Big-Math-RL-Verified.simpleqa-verified
SimpleQA Verified
A 1,000-prompt factuality benchmark from Google DeepMind and Google Research, designed to reliably evaluate LLM parametric knowledge.
▶ SimpleQA Verified Leaderboard on Kaggle▶ Technical Report▶ Evaluation Starter Code
Benchmark
SimpleQA Verified is a 1,000-prompt benchmark for reliably evaluating Large Language Models (LLMs) on short-form factuality
and parametric knowledge. The authors from Google DeepMind and Google Research… See the full description on the dataset page: https://huggingface.co/datasets/google/simpleqa-verified.MedCalc-Bench-Verified
Updates
Updates to MedCalc-Bench Verified will be made on this page going forward.
Here is the github link for our repository: https://github.com/nikhilk7153/MedCalc-Bench-Verified
This is an updated version that is modified from MedCalc-Bench-v1.2.
While we have audited MedCalc-Bench Verified on mulitple occasions, should there by any corrections or enhancements, we will update with a new release and specify any changes.
The HuggingFace dataset and main branch will always… See the full description on the dataset page: https://huggingface.co/datasets/nsk7153/MedCalc-Bench-Verified.SWE-Bench-Verified-O1-native-tool-calling-reasoning-high-results
SWE-Bench Verified O1 Dataset
Executive Summary
This repository contains verified reasoning traces from the O1 model evaluating software engineering tasks. Using OpenHands + CodeAct v2.2, we tested O1's bug-fixing capabilities using their native tool calling capabilities on the SWE-Bench Verified dataset, achieving a 45.8% success rate across 500 test instances.
Overview
This dataset was generated using the CodeAct framework, which aims to improve code… See the full description on the dataset page: https://huggingface.co/datasets/AlexCuadron/SWE-Bench-Verified-O1-native-tool-calling-reasoning-high-results.TACO-verified
Introduction
This dataset contains verified solutions from the TACO dataset's training set. Solutions that fail to pass all the test cases are removed. Problems with no correct solution are also removed.
The solutions were executed on Intel E5-2620 v3 CPUs with the execution timeout set to 10 seconds.
Statistics in the training set
Dataset
# Problems
# Solutions
TACO
25443
1468722
TACO-verified
12898
1043251
Correct Ratio
50.69 %
71.03 %… See the full description on the dataset page: https://huggingface.co/datasets/likaixin/TACO-verified.WebInstruct-verified
General-Reasoner: Advancing LLM Reasoning Across All Domains
💻 Code |
📄 Paper |
📊 Dataset |
🤗 Model |
🌐 Project Page
Overview
Figure: Effectiveness of General-Reasoner trained with diverse verifiable reasoning questions using model-based verifier compared to baseline methods on various reasoning tasks.
General-Reasoner is a training paradigm for large language models (LLMs), designed to robustly enhance reasoning abilities across… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/WebInstruct-verified.finqa-verified
FinQA: Financial Question Answering Dataset
Description
The FinQA dataset is designed to facilitate research and development in the area of question answering (QA) using financial texts.
It consists of a subset of QA pairs from a larger dataset, originally created through a collaboration between researchers from the University of Pennsylvania,
J.P. Morgan, and Amazon.The original dataset includes 8,281 QA pairs built against publicly available earnings reports of S&P… See the full description on the dataset page: https://huggingface.co/datasets/Aiera/finqa-verified.VisualWebInstruct-verified
🧠 VisualWebInstruct-Verified: High-Confidence Multimodal QA for Reinforcement Learning
VisualWebInstruct-Verified is a high-confidence subset of VisualWebInstruct, curated specifically for Reinforcement Learning (RL) and Reward Model training.
It contains verified multimodal question–answer pairs where correctness, reasoning quality, and image–text alignment have been explicitly validated.
This dataset is ideal for RLVR training pipelines.
📘 Dataset Overview… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/VisualWebInstruct-verified.trivia_qa_verified
TriviaQA Verified
A quality-verified subset of TriviaQA (Joshi et al., 2017) containing 4,170 question-answer pairs with confirmed correct answers, available in 5 languages.
Splits
Split
Language
Rows
english
English
4,170
mandarin
Mandarin Chinese
4,170
japanese
Japanese
4,170
arabic
Arabic
4,170
french
French
4,170
validation
English
3,381
The validation split contains a separate set of verified English questions (no overlap with other splits)… See the full description on the dataset page: https://huggingface.co/datasets/ai-safety-institute/trivia_qa_verified.SimpleQA-VerifiedSimpleQA Verified is a 1,000-prompt benchmark for reliably evaluating Large Language Models (LLMs) on short-form factuality and parametric knowledge. The authors from Google DeepMind and Google Research address various limitations of SimpleQA, originally designed by Wei et al. (2024) at OpenAI, including noisy and incorrect labels, topical biases, and question redundancy.
SimpleQA Verified was created to provide the research community with a more precise instrument to track genuine progress in… See the full description on the dataset page: https://huggingface.co/datasets/stalkermustang/SimpleQA-Verified.BixBench-Verified-50
BixBench-Verified-50
A curated subset of BixBench with 50 verified questions across 33 unique data capsules, designed for reliable evaluation of AI agents on computational biology tasks.
Overview
BixBench-Verified-50 was created to isolate real agent performance from benchmark issues. Starting from the full BixBench benchmark, we sampled questions and identified problematic ones. Some were removed entirely. For others, we revised the question text for clarity or corrected… See the full description on the dataset page: https://huggingface.co/datasets/phylobio/BixBench-Verified-50.simpleqa-verified-multilingual
SimpleQA Verified Multilingual
Multilingual translations of SimpleQA Verified, a 1,000-prompt factuality benchmark from Google DeepMind that evaluates short-form parametric knowledge (facts stored in model weights).
Source: google/simpleqa-verified (eval split, 1,000 examples)
Languages
Config
Language
Examples
ces
Czech
100
dan
Danish
100
deu
German
1,000
fra
French
100
ita
Italian
100
nld
Dutch
100
pol
Polish
100
spa
Spanish
100
More to… See the full description on the dataset page: https://huggingface.co/datasets/ellamind/simpleqa-verified-multilingual.verified-sql-rewards
Verified SQL Rewards
A text-to-SQL corpus where every reward carries a machine-checkable proof
that it is correct.
Questions, all independently verified
109,306
Databases
1,400 across 7 schema families
Tables / data rows
4,400 / ~19.6 million
Unique (question, answer) pairs
102,764
Candidates refused and published
12,150
Verification pass rate
90.00%
Trivial baseline (always answer 0)
1.83%
Each item is a natural-language question, a gold SQL query… See the full description on the dataset page: https://huggingface.co/datasets/rasinmuhammed/verified-sql-rewards.MAPS_Verified
Dataset Card for Multilingual Benchmark for Global Agent Performance and Security
This is the first Multilingual Agentic AI Benchmark for evaluating agentic AI systems across different languages and diverse tasks. Benchmark enables systematic analysis of how agents perform under multilingual conditions. This dataset contains 550 instances for GAIA, 660 instances for ASB, 737 instances for Maths, and 1100 instances for SWE. Each task was translated into 10 target languages resulting… See the full description on the dataset page: https://huggingface.co/datasets/Fujitsu-FRE/MAPS_Verified.verified-math-code-17k
Verified Math & Code, 17,000 rows
A math and code instruction dataset where every single row was mechanically checked before it was
allowed in. Not filtered by a heuristic, not scored by a model. Checked.
Two layers of verification, one per domain:
Every math answer was compared against an independent gold answer by exact, numeric and
symbolic (SymPy) comparison. If the worked solution did not arrive at the gold answer, the row
was dropped.… See the full description on the dataset page: https://huggingface.co/datasets/manifesta/verified-math-code-17k.verified_wiki_historian_the_beatles_anthology_dataset_active
Verified-Wiki-Historian: The Beatles Anthology
Verified-Wiki-Historian (The Beatles Anthology) is a refined, citation-grounded instruction dataset for Beatles-specific historical question answering, summarization, and supervised fine-tuning.
This dataset is a cleaned and rebuilt refinement of:
Mungus451/verified_wiki_historian_the_beatles_anthology_dataset_active
The current release contains 4,000 instruction records focused on Beatles history, recording sessions, release… See the full description on the dataset page: https://huggingface.co/datasets/Mungus451/verified_wiki_historian_the_beatles_anthology_dataset_active.verified-research-reasoning-trajectories
Verified Research Reasoning Trajectories for RLVR
This repository is the public sample and schema repository for Ulam's research-level mathematical reasoning trajectories for reinforcement learning with verifiable rewards (RLVR), process supervision, judge training, proof criticism, and private evaluations.
Ulam Verified Research Reasoning Trajectories are proof-process data for RLVR. Each record contains a normalized research problem, a golden or partial-golden proof graph… See the full description on the dataset page: https://huggingface.co/datasets/ulamai/verified-research-reasoning-trajectories.SAFE-Verified-MultiHopQA
SAFE-Verified-MultiHopQA
Paper | Project Page
Overview
SAFE-Verified-MultiHopQA is released with the EMNLP 2026 paper SAFE: An LLM-as-Verifier Framework for Evidence-Grounded Multi-Hop Reasoning. It provides Knowledge Graph (KG) based, cleaned versions of three multi-hop question-answering benchmarks:
2WikiMultiHopQA (2wiki)
HotpotQA (hotpotqa)
MuSiQue (musique)
SAFE uses KG-grounded verification to identify questions with invalid or ungrounded reasoning. This… See the full description on the dataset page: https://huggingface.co/datasets/Daeyongkwon98/SAFE-Verified-MultiHopQA.chart-reasoning-verified
chart-reasoning-verified
Chart reasoning examples generated from an explicit latent representation.
The data, the question and the answer are computed before the chart is
drawn, so the image is a rendering of known ground truth rather than the
source of it. No model was asked to label anything.
Each row carries both a rendered chart and a text serialisation of the same
chart, so the set is usable for vision-language training and for text-only
language model training without… See the full description on the dataset page: https://huggingface.co/datasets/vinod-anbalagan/chart-reasoning-verified.verified-tool-use-dataset
Verified tool-use trajectories for LLM agents
This was a time-boxed experiment by an autonomous agent (Protogonos), now concluded. Nothing here is offered for sale or for hire, and no payment is accepted.
Multi-turn function-calling conversations for training and evaluating
tool-using agents — 48 trajectories across 16 domains, with every tool call
checked against its tool's JSON-Schema. The free sample in this repo is a real
slice of the full set: the viewer above renders it… See the full description on the dataset page: https://huggingface.co/datasets/protogonos/verified-tool-use-dataset.physics-verified
PHYSICS-Verified
PHYSICS-Verified is a benchmark of 1,109 PhD-qualifying-exam physics problems with 2,803 scored answers, covering six core areas of physics. Every problem asks for results that can be checked: numbers, formulas, or short verbal conclusions. Each answer has been checked against its reference solution.
This release is a cleaned, results-only version of the original PHYSICS benchmark (GitHub). Problems that required a proof, explanation, or drawing were removed, as… See the full description on the dataset page: https://huggingface.co/datasets/yale-nlp/physics-verified.verified-math-reasoning
verified-math-reasoning (CargoDash flagship recipe)
A CargoDash framework demonstration. 999-row showcase of three-layer,
program-verified, vote-stratified math reasoning traces — the dataset is
small on purpose (its job is to prove the framework works on real
production LLM endpoints, not to be a serious math benchmark). Each row
carries three independent chain-of-thought solutions to the same problem
(from DeepSeek, Doubao, and Qwen3.5) plus a programmatically extracted
\boxed{}… See the full description on the dataset page: https://huggingface.co/datasets/Bdyskov/verified-math-reasoning.IMO-AnswerBench-Verified
IMO AnswerBench Verified
IMO AnswerBench Verified is a human-expert-verified derivative of OpenEvals/IMO-AnswerBench, originally curated by the Google DeepMind Superhuman Reasoning team. Every record in the 400-problem benchmark was reviewed individually. The review identified and corrected 13 records while preserving the benchmark's balanced coverage of four major mathematical areas.
Dataset summary
Total records: 400
Verification method: record-by-record human… See the full description on the dataset page: https://huggingface.co/datasets/dots-studio/IMO-AnswerBench-Verified.verified-agronomy-17k
Verified Agronomy, 17,199 rows
The arithmetic a working agronomist actually does: fertiliser rates, growing degree days,
irrigation scheduling, seeding rates, grain moisture, sprayer calibration, forage budgeting.
Every formula traced to a citable published source, and every number checked.
And 1,796 rows (10.4%) that cannot be answered at all, where the correct response is to say so
and name the missing input.
Verify this card… See the full description on the dataset page: https://huggingface.co/datasets/manifesta/verified-agronomy-17k.APPS-verified
Introduction
This dataset contains verified solutions from the APPS dataset's training set. Solutions that fail to pass all the test cases are removed. Problems with no correct solution are also removed.
The solutions were executed on Intel E5-2620 v3 CPUs with the execution timeout set to 10 seconds.
Statistics in the training set
Dataset
# Problems
# Solutions
TACO
5000
117232
TACO-verified
4211
93921
Correct Ratio
84.22%
80.12%
WebInstruct-verified-unfilteredThis repo contains the unfiltered version WebInstruct-verified in the General Reasoner work.
General-Reasoner: Advancing LLM Reasoning Across All Domains
💻 Code |
📄 Paper |
📊 Dataset |
🤗 Model |
🌐 Project Page
Overview
Figure: Effectiveness of General-Reasoner trained with diverse verifiable reasoning questions using model-based verifier compared to baseline methods on various reasoning tasks.
General-Reasoner is a training paradigm… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/WebInstruct-verified-unfiltered.BIRD-Verified-CoT-2462-GPT5.4
BIRD-Verified-CoT-2462 (GPT-5.4 distilled)
Likely the first publicly available CoT-augmented Text-to-SQL dataset built on top of expert-verified BIRD data.
This dataset combines two state-of-the-art ingredients:
ReViSQL's BIRD-Verified subset — 2,462 SQL-expert verified examples (multi-round review by UIUC team), eliminating the ~50% annotation noise of the original BIRD train set.
GPT-5.4 (via Codex CLI) — distilled into structured 6-section Chain-of-Thought traces using… See the full description on the dataset page: https://huggingface.co/datasets/wenyupapa/BIRD-Verified-CoT-2462-GPT5.4.verisci-verified-science-math-code
VeriSci Verified Science Math Code
Verifier-grounded dataset for the Adaption AutoScientist Challenge Part 2, targeting the Science category with secondary Math and Code coverage.
Summary
VeriSci trains models to solve scientific computations, finite-difference PDE updates, numerical ODE steps, unit-checked mechanics, thermodynamics, circuits, chemistry stoichiometry, molarity, unit conversion, vector decomposition, two-point linear modeling, small Python… See the full description on the dataset page: https://huggingface.co/datasets/TheJackBright/verisci-verified-science-math-code.verified-math-reasoning-3k
HSH Verified Math Reasoning — Fine-Tuning Ready
A clean, answer-verified dataset of step-by-step math word problems with chain-of-thought reasoning, formatted for instruction fine-tuning. This is foundational reasoning data designed for first fine-tunes — single-concept arithmetic word problems with fully verified answers, ideal for a reliable, clean starter run. Every single answer in this dataset has been programmatically verified against a ground-truth value computed in… See the full description on the dataset page: https://huggingface.co/datasets/HSH-Intelligence/verified-math-reasoning-3k.