datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
rlvr-reward-hacking-scale-no-conftest-20260909-completion
Matched no-conftest RLVR study 20260909-completion
Lossless research records, grouped by model and trajectory type. Only the listed
configurations have published records. Canary diagnostics are excluded from study
estimates; run status in provenance distinguishes retired diagnostics from active
or completed training. Valid failures, refusals and truncations are retained.
The train split name is a dataset-loader convention; record_type identifies
whether a record is training… See the full description on the dataset page: https://huggingface.co/datasets/lucabaroni/rlvr-reward-hacking-scale-no-conftest-20260909-completion.Scale-SWE
Immersion in the GitHub Universe: Scaling Coding Agents to Mastery
🔥 Highlights
Source from 6M+ pull requests and 23000+ repositories.
Cover 5200 Repositories.
100k high-quality instances.
71k trajectories from DeepSeek v3.2 with 3.5B token.
Strong performance: 64% in SWE-bench-Verified trained from Qwen3-30A3B-Instruct.
📣 News
2026-02-26 🚀 We released a portion of our data on Hugging Face. This release includes 20,000 SWE task… See the full description on the dataset page: https://huggingface.co/datasets/AweAI-Team/Scale-SWE.Scaffold-CoT
Scaffold-CoT
Structured chain-of-thought training data with 3,726,548 examples in 76 JSONL shards.
Fields
Every row has exactly four top-level fields:
Field
Contents
metadata
domain, subdomain, difficulty, length_bucket
input
Ordered user messages as {index, content} objects
cot
Ordered {index, type, content} events, including reasoning, tool calls, and tool results
output
Ordered final assistant answers as {index, content} objects
The index… See the full description on the dataset page: https://huggingface.co/datasets/Specific-Labs/Scaffold-CoT.ThoughtTrace
ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactions
🌐 Project Page · 📄 Paper · 💻 Code
Conversational AI has reached billions of users, yet existing datasets capture only what people say, not what they think.
ThoughtTrace is the first large-scale dataset that pairs real-world multi-turn human–AI conversations with users' self-reported thoughts: their reasons for sending prompts and reactions to assistant responses.
ThoughtTrace comprises 1,058 users, 2,155… See the full description on the dataset page: https://huggingface.co/datasets/SCAI-JHU/ThoughtTrace.JudgeBench
JudgeBench: A Benchmark for Evaluating LLM-Based Judges
📃 [Paper] • 💻 [Github] • 🤗 [Dataset] • 🏆 [Leaderboard]
JudgeBench is a benchmark aimed at evaluating LLM-based judges for objective correctness on challenging response pairs. For more information on how the response pairs are constructed, please see our paper.
Data Instance and Fields
This release includes two dataset splits. The gpt split includes 350 unique response pairs generated by GPT-4o and the claude… See the full description on the dataset page: https://huggingface.co/datasets/ScalerLab/JudgeBench.researchrubricshttps://www.arxiv.org/abs/2511.07685
monkey_business
Monkey Business
Monkey Business is a dataset of samples from large language models. It contains both correct and incorrect samples from a variety of models (the Llama-3, Gemma, and Pythia series) on a variety of tasks (problems from GSM8K, MATH, CodeContests, and MiniF2F-MATH). We hope that it can be useful for developing improved verification methods that assess whether a model generated answer is correct.
This dataset was created as part of the project: "Large Language Monkeys:… See the full description on the dataset page: https://huggingface.co/datasets/ScalingIntelligence/monkey_business.scaling-data-constrained-llms
Scaling Data-Constrained Language Models with Synthetic Data
This repository provides the pre-training corpora used in Scaling Data-Constrained Language Models with Synthetic Data (Findings of EACL 2026).
Overview
This repository contains multiple corpora designed to study data augmentation strategies for pre-training Japanese LLMs under a data-constrained data setting.
Starting from a limited Japanese Web corpus and a larger English Web corpus, we construct three… See the full description on the dataset page: https://huggingface.co/datasets/llm-jp/scaling-data-constrained-llms.rlvr-reward-hacking-scale-no-conftest-20260909
Matched no-conftest RLVR study 20260909
Complete immutable training, monitoring and comparison trajectories for six models.
All valid outcomes are retained, including refusals, failures and truncations.
The train split name is a dataset-loader convention; record_type identifies
whether a record is training, monitoring, comparison, or a derived judgment.
import json
from datasets import load_dataset
rows = load_dataset("lucabaroni/rlvr-reward-hacking-scale-no-conftest-20260909"… See the full description on the dataset page: https://huggingface.co/datasets/lucabaroni/rlvr-reward-hacking-scale-no-conftest-20260909.VietPET-RoI
VietPET-RoI
VietPET-RoI is a Vietnamese whole-body PET/CT dataset containing paired
cropped 3D volumes, regional reports, and modality-specific 3D ROI bounding
boxes. It is intended for medical multimodal research, report generation,
visual question answering, and ROI grounding.
Research use only. This dataset is not intended for diagnosis, treatment
decisions, or direct patient care.
Summary
Split
Patients
CT/PET region pairs
ROIs
Train
160
480
1,544… See the full description on the dataset page: https://huggingface.co/datasets/scarlettlin/VietPET-RoI.lhaw
Dataset Card for LHAW: Long-Horizon Augmented Workflows
LHAW is a modular, dataset-agnostic synthetic pipeline that transforms well-specified tasks into controllable underspecified variants by systematically removing information across four dimensions—Goals, Constraints, Inputs, and Context—at configurable severity levels.
This dataset release contains 285 underspecified task variants derived from TheAgentCompany, SWE-Bench Pro, and MCP-Atlas, and is used to study how current agents… See the full description on the dataset page: https://huggingface.co/datasets/ScaleAI/lhaw.model-written-evalsScaleQuest-MathWe introduce ScaleQuest, a scalable and novel data synthesis method that utilizes small-size open-source models to generate questions from scratch.
Paper: Unleashing Reasoning Capability of LLMs via Scalable Question Synthesis from Scratch
DrugDiscoveryBench-Preview
DrugDiscoveryBench (Preview)
DrugDiscoveryBench is a benchmark of 82 expert-authored, execution-grounded tasks spanning the early
drug-discovery and life-sciences workflow (target identification & genetics, database screening,
patent mining, cheminformatics, structural reasoning, SAR & affinity, molecular biology). Each task
asks an agent to carry out a multi-step biomedical investigation and produce a terse final answer.
This is the Preview release: task prompts and metadata… See the full description on the dataset page: https://huggingface.co/datasets/ScaleAI/DrugDiscoveryBench-Preview.bbh-train-p1.0-bm25scarce_tsscaffold
SCAFFOLD
SCAFFOLD: A Large-Scale Structured Dataset of Computer Science Research Figures with Diagram QA and Chain-of-Thought Reasoning Traces is a large-scale multimodal reasoning dataset designed for training and evaluating Vision-Language Models (VLMs) on scientific figure understanding and visual reasoning.
The dataset is constructed from figures extracted from publicly available arXiv research papers and contains 157,387 question-answer pairs covering diverse scientific… See the full description on the dataset page: https://huggingface.co/datasets/ranjitraut/scaffold.scanqa-community-mirrorquote-repetition
quote-repetition (Joe Cavanagh, Andrew Gritsevskiy, and Derik Kauffman of Cavendish Labs)
General description
In this task, the authors ask language models to repeat back sentences given in the prompt, with few-shot examples to help it recognize the task. Each prompt contains a famous quote with a modified ending to mislead the model into completing the sequence with the famous ending rather than with the ending given in the prompt. The authors find that smaller models… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling/quote-repetition.Void-Witch-Astra-Vanta
Void Witch Astra Vanta
Source-derived release with authored context (schema 4)
448 rows: 93 unchanged conversation exchanges and 355 document chunks.
All 1,623 nonblank authored source lines appear exactly once as body text.
No passages are omitted. The row count changed from 788 because passages,
headings and lists are now grouped by their source relationships.
The seven original .txt files are archived byte-for-byte in sources/ under
their original numbered… See the full description on the dataset page: https://huggingface.co/datasets/scarletdeath/Void-Witch-Astra-Vanta.DrugDiscoveryBench
DrugDiscoveryBench
DrugDiscoveryBench is a benchmark of 82 expert-authored, execution-grounded tasks spanning the early
drug-discovery and life-sciences workflow (target identification & genetics, database screening,
patent mining, cheminformatics, structural reasoning, SAR & affinity, molecular biology). Each task
asks an agent to carry out a multi-step biomedical investigation and produce a terse final answer,
graded against a ground-truth answer and an outcome + process… See the full description on the dataset page: https://huggingface.co/datasets/ScaleAI/DrugDiscoveryBench.MetRex
MetRex
This dataset card contains Verilog modules along with their post-synthesis metrics (area, delay, static power) explained in natural language format. The synthesis results are obtained using Yosys and the Skywater 130nm PDK.
Dataset Details
Use
# pip install datasets
from datasets import load_dataset
ds = load_dataset("scale-lab/MetRex", split="train")
print(next(iter(ds)))
Data Splits
The dataset only contains the training split.… See the full description on the dataset page: https://huggingface.co/datasets/scale-lab/MetRex.inverse-scaling-ttc-main
Inverse Scaling in Test-Time Compute
Paper: Inverse Scaling in Test-Time Compute
Project Page: https://safety-research.github.io/inverse-scaling-ttc/
Abstract
We construct evaluation tasks where extending the reasoning length of Large Reasoning Models (LRMs) deteriorates performance, exhibiting an inverse scaling relationship between test-time compute and accuracy. Our evaluation tasks span four categories: simple counting tasks with distractors, regression tasks with… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling-ttc/inverse-scaling-ttc-main.fusion-pairwise-evals-test-time-scaling
Automatic pairwise preference evaluations for: Making, not taking, the Best-of-N
Content
This data contains pairwise automatic win-rate evaluations for the m-ArenaHard-v2.0 benchmark and it compares CommandA against gemini-2.5-pro in 2 settings:
Test-time scaling with Fusion : 5 samples are generated from CommandA, then fused with CommandA into one completion and compared to a single completion from gemini-2.5-pro
Test-time scaling with BoN : 5 samples are generated from… See the full description on the dataset page: https://huggingface.co/datasets/CohereLabs/fusion-pairwise-evals-test-time-scaling.MV-ScanQANeQA
NeQA: Can Large Language Models Understand Negation in Multi-choice Questions? (Zhengping Zhou and Yuhui Zhang)
General description
This task takes an existing multiple-choice dataset and negates a part of each question to see if language models are sensitive to negation. The authors find that smaller language models display approximately random performance whereas the performance of larger models become significantly worse than random.
Language models failing to follow… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling/NeQA.ScanQA_Finetuneicelandic_qa_scandevalA question answering dataset for evaluating LLMs' ability to answer Icelandic questions on Icelandic culture and history.
The dataset contains 2,000 pairs of questions and answers in Icelandic on the topic of Icelandic culture and history. All pairs were automatically created using GPT-4-turbo and then manually reviewed and augmented. 1,900 pairs were created from Icelandic Wikipedia articles and 100 pairs were created from Icelandic online news, the RÚV subcorpus of the Icelandic Gigaword… See the full description on the dataset page: https://huggingface.co/datasets/mideind/icelandic_qa_scandeval.redefine-math
redefine-math (Xudong Shen)
General description
In this task, the author tests whether language models are able to work with common symbols when they are redefined to mean something else. The author finds that larger models are more likely to pick the answer corresponding to the original definition rather than the redefined meaning, relative to smaller models.
This task demonstrates that it is difficult for language models to work with new information given at inference… See the full description on the dataset page: https://huggingface.co/datasets/inverse-scaling/redefine-math.scaling_law_discovery_results
Scaling Law Discovery Results Dataset
Results dataset for the paper: "Can Language Models Discover Scaling Laws?"
This dataset contains the complete collection of results from the Scaling Law Discovery (SLDBench) benchmark, where various AI agents attempt to discover mathematical scaling laws from experimental LLM training data.
🔗 Quick Links
Resource
Link
📄 Paper
arXiv:2507.21184
📊 Original Benchmark
SLDBench Dataset
🧪 Benchmark Code… See the full description on the dataset page: https://huggingface.co/datasets/pkuHaowei/scaling_law_discovery_results.