datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
model-written-evals
Model-Written Evaluation Datasets
This repository includes datasets written by language models, used in our paper on "Discovering Language Model Behaviors with Model-Written Evaluations."
We intend the datasets to be useful to:
Those who are interested in understanding the quality and properties of model-generated data
Those who wish to use our datasets to evaluate other models for the behaviors we examined in our work (e.g., related to model persona, sycophancy, advanced AI risks… See the full description on the dataset page: https://huggingface.co/datasets/Anthropic/model-written-evals.math-evals
math-evals
Uniform {question, answer} math evaluation splits for a single source of
truth across benchmarks. Every split exposes exactly two columns:
question and answer.
split
source
source split
rows
clean_gsm8k_aug
cs-giung/clean-gsm8k-aug @60f9c039
test
1319
clean_gsm8k_aug_val
cs-giung/clean-gsm8k-aug @60f9c039
validation
500
gsm_hard
reasoning-machines/gsm-hard @960448f7
train
1319
gsm1k
ScaleAI/gsm1k @bc09569d
test
1205
gsm8k
openai/gsm8k @740312ad
test… See the full description on the dataset page: https://huggingface.co/datasets/cs-giung/math-evals.model-written-evalsThis new dataset is designed to solve this great NLP task and is crafted with a lot of care.generalization-dynamics-evals
Generalization Dynamics — Main Eval Suite
Prepared test sets for the 6 main evaluation families from
Generalization dynamics across fine-tuning
(Table 1).
Use with the unified runner:
https://github.com/jiaxin-wen/FT-generalization/tree/main/release
from huggingface_hub import snapshot_download
root = snapshot_download(
repo_id="jiaxin-wen/generalization-dynamics-evals", repo_type="dataset")
Or browse a single task (the dataset viewer shows all configs):
from datasets… See the full description on the dataset page: https://huggingface.co/datasets/jiaxin-wen/generalization-dynamics-evals.desearch-search-evals
Desearch Search Benchmark
Graded results from comparing web search APIs on public question-answering benchmarks. Every provider answers the same questions under the same conditions, and its results are scored against each benchmark's reference answers or reference pages.
Results can be browsed at 22.desearch.ai. The code that produced them is at github.com/Desearch-ai/desearch-search-evals.
Benchmarks
SimpleQA Verified (463 questions): Short factual questions… See the full description on the dataset page: https://huggingface.co/datasets/desearch/desearch-search-evals.oreolook-research-evals
OreoLook Research Evaluations
A small, inspectable evaluation suite for current-information search agents. It
covers routing, clarification, conversational continuity, citation discipline,
freshness, PDF artifacts, protocol safety, and semantic-cache equivalence.
This repository accompanies:
OreoLook live Space
OreoLook source
Architecture paper
Project website
Contents
Configuration
Rows
Purpose
evaluations
24
Synthetic user turns and expected agent… See the full description on the dataset page: https://huggingface.co/datasets/p-research/oreolook-research-evals.evals
Instruct Augmented Datasets
This dataset takes various other multiple choice, summarization, etc datasets and augments them to be instruct finetuned.
gacar-assistant-evals
GACAR Assistant Evals (Saudi Civil Aviation)
The official evaluation dataset for Captain Adel (captadel.com / Fly GACA) — the independent, retrieval-grounded AI flight instructor for Saudi civil aviation regulations (GACAR).
Dataset Summary
Every prompt or model change to Captain Adel is eval-gated in both English and Arabic against this suite.
Total Cases: 150
Domains Covered: 30+ GACAR Parts (Part 1, 61, 67, 91, 107, 121, 135, 139, etc.)
Categories: citation… See the full description on the dataset page: https://huggingface.co/datasets/flygaca/gacar-assistant-evals.financial-model-evals
Worthune Financial Model Evals (open sample)
Ground truth for financial AI: 3 open datasets × 250
input/expected-output pairs — the free-sample slice of a 55-model catalog
covering refinance break-evens, retirement projections, Roth conversions,
equity compensation, loan payoff, and more. The full catalog's datasets
download with a Worthune Pro key from
GET https://worthune.com/api/v1/evals/{model} (index).
Every expected output comes from two independent implementations that… See the full description on the dataset page: https://huggingface.co/datasets/worthune/financial-model-evals.model-written-evals
Model-Written Evaluation Datasets
This repository includes datasets written by language models, used in our paper on "Discovering Language Model Behaviors with Model-Written Evaluations."
We intend the datasets to be useful to:
Those who are interested in understanding the quality and properties of model-generated data
Those who wish to use our datasets to evaluate other models for the behaviors we examined in our work (e.g., related to model persona, sycophancy, advanced AI… See the full description on the dataset page: https://huggingface.co/datasets/rAVEUK/model-written-evals.ballast-evalsets
Ballast eval sets
The probe sets behind the Ballast
measurements. Two files:
matrix_probes.parquet: 50,147 factual questions (recall), each with a
gold answer, alias set, 7 type-matched distractors, and (where resolvable)
a Wikidata subject Q-id linking it to the
Ballast T0 corpus
(90.5% linked at full corpus). Measured on two model families: Gemma-4
(E2B/E4B/12B) and Qwen3.5 (0.8B/2B/4B/9B), plus a quantization sweep over
bf16 / fp8 / nf4 / Q6_K / Q4_K_M.… See the full description on the dataset page: https://huggingface.co/datasets/OpenBallast/ballast-evalsets.blockrank-beir-evals
ICR-BEIR-Evals: In-Context Ranking Evaluation Dataset
Dataset Description
ICR-BEIR-Evals is a curated evaluation dataset for In-Context Ranking (ICR) models, derived from the BEIR benchmark. This dataset is specifically designed to evaluate the effectiveness of generative language models on document ranking tasks where queries and candidate documents are provided in-context.
The dataset contains 28,759 queries across 11 diverse BEIR datasets, with each query paired with… See the full description on the dataset page: https://huggingface.co/datasets/quicktensor/blockrank-beir-evals.daft-math
DAFT Math: Difficult Automatically-scorable Free-response Tasks for Math
Dataset Description
⚠️ Note: The dataset has important limitations and we strongly recommend reading the limitations section below before using it. It is not a formal METR benchmark and was originally designed for a very niche use-case. We present it only as a research artifact.
DAFT-Math is a collection of 199 challenging mathematical problems chosen to be at the limit of current LLM abilities… See the full description on the dataset page: https://huggingface.co/datasets/metr-evals/daft-math.cibelex-qa-rag-evals
Cibelex QA RAG Evals
Description
Question-answering evaluation set over the regulatory corpus of the Ayuntamiento de Madrid (LoRO ontology / Cibelex knowledge graph). Each item pairs a natural-language question with a ground-truth answer, the top-4 passages returned by a baseline retriever, and the answer generated by a baseline LLM from those passages.
Designed as a benchmark for:
Retrieval-augmented generation (RAG): comparing new retrievers and answer generators… See the full description on the dataset page: https://huggingface.co/datasets/MAIA-Madrid-IA/cibelex-qa-rag-evals.boe-rag-evalset
BOE RAG Assistant — Evaluation Set
Question–answer pairs with chunk-level relevance judgments for evaluating retrieval
and grounded generation over the
BOE corpus. Every example
names the corpus chunk(s) that answer it, so it doubles as a retrieval relevance set
and as a grounded-QA reference.
Built for the BOE RAG Assistant
project, where the seed split is the CI regression fixture.
Splits
Config
Examples
Provenance
Use
seed (default)
20
Hand-curated… See the full description on the dataset page: https://huggingface.co/datasets/gonzalonao/boe-rag-evalset.rcqa-system-XiaoHong-v1-golden-evalSets
紅樓夢相關古漢語知識問答系統-黃金驗證集
本資料集為「基於 RA-LLMs 架構之古漢語知識問答系統」的整體系統驗證集，包含 60 題以《紅樓夢》為核心的學術題目，依 Bloom 認知分類法設計，用於 RAG + LM-ft 系統之自動化評估管線。
皆為AI(LLM)生成。