datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
thinking-benchmark-90
Thinking Benchmark
A calibration pool of 90 competition-mathematics problems assembled to study how output / reasoning-trace length varies with problem difficulty across frontier language models. Part of the Cost of Overthinking research project.
Dataset at a glance
Source
n
Difficulty
Contamination risk
AIME 2026
29
3–5
low
OlymMATH
41
4–6
medium
HMMT February 2026
12
4–5
low
MATH-500
5
2–3
high
FrontierMath-style
3
6
medium
Difficulty is… See the full description on the dataset page: https://huggingface.co/datasets/tyrtleli/thinking-benchmark-90.legal-llm-benchmark
Legal LLM Benchmark Dataset
Safety-Utility Trade-offs in Legal AI: An LLM Evaluation Across 12 Models
Quick Start
from datasets import load_dataset
# Core datasets
questions = load_dataset("marvintong/legal-llm-benchmark", "questions")
phase1_evals = load_dataset("marvintong/legal-llm-benchmark", "phase1_evaluations")
phase3_evals = load_dataset("marvintong/legal-llm-benchmark", "phase3_evaluations")
# Additional helpful datasets
contracts =… See the full description on the dataset page: https://huggingface.co/datasets/marvintong/legal-llm-benchmark.YOXLA-Benchmark
YOXLA Benchmark
1443 frozen examples for evaluating large language models in
Azerbaijani, across four blocks and eleven tasks.
Run with the YOXLA framework:
pip install "yoxla[api]"
yoxla run --provider openrouter --model <model> --block all
Or load a config directly:
from datasets import load_dataset
data = load_dataset("LocalDoc/YOXLA-Benchmark", "rag_selection_v1")["test"]
What makes it different
Every answer space is closed. A label, a number, or a span… See the full description on the dataset page: https://huggingface.co/datasets/LocalDoc/YOXLA-Benchmark.LEM-benchmarks
LEM-benchmarks
Canonical 8-PAC benchmark results for the Lemma model family.
This dataset is an aggregated store of per-round evaluation data produced by
lthn/LEM-Eval. Every row
represents one model's answer to one question in one round of a paired A/B
run against its unmodified base, and the dataset grows monotonically as more
workers contribute — different machines, different sampling states, different
hardware paths — which is the whole point of 8-PAC: multiple independent… See the full description on the dataset page: https://huggingface.co/datasets/lthn/LEM-benchmarks.ckg-benchmark
CKG Benchmark
Pre-structured knowledge graphs outperform RAG by 4× F1 at 11× lower token cost — across 65 benchmarked domains.
System
Macro F1
Tokens/query
RDS
Run Cost
CKG
0.4709
269
0.00175
$7.81
RAG
0.1231
2,982
0.0000413
$76.23
GraphRAG
0.1200
3,450
0.0000452
$44.43
42× more intelligence per token than RAG. Auditable by design — every answer traces to a declared edge.
Dataset Contents
domains/{domain}/learning-graph.csv — structured DAG… See the full description on the dataset page: https://huggingface.co/datasets/danyarm/ckg-benchmark.Benchmarks_CyberSec_RedSageMCQ
Dataset Card for RedSage-MCQ
Dataset Summary
RedSage-MCQ is a large-scale, high-quality multiple-choice question (MCQ) benchmark designed to evaluate the cybersecurity knowledge, skills, and tool proficiency of Large Language Models (LLMs). It is a component of the RedSage-Bench suite introduced in the paper "RedSage: A Cybersecurity Generalist LLM".
The dataset comprises 30,000 questions derived from RedSage-Seed, a curated collection of authoritative… See the full description on the dataset page: https://huggingface.co/datasets/RISys-Lab/Benchmarks_CyberSec_RedSageMCQ.hallucination-autopsy-benchmark
Hallucination Autopsy Benchmark
A unified, standardized benchmark for cross-model, cross-parameter analysis of LLM hallucination phenomena.
Overview
This dataset merges multiple hallucination detection benchmarks into a single standardized schema, enabling systematic etiological analysis of why and how different LLM architectures hallucinate under specific configurations.
Version: 3.0.0Total Records: 69,002Base Records: 69,002Augmented Records: 0Hallucination… See the full description on the dataset page: https://huggingface.co/datasets/OiQ/hallucination-autopsy-benchmark.benchmark-radar
Benchmark Radar Dataset
Overview
Benchmark Radar is a living registry, search engine, and discovery pipeline
for AI evaluation benchmarks. This dataset mirrors the full-corpus findings of the
Benchmark Radar technical report ("Benchmark Radar: Daily Discovery and
Full-Corpus Search Across the AI Evaluation Landscape", arXiv:2609.11115).
As described in the paper's Two Input Paths framework, Benchmark Radar
combines two complementary systems:… See the full description on the dataset page: https://huggingface.co/datasets/ktwu01/benchmark-radar.math_benchmark_test_saturation
LLM Leaderboard Data for Hendrycks MATH Dataset (2022–2024)
This dataset aggregates yearly performance (2022–2024) of large language models (LLMs) on the Hendrycks MATH benchmark. It is specifically compiled to explore performance evolution, benchmark saturation, parameter scaling trends, and evaluation metrics of foundation models solving complex math word problems.
Original source data: Math Word Problem Solving on MATH (Papers with Code)
About Hendrycks' MATH… See the full description on the dataset page: https://huggingface.co/datasets/nlile/math_benchmark_test_saturation.MEME
MEME: Multi-Entity and Evolving Memory Evaluation
A benchmark for evaluating LLM memory systems along two orthogonal dimensions: entity scope (single vs. multi-entity) and temporal dynamics (static vs. evolving). MEME defines six tasks targeting memory-intensive operations in each quadrant, including two task types that no prior benchmark covers: Cascade (propagating updates through dependency rules) and Absence (recognizing uncertainty when a previously valid answer becomes… See the full description on the dataset page: https://huggingface.co/datasets/meme-benchmark/MEME.ldr-benchmarks
LDR Community Benchmarks (Leaderboards)
Aggregated leaderboards for Local Deep Research (LDR) community benchmark
runs against SimpleQA, BrowseComp, and xbench-DeepSearch.
👉 Submit results, read raw YAMLs, open PRs:
github.com/LearningCircuit/ldr-benchmarks
This Hugging Face dataset hosts only the aggregated CSV leaderboards.
It is regenerated automatically on every merge to main in the GitHub
repo above. Each CSV row represents one benchmark run (one strategy… See the full description on the dataset page: https://huggingface.co/datasets/local-deep-research/ldr-benchmarks.singapore-legal-ai-benchmark
Singapore Legal AI Benchmark
Public research release of 102 Singapore legal research questions, model
responses from 6 systems, and overlapping grades on five dimensions.
Headline metrics are overlapping binary flags, not a ranking and not a
partition of 100%.
Interactive explorer
Open the explorer →
— comparison table, category heatmap, per-question comparison, and every answer
with its sources and grades.
(Space page)
Overall (n = 612)… See the full description on the dataset page: https://huggingface.co/datasets/JonathanSu/singapore-legal-ai-benchmark.tw-legal-benchmark-v2
Taiwan Legal Benchmark v2
A multiple-choice benchmark for evaluating LLMs on Taiwan law in Traditional Chinese,
built from 15 years (2012–2026) of national examinations published by the
Ministry of Examination (考選部).
Supersedes tw-legal-benchmark-v1
(209 questions) with 17,002 deduplicated questions across 15 legal domains.
Overview
Property
Value
Questions
17,002 (deduplicated)
Years
2012–2026
Source papers
1,040 official exam papers
Format… See the full description on the dataset page: https://huggingface.co/datasets/lianghsun/tw-legal-benchmark-v2.nasa-science-repos-sme-benchmark
NASA Science Repos SME Benchmark
A benchmark dataset for evaluating retrieval systems on NASA science repository discovery tasks. This dataset contains expert queries, a corpus of NASA science GitHub repositories, and relevance judgments.
Dataset Structure
Files
├── corpus.jsonl # 5,264 repositories with full metadata
├── queries.jsonl # 219 expert queries
└── qrels/
├── earth.tsv # Earth Science relevance judgments (162)
├──… See the full description on the dataset page: https://huggingface.co/datasets/nasa-impact/nasa-science-repos-sme-benchmark.kdv-rag-benchmark
KDV RAG Benchmark
A retrieval benchmark dataset for Turkish VAT (KDV, Katma Değer Vergisi) law — built by adding retrieval layers one at a time (chunking, model choice, hybrid search, reranking, query rewriting, historical/date filtering) and statistically validating each one individually (see Results).
Dataset structure
Splits
Split
Records
Period
train
728
2018-2023
test
154
2024-2026
Split strategy: temporal — train and test… See the full description on the dataset page: https://huggingface.co/datasets/dokukoza/kdv-rag-benchmark.enterprise-llm-inference-benchmarks-2026
🚀 Enterprise LLM Inference & Fine-Tuning Benchmarks (2026 Guide)
A curated benchmark index and architectural guide evaluating open-source foundation models, real-time inference engines (vLLM vs. TensorRT-LLM), and cloud GPU economics for enterprise deployments.
🧠 Open-Source Foundation Model Benchmarks (RAG & Code Generation)
Flagship Evaluation: Top Open-Source LLMs for Enterprise RAG & Code Generation (2026 In-Depth Guide) — Comparing Qwen 2.5 Coder, Llama… See the full description on the dataset page: https://huggingface.co/datasets/Abdulrahmankalil/enterprise-llm-inference-benchmarks-2026.Multilingual-BenchmarkThese are the GSM8K and ARC dataset translated by Google Translate.
BibTex
@misc{lu2024languagecountslearnunlearn,
title={Every Language Counts: Learn and Unlearn in Multilingual LLMs},
author={Taiming Lu and Philipp Koehn},
year={2024},
eprint={2406.13748},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2406.13748},
}
Benchmark
Description
The document describes the LiveRAG benchmark.
For more details regarding Q&A generation see [1,2].
The LiveRAG benchmark includes 895 questions:
500 questions from Session 1, 500 questions from Session 2, with 105 shared questions from both Sessions
A total of 895 unique questions
Benchmark Fields
Field name
Description
Type
Remarks
Index
Benchmark index
int64 [0,1,...,894]
Question
DataMorgana question
String
Answer
DataMorgana ground… See the full description on the dataset page: https://huggingface.co/datasets/LiveRAG/Benchmark.memdelta-benchmark
Dataset Card for MemDelta Benchmark Results
What this release is (and is not)
This is not a new benchmark dataset. The questions, conversations, and ground-truth
answers come from LongMemEval-S, which is publicly available and remains the citation
of record for the underlying data. This release contains two things that do not exist
elsewhere: (1) per-question evaluation results for every (question, memory strategy,
answer model) run in the MemDelta paper… See the full description on the dataset page: https://huggingface.co/datasets/memdelta-bench/memdelta-benchmark.odia-eval-benchmark
Odia Eval Benchmark
Dataset Summary
odia-eval-benchmark is a comprehensive, curated evaluation benchmark for Odia (Oriya) natural language understanding and generation. It consolidates 34 publicly available Odia datasets into a single, normalized format covering 7 task families and 121,947 evaluation rows.
This benchmark was built from authoritative sources with three major improvements:
Curated sources - 8 datasets were re-pulled from their… See the full description on the dataset page: https://huggingface.co/datasets/MaelisResearch/odia-eval-benchmark.ecommerce-ai-data-analyst-agent-benchmark
E-commerce AI Data Analyst Agent Benchmark
A synthetic e-commerce dataset for evaluating AI data analyst agents on
realistic, multi-step business analysis, data-quality investigation, and
analytical reasoning.
This dataset is part of the
E-commerce AI Data Analyst Agent Benchmark.
Dataset summary
This dataset supports evaluation of AI data analyst agents on realistic,
multi-step e-commerce analysis.
It contains:
customers.csv
products.csv
orders.csv
returns.csv… See the full description on the dataset page: https://huggingface.co/datasets/Omcrec/ecommerce-ai-data-analyst-agent-benchmark.mea-benchmark
MEA-Benchmark
A benchmark dataset for evaluating explainability of neural network models across three modalities (tabular, vision, text) with ten question types (Q1–Q10).
Dataset Structure
Each split (train, test) is organized by modality and then by {dataset}_{model}_{q_type}.json:
{split}/
├── tabular/
│ ├── adult_2layernn_q1.json
│ ├── adult_tabnn_q1.json
│ ├── cancer_2layernn_q1.json
│ ├── cancer_tabnn_q1.json
│ └── ...
├── text/
│ ├──… See the full description on the dataset page: https://huggingface.co/datasets/EstherrrCheng/mea-benchmark.soul-benchmarks-locomo
soul.py LoCoMo Benchmark Results
Benchmark results for soul.py on the LoCoMo long-conversation memory benchmark.
Benchmarks repo: github.com/menonpg/soul-benchmarksInteractive results: menonpg.github.io/soul-benchmarks
What is soul.py?
soul.py is an open-source conversational memory layer for LLM agents. It provides multiple retrieval backends (BM25, Qdrant vector search, Relational Learning Model) and an auto-router that selects the best strategy per query.… See the full description on the dataset page: https://huggingface.co/datasets/pgmenon/soul-benchmarks-locomo.quantum-error-mitigation-and-benchmarking
Neura Parse — Quantum Error Mitigation, Characterization & Benchmarking
A pre-fault-tolerance, code-backed vertical on getting trustworthy answers from noisy hardware and rigorously measuring device quality: error-mitigation techniques, characterization/tomography protocols, and benchmarking suites. Runnable Mitiq, pyGSTi, and Qiskit Experiments pipelines with honest sampling-overhead and bias/variance accounting — the practitioner and research toolkit the general dataset… See the full description on the dataset page: https://huggingface.co/datasets/Neura-parse/quantum-error-mitigation-and-benchmarking.emission-factor-benchmark
Emission-Factor Accuracy Benchmark
3,299 rows. Five frontier models answering identical factual questions, with
ground truth traced to a named document and an exact cell — plus the same
questions re-run with a lookup tool, and a second study on which data vendors
those models recommend unprompted.
Collected 10 September 2026. Models: claude-opus-5, gpt-5.5,
gemini-3.1-pro-preview, gemini-3.6-flash, grok-4.6. All answers were
produced through each provider's API with no tools and… See the full description on the dataset page: https://huggingface.co/datasets/greencalculus/emission-factor-benchmark.tw-legal-benchmark-v2
Taiwan Legal Benchmark v2
A multiple-choice benchmark for evaluating LLMs on Taiwan law in Traditional Chinese,
built from 15 years (2012–2026) of national examinations published by the
Ministry of Examination (考選部).
Supersedes tw-legal-benchmark-v1
(209 questions) with 17,002 deduplicated questions across 15 legal domains.
Overview
Property
Value
Questions
17,002 (deduplicated)
Years
2012–2026
Source papers
1,040 official exam papers
Format… See the full description on the dataset page: https://huggingface.co/datasets/Jel1f1sh/tw-legal-benchmark-v2.rag-hallucination-benchmark
RAG Hallucination Benchmark
Context
Retrieval-Augmented Generation (RAG) is the industry standard for reducing LLM hallucinations, but detecting when a RAG system fails is a massive challenge. Most existing benchmarks focus only on massive Deep Learning models and lack tabular features.
This dataset provides a clean, engineered setup to train models (from XGBoost to RoBERTa) to detect hallucinations, predict context faithfulness, and measure answer relevance.… See the full description on the dataset page: https://huggingface.co/datasets/vkshdev/rag-hallucination-benchmark.turkish-seo-reasoning-benchmark-results
Turkish SEO Reasoning Benchmark Results
Bu dataset, Turkish SEO Reasoning benchmark'ının altı farklı model/checkpoint üzerinde çalıştırılmış ham tahminlerini, metriklerini ve tekrar üretim manifestlerini içerir.
Fine-tuned model: berkbirkan/gemma-3-1b-turkish-seo-reasoning-lora
Sonuç
Fine-tuned Gemma 3 1B modeli 22,23 skorla ilk sırada yer aldı. Aynı base model 11,96 skor elde etti.
Mutlak artış: +10,28 puan
Göreli artış: %85,97
Fine-tuned model hata sayısı:… See the full description on the dataset page: https://huggingface.co/datasets/berkbirkan/turkish-seo-reasoning-benchmark-results.llm-benchmark-suite
Tokonomix LLM Benchmark Suite
Version 0.1 · 2026-07-27 · 36,799 runs · 256 models · 11 providers
Authors: Mes Kalkan (InterIP Networks BV) · Tokonomix Editorial Team
Live dashboards: https://tokonomix.ai
Methodology: https://tokonomix.ai/en/benchmarks/methodology
Data license: CC-BY-4.0 · Code excerpts: MIT
Abstract
Tokonomix is an independent, open benchmark platform that evaluates large language
models across six task categories (coding, reasoning, creative… See the full description on the dataset page: https://huggingface.co/datasets/tokonomix/llm-benchmark-suite.pedagogy-benchmark-multilingual
Pedagogy Benchmark - Multilingual
A multilingual translation of the AI-for-Education/pedagogy-benchmark dataset into African languages.
Dataset Description
This dataset provides translations of Chilean teacher training exam questions into African languages. The original dataset contains multiple-choice questions covering various pedagogical domains, education levels, and subject areas.
Languages
Luganda (Uganda)
Nyankore (Uganda)
Coming soon: Swahili… See the full description on the dataset page: https://huggingface.co/datasets/CraneAILabs/pedagogy-benchmark-multilingual.