datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
mmlu-prox-eval-predictions
MMLU-ProX Multilingual Model Predictions
Raw per-sample model predictions on MMLU-ProX
across 29 languages and 25 open-weight LLMs, produced with
lm-evaluation-harness.
This dataset releases the full prediction logs (not just aggregate scores) so that
item-level responses can be re-analysed — e.g. for Item Response Theory (IRT) modelling
of multilingual benchmarks, error analysis, or per-item difficulty estimation.
Repository structure
mmlu_prox_<lang>/
└──… See the full description on the dataset page: https://huggingface.co/datasets/gililior/mmlu-prox-eval-predictions.MMLU-Pro_Llama-3.1-8B-Instruct_gPRM_train
MMLU-Pro_Llama-3.1-8B-Instruct_gPRM_train
MMLU-Pro_Llama-3.1-8B-Instruct_gORM_train
MMLU-Pro_Llama-3.1-8B-Instruct_gORM_train
GlobMed_MMLU-Pro
🌍 GlobMed: MMLU-Pro(Health)
GlobMed_MMLU-Pro (Health) covers 20 languages, including 13 high-resource languages (Arabic, Chinese, English, French, German, Hindi, Indonesian, Japanese, Korean, Portuguese, Russian, Spanish, and Thai) and 7 low-resource languages (Bengali, Malay, Swahili, Urdu, Wolof, Yoruba, and Zulu).
Code
ar
bn
zh
en
fr
de
hi
id
ja
ko
ms
pt
ru
es
sw
th
ur
wo
yo
zu
Language
Arabic
Bengali
Chinese
English
French
German
Hindi
Indonesian
Japanese
Korean… See the full description on the dataset page: https://huggingface.co/datasets/ruiyang-medinfo/GlobMed_MMLU-Pro.MMLU-Pro_Llama-3.1-8B-Instruct_test
MMLU-Pro_Llama-3.1-8B-Instruct_test
MMLU-Pro_Llama-3.1-8B-Instruct_train
MMLU-Pro_Llama-3.1-8B-Instruct_train
MMLU-Pro-Stratified
MMLU-Pro-Stratified: A High-Quality & Balanced Teaching-Oriented Testbed
🌟 Definition and Value
MMLU-Pro-Stratified is a meticulously curated subset of MMLU-Pro, specifically designed to serve as a high-quality and balanced teaching-oriented testbed for Large Language Models (LLMs).
Why "Teaching-Oriented"?
Unlike traditional benchmarks that focus on single-turn accuracy, a teaching-oriented testbed evaluates a model's pedagogical capabilities:
Concept… See the full description on the dataset page: https://huggingface.co/datasets/SunriserFuture/MMLU-Pro-Stratified.MMLU-Pro-json
MMLU-Pro json
This is a reupload of MMLU-Pro in json format. Please, refer to the original dataset for details.
MMLU-Pro_Llama-3.1-70B-Instruct_test
MMLU-Pro_Llama-3.1-70B-Instruct_test
MMLU-Pro_Qwen2.5-7B-Instruct_test
MMLU-Pro_Qwen2.5-7B-Instruct_test
mmlu-pro-clean
MMLU-Pro-Clean
A corrected drop-in for MMLU-Pro: 12,032 → 10,689 items, with 1,343 broken items removed.
📄 Paper: When the Answer Key Is Wrong — Allcock 2026 · 💻 Source + evidence: github.com/adamallcock/mmlu-pro-clean
True drop-in — identical schema to the original
The default config mirrors TIGER-Lab/MMLU-Pro exactly: same columns (question_id int, question, options, answer, answer_index, cot_content, category, src) and both the test (10,689 cleaned) and… See the full description on the dataset page: https://huggingface.co/datasets/adamallcock/mmlu-pro-clean.MMLU-PRO-Leveled-TinyBench
MMLU Pro 难度分级子集 (MMLU Pro Difficulty Subset)
📊 数据集简介基于 MMLU Pro 构建的子数据集，包含 多领域学术问题 及其难度评分。难度值由多个 LLM 模型的回答准确率计算得出（范围 0.0-1.0，数值越小表示难度越高）。
⏬ 适用场景：
LLM 能力评估与对比
难度敏感型模型训练
知识盲点分析
🗂️ 数据集结构
├── data_sets/
│ ├── combined.json # 完整数据集（默认展示）
│ ├── extremely_hard_0.0_0.1.json # LLM 准确率 0-10% (最难)
│ ├── very_hard_0.1_0.2.json # LLM 准确率 10-20%
│ └── ...（共10个难度分级文件）
└── problem_ids/ # 原始 MMLU Pro 题目 ID 映射
📈 难度分级标准… See the full description on the dataset page: https://huggingface.co/datasets/wzzzq/MMLU-PRO-Leveled-TinyBench.MMLU-Pro_gemma-2-9b-it_test
MMLU-Pro_gemma-2-9b-it_test
Qwen3.5-4B-Claude-Opus-Reasoning-Distill-MMLU-Pro-benchmarkBenchmark of TeichAI/Qwen3.5-4B-Claude-Opus-Reasoning-Distill against TIGER-Lab/MMLU-Pro dataset.
Accuracy: 72.89999999999999% with Python tool.
Metric
Value
Correct
729
Incorrect
268
Errors
3
Total samples
1000
Python tool calls
1078
Total completion tokens
2,449,614
Raw stats:
{
"accuracy": 0.729,
"correct": 729,
"incorrect": 268,
"error": 3,
"total": 1000,
"python_tool_calls": 1078,
"completion_tokens":2449614
}
MMLU-Pro_SmolLM3-3B_test
MMLU-Pro_SmolLM3-3B_test
mmlu-professional-medicineMATH_500_MMLU_Pro
Description
The MATH_500 and MMLU_Pro datasets combined in a custom format.
Format for the data
Field
Type
Description
unique_id
string
MD5 hash of the entire record (all other columns), computed on the canonical JSON representation with sorted keys.
question
string
The full question text.
category
string
The subject or category of the question.
choices
array of strings | null
List of answer choices for multiple-choice questions; null for open-ended… See the full description on the dataset page: https://huggingface.co/datasets/HPC-Boys/MATH_500_MMLU_Pro.gpt-oss-20b-MMLU-Pro-benchmarkBenchmark of openai/gpt-oss-20b against TIGER-Lab/MMLU-Pro dataset.
Accuracy: 73.7% with Python tool.
Metric
Value
Correct
7369
Incorrect
2629
Errors
2
Total samples
10000
Python tool calls
5719
Python tool errors
235
Total completion tokens
10,713,598
Raw stats:
{
"accuracy": 0.737,
"correct": 7369,
"incorrect": 2629,
"error": 2,
"total": 10000,
"python_tool_calls": 5719,
"python_tool_errors": 235,
"completion_tokens": 10713598
}
French-MMLU-Professional-Medicine-Benchmark
💻 Dataset Usage
Run the following command to load the testing set:
from datasets import load_dataset
dataset = load_dataset("shuyuej/French-MMLU-Professional-Medicine-Benchmark", split="test")
print(dataset)
MMLU_Pro_etmmlu-professional-accountinggemma-4-E4B-it-MMLU-Pro-benchmarkBenchmark of google/gemma-4-E4B-it against TIGER-Lab/MMLU-Pro dataset.
Accuracy: 69.2% with Python tool.
Metric
Value
Correct
1383
Incorrect
617
Errors
0
Total samples
2000
Python tool calls
235
Python tool errors
11
Total completion tokens
3,328,419
Raw stats:
{
"accuracy": 0.692,
"correct": 1383,
"incorrect": 617,
"error": 0,
"total": 2000,
"python_tool_calls": 235,
"python_tool_errors":11,
"completion_tokens": 3328419
}
Spanish-MMLU-Professional-Medicine-Benchmark
💻 Dataset Usage
Run the following command to load the testing set:
from datasets import load_dataset
dataset = load_dataset("shuyuej/Spanish-MMLU-Professional-Medicine-Benchmark", split="test")
print(dataset)
mmlu-professional-lawMMLU-Pro-samplemmlu-pro_stop_stringsQwen3.5-4B-MMLU-Pro-benchmarkBenchmark of Qwen/Qwen3.5-4B against TIGER-Lab/MMLU-Pro dataset.
Accuracy: 75.5% with Python tool.
Metric
Value
Correct
755
Incorrect
238
Errors
7
Total samples
1000
Python tool calls
1173
Total completion tokens
2,187,462
Raw stats:
{
"accuracy": 0.755,
"correct": 755,
"incorrect": 238,
"error": 7,
"total": 1000,
"python_tool_calls": 1173,
"completion_tokens": 2187462
}
Qwen3.5-9B-MMLU-Pro-benchmarkBenchmark of Qwen/Qwen3.5-9B against TIGER-Lab/MMLU-Pro dataset.
Accuracy: 79.0% with Python tool.
Metric
Value
Correct
790
Incorrect
210
Errors
0
Total samples
1000
Python tool calls
883
Total completion tokens
1,689,029
Raw stats:
{
"accuracy": 0.79,
"correct": 790,
"incorrect": 210,
"error": 0,
"total": 1000,
"python_tool_calls": 883,
"completion_tokens": 1689029
}
Hindi-MMLU-Professional-Medicine-Benchmark
💻 Dataset Usage
Run the following command to load the testing set (272 examples):
from datasets import load_dataset
dataset = load_dataset("shuyuej/Hindi-MMLU-Professional-Medicine-Benchmark", split="test")
print(dataset)
gemma-4-E2B-it-MMLU-Pro-benchmarkBenchmark of google/gemma-4-E2B-it against MMLU-Pro dataset. Model's answer is considered correct if it matches the ground truth answer index exactly.
Accuracy: 61.6% with Python tool.
Metric
Value
Correct
617
Incorrect
381
Errors
3
Total samples
1001
Python tool calls
314
Python tool errors
12
Total completion tokens
1,499,382