datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
MMLU-Pro
MMLU-Pro Dataset
MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines.
|Github | 🏆Leaderboard | 📖Paper |
🚀 What's New
[2026.03.11] Added more cutting-edge frontier models to the leaderboard, including the Claude-4.6 series, Seed2.0 series, Qwen3.5 series, and Gemini-3.1-Pro… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro.mmlu-prox-eval-predictions
MMLU-ProX Multilingual Model Predictions
Raw per-sample model predictions on MMLU-ProX
across 29 languages and 25 open-weight LLMs, produced with
lm-evaluation-harness.
This dataset releases the full prediction logs (not just aggregate scores) so that
item-level responses can be re-analysed — e.g. for Item Response Theory (IRT) modelling
of multilingual benchmarks, error analysis, or per-item difficulty estimation.
Repository structure
mmlu_prox_<lang>/
└──… See the full description on the dataset page: https://huggingface.co/datasets/gililior/mmlu-prox-eval-predictions.PromptEval_MMLU_full
MMLU Multi-Prompt Evaluation Data
Overview
This dataset contains the results of a comprehensive evaluation of various Large Language Models (LLMs) using multiple prompt templates on the Massive Multitask Language Understanding (MMLU) benchmark. The data is introduced in
Maia Polo, Felipe, Ronald Xu, Lucas Weber, Mírian Silva, Onkar Bhardwaj, Leshem Choshen, Allysson Flavio Melo de Oliveira, Yuekai Sun, and Mikhail Yurochkin. "Efficient multi-prompt evaluation of LLMs."… See the full description on the dataset page: https://huggingface.co/datasets/PromptEval/PromptEval_MMLU_full.PromptEval_MMLU_correctness
MMLU Multi-Prompt Evaluation Data (correctness scores)
Overview
This dataset contains the results of a comprehensive evaluation of various Large Language Models (LLMs) using multiple prompt templates on the Massive Multitask Language Understanding (MMLU) benchmark. The data is introduced in
Maia Polo, Felipe, Ronald Xu, Lucas Weber, Mírian Silva, Onkar Bhardwaj, Leshem Choshen, Allysson Flavio Melo de Oliveira, Yuekai Sun, and Mikhail Yurochkin. "Efficient multi-prompt… See the full description on the dataset page: https://huggingface.co/datasets/PromptEval/PromptEval_MMLU_correctness.EU-MMLU
EU-oriented Massive Multitask Language Understanding (EU-MMLU)
Dataset description
The MMLU benchmarking dataset (https://arxiv.org/pdf/2009.03300) is a widely recognized benchmark of general knowledge attained by AI models. It covers a broad range of topics from 57 different categories, covering elementary-level knowledge up to advanced professional subjects like law, physics, history, and computer science.
The European Commission’s Directorate-General for… See the full description on the dataset page: https://huggingface.co/datasets/EC-DGT-AI/EU-MMLU.mmlu-okapi-eval-es
MMLU translated to Spanish
This dataset was generated by the Natural Language Processing Group of the University of Oregon, where they used the
original MMLU dataset in English and translated it into different languages using ChatGPT.
This dataset only contains the Spanish translation, but the following languages are also covered within the original
subsets posted by the University of Oregon at http://nlp.uoregon.edu/download/okapi-eval/datasets/.
Disclaimer
All the… See the full description on the dataset page: https://huggingface.co/datasets/alvarobartt/mmlu-okapi-eval-es.mmlu-auxiliary-train-10-choices
Dataset Card for Augmented MMLU (STEM) with Additional Distractors
Dataset Description
This dataset is an augmented version of the STEM portion of the MMLU auxiliary training set kz919/mmlu-auxiliary-train-auto-labelled, where each original 4-option multiple-choice question has been expanded to include 10 options (A-J) through the addition of six carefully constructed distractors.
Dataset Summary
Original Dataset: MMLU auxiliary training set (STEM portion)… See the full description on the dataset page: https://huggingface.co/datasets/andresnowak/mmlu-auxiliary-train-10-choices.mmlu_pro_categories
MMLU-Pro Dataset : Per-Category Splits
This dataset was created from TIGER-Lab/MMLU-Pro, by dividing the original dataset into a dataset per different category. The objective is to make it easier to work on sub-categories.
from datasets import load_dataset
ds = load_dataset('RawthiL/mmlu_pro_categories', 'category_biology')
The available tasks are:
Category Name
Split Name
Biology
category_biology
Business
category_business
Chemistry
category_chemistry
Computer… See the full description on the dataset page: https://huggingface.co/datasets/RawthiL/mmlu_pro_categories.MMLU-SemiProThis dataset is derived from TIGER-Lab/MMLU-Pro as part of our MMLU-Leagues Encoder benchmark series, containing:
MMLU-Amateur, where the train set contains all questions Llama-3-8B-Instruct (5-shot) gets wrong and the test set contains all questions it gets right. The aim is to measure the ability of an encoder, with relatively limited training data, to match the performance of a small frontier model.
MMLU-SemiPro (this dataset), where the data is evenly split between a train and a test set.… See the full description on the dataset page: https://huggingface.co/datasets/answerdotai/MMLU-SemiPro.MMLU-Pro
MMLU-Pro Dataset
MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines.
|Github | 🏆Leaderboard | 📖Paper |
🚀 What's New
[2026.03.11] Added more cutting-edge frontier models to the leaderboard, including the Claude-4.6 series, Seed2.0 series, Qwen3.5 series, and Gemini-3.1-Pro, among… See the full description on the dataset page: https://huggingface.co/datasets/lthn/MMLU-Pro.MMLU-Pro評価スコアの再現性確保と SB Intuitions 修正版の公開用クローン
ソース： TIGER-Lab/MMLU-Pro on Hugging Face
MMLU-Pro
MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset
tailored to more rigorously benchmark large language models' capabilities.
This dataset contains 12K complex questions across various disciplines.
Licensing Information
MIT
Citation Information
@misc{wang2024mmlupro,
title={MMLU-Pro: A More Robust and Challenging Multi-Task… See the full description on the dataset page: https://huggingface.co/datasets/sbintuitions/MMLU-Pro.MMLU-Pro_Kazakh_Russian
Dataset Summary
These are the machine-translated Kazakh and Russian versions of the MMLU-Pro (Massive Multitask Language Understanding Pro) dataset (test set).
These datasets are used to test the world knowledge and problem-solving capabilities of large language models across a vast range of subjects in the Kazakh and Russian languages. As an enhanced version of the original MMLU, it serves as a more rigorous benchmark for evaluating how well models understand complex academic… See the full description on the dataset page: https://huggingface.co/datasets/issai/MMLU-Pro_Kazakh_Russian.MMLU-pro-TR
MMLU-Pro Dataset (Turkish)
The MMLU-Pro dataset (TIGER-Lab/MMLU-Pro) is a robust and challenging massive multi-task understanding dataset designed to rigorously benchmark the capabilities of large language models (LLMs). This Turkish-translated version aims to provide a comprehensive evaluation for Turkish language models, addressing inherent challenges and complexities.
Overview
Containing 12,000 complex questions across various disciplines, this dataset was translated… See the full description on the dataset page: https://huggingface.co/datasets/bezir/MMLU-pro-TR.MMLU_ExpertPrompt_RAGThis dataset contains a copy of the cais/mmlu HF dataset but without the auxiliary_train split that takes a long time to generate again each time when loading multiple subsets of the dataset.
Please visit https://huggingface.co/datasets/cais/mmlu for more information on the MMLU dataset.
MMLU-Pro-Stratified
MMLU-Pro-Stratified: A High-Quality & Balanced Teaching-Oriented Testbed
🌟 Definition and Value
MMLU-Pro-Stratified is a meticulously curated subset of MMLU-Pro, specifically designed to serve as a high-quality and balanced teaching-oriented testbed for Large Language Models (LLMs).
Why "Teaching-Oriented"?
Unlike traditional benchmarks that focus on single-turn accuracy, a teaching-oriented testbed evaluates a model's pedagogical capabilities:
Concept… See the full description on the dataset page: https://huggingface.co/datasets/SunriserFuture/MMLU-Pro-Stratified.MMLU-Pro
MMLU-Pro Dataset
MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines.
|Github | 🏆Leaderboard | 📖Paper |
🚀 What's New
[2026.03.11] Added more cutting-edge frontier models to the leaderboard, including the Claude-4.6 series, Seed2.0 series, Qwen3.5 series, and Gemini-3.1-Pro… See the full description on the dataset page: https://huggingface.co/datasets/Vanedap/MMLU-Pro.MMLU-Pro-json
MMLU-Pro json
This is a reupload of MMLU-Pro in json format. Please, refer to the original dataset for details.
MMLU-Pro-ita
MMLU-Pro-ita Dataset Introduction
This is an Italian translation of MMLU-Pro, a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines.
1. What's new about MMLU-Pro
Compared to the original MMLU, there are three major differences:
The original MMLU dataset only contains 4 options, MMLU-Pro increases it to 10… See the full description on the dataset page: https://huggingface.co/datasets/efederici/MMLU-Pro-ita.MMLU-ProX_EN_Cleaned
MMLU-ProX English Cleaned
Dataset Description
This is a cleaned version of the English subset from MMLU-ProX (arXiv:2503.10497),
a comprehensive multilingual benchmark for evaluating large language models. The original MMLU-ProX dataset
contains 11,829 questions across 29 languages, built on the English MMLU-Pro benchmark.
Why This Cleaned Version?
The original English subset of MMLU-ProX contained spacing issues where words were concatenated without
proper… See the full description on the dataset page: https://huggingface.co/datasets/ZQ-Dev/MMLU-ProX_EN_Cleaned.mmlu_hinted_questions
MMLU Hinted Questions
Dataset Description
This dataset contains multiple-choice questions derived from MMLU and augmented with misleading hints. The misleading hints are intentionally designed to point to an incorrect answer.
The dataset was developed as part of the UnfaithRL project, which studies cue-following and unfaithful reasoning under reinforcement learning with verifiable rewards.
Specifically, it was used to investigate whether language models follow… See the full description on the dataset page: https://huggingface.co/datasets/UnfaithRL/mmlu_hinted_questions.ACL-SRW-2025
Dataset Components
The dataset is partitioned into three discrete tables stored in CSV or Parquet format:
Questions
Recipes
Evaluation Results
Each component is described in detail below.
Questions
area
domain
question_number
An integer index uniquely identifying each question inside the knowledge domain.
translation_method
English, Google Translate, GPT-3.5-Turbo, GPT-4o, Human
question
option_a, option_b, option_c, option_d
Recipes
area… See the full description on the dataset page: https://huggingface.co/datasets/Translated-MMLU-Blind-Review/ACL-SRW-2025.MMLU-Pro
MMLU-Pro Dataset
MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines.
|Github | 🏆Leaderboard | 📖Paper |
🚀 What's New
[2026.03.11] Added more cutting-edge frontier models to the leaderboard, including the Claude-4.6 series, Seed2.0 series, Qwen3.5 series, and Gemini-3.1-Pro… See the full description on the dataset page: https://huggingface.co/datasets/quantiles/MMLU-Pro.mmlu-pro-clean
MMLU-Pro-Clean
A corrected drop-in for MMLU-Pro: 12,032 → 10,689 items, with 1,343 broken items removed.
📄 Paper: When the Answer Key Is Wrong — Allcock 2026 · 💻 Source + evidence: github.com/adamallcock/mmlu-pro-clean
True drop-in — identical schema to the original
The default config mirrors TIGER-Lab/MMLU-Pro exactly: same columns (question_id int, question, options, answer, answer_index, cot_content, category, src) and both the test (10,689 cleaned) and… See the full description on the dataset page: https://huggingface.co/datasets/adamallcock/mmlu-pro-clean.MMLU-Pro
MMLU-Pro Dataset
MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines.
|Github | 🏆Leaderboard | 📖Paper |
🚀 What's New
[2026.03.11] Added more cutting-edge frontier models to the leaderboard, including the Claude-4.6 series, Seed2.0 series, Qwen3.5 series, and Gemini-3.1-Pro… See the full description on the dataset page: https://huggingface.co/datasets/chilomax/MMLU-Pro.MMLU-Pro
MMLU-Pro Dataset
MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines.
|Github | 🏆Leaderboard | 📖Paper |
🚀 What's New
[2025.04.06] We corrected 15 answers in medical domain based on the recommendations of medical professionals, thanks to Dr. Robert (Bob) Hoyt and the subspecialists… See the full description on the dataset page: https://huggingface.co/datasets/nezumikozo/MMLU-Pro.MMLU-Philosophy-Marathi
MMLU Philosophy Questions in Marathi
This dataset contains philosophy questions from the MMLU (Massive Multitask Language Understanding) benchmark translated into Marathi.
Dataset Information
Source: MMLU Philosophy subset from cais/mmlu
Translation API: OpenAI GPT-4
Languages: English (original) and Marathi (translated)
Total Questions: 311
Task Type: Multiple choice questions with 4 options each
Dataset Structure
Each row contains:
original_question: The… See the full description on the dataset page: https://huggingface.co/datasets/shubhamugare/MMLU-Philosophy-Marathi.MMLU-pro-TR
Not: Bu veri setinin dokümantasyonu Türk yapay zeka topluluğuna katkı sağlamak amacıyla VeriPazarı tarafından Türkçeye çevrilmiştir. Orijinal veri seti bezir (Abdullah Bezir) tarafından geliştirilmiş olup, VeriPazarı tarafından Türk AI ekosistemi için arşivlenmiştir.
🔗 Orijinal Kaynak: bezir/MMLU-pro-TR
🔗 Derleyen Platform: VeriPazarı
MMLU-Pro Veri Seti (Türkçe)
MMLU-Pro veri seti (TIGER-Lab/MMLU-Pro), büyük dil modellerinin (LLM) yeteneklerini titizlikle ölçmek… See the full description on the dataset page: https://huggingface.co/datasets/Taklaxbr/MMLU-pro-TR.MMLU-Pro
MMLU-Pro Dataset
MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines.
|Github | 🏆Leaderboard | 📖Paper |
🚀 What's New
[2026.03.11] Added more cutting-edge frontier models to the leaderboard, including the Claude-4.6 series, Seed2.0 series, Qwen3.5 series, and Gemini-3.1-Pro, among… See the full description on the dataset page: https://huggingface.co/datasets/khaiise/MMLU-Pro.MMLU-Pro_greekmmlu_mixed_questions
MMLU Mixed Hinted and Unhinted Questions
Dataset Description
This dataset contains multiple-choice questions derived from MMLU and augmented with misleading hints. The misleading hints are intentionally designed to point to an incorrect answer.
The dataset contains a random mixture of:
Hinted examples, where a misleading cue points toward an incorrect answer.
Unhinted examples, where no misleading cue is provided.
The dataset was developed as part of the… See the full description on the dataset page: https://huggingface.co/datasets/UnfaithRL/mmlu_mixed_questions.