datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
mmlu
Dataset Card for MMLU
Dataset Summary
Measuring Massive Multitask Language Understanding by Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt (ICLR 2021).
This is a massive multitask test consisting of multiple-choice questions from various branches of knowledge. The test spans subjects in the humanities, social sciences, hard sciences, and other areas that are important for some people to learn. This covers 57… See the full description on the dataset page: https://huggingface.co/datasets/cais/mmlu.MMLU-Pro
MMLU-Pro Dataset
MMLU-Pro dataset is a more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines.
|Github | 🏆Leaderboard | 📖Paper |
🚀 What's New
[2026.03.11] Added more cutting-edge frontier models to the leaderboard, including the Claude-4.6 series, Seed2.0 series, Qwen3.5 series, and Gemini-3.1-Pro… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro.mmlu_no_trainThis dataset contains a copy of the cais/mmlu HF dataset but without the auxiliary_train split that takes a long time to generate again each time when loading multiple subsets of the dataset.
Please visit https://huggingface.co/datasets/cais/mmlu for more information on the MMLU dataset.
mmluMMLU (hendrycks_test on huggingface) without auxiliary train. It is much lighter (7MB vs 162MB) and faster than the original implementation, in which auxiliary train is loaded (+ duplicated!) by default for all the configs in the original version, making it quite heavy.
We use this version in tasksource.
Reference to original dataset:
Measuring Massive Multitask Language Understanding - https://github.com/hendrycks/test
@article{hendryckstest2021,
title={Measuring Massive Multitask Language… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/mmlu.mmlu-redux-2.0
Dataset Card for MMLU-Redux-2.0
MMLU-Redux is a subset of 5,700 manually re-annotated questions across 57 MMLU subjects.
News
[2025.02.25] We corrected one annotation in Abstract Algebra subset, as noted in the Issue #2.
[2025.02.08] We corrected one annotation in High School Mathematics subset, as noted in the PlatinumBench paper.
[2025.01.23] MMLU-Redux is accepted to NAACL 2025!
Dataset Details
Dataset Description
Each data point in… See the full description on the dataset page: https://huggingface.co/datasets/edinburgh-dawg/mmlu-redux-2.0.mmlu-prox-eval-predictions
MMLU-ProX Multilingual Model Predictions
Raw per-sample model predictions on MMLU-ProX
across 29 languages and 25 open-weight LLMs, produced with
lm-evaluation-harness.
This dataset releases the full prediction logs (not just aggregate scores) so that
item-level responses can be re-analysed — e.g. for Item Response Theory (IRT) modelling
of multilingual benchmarks, error analysis, or per-item difficulty estimation.
Repository structure
mmlu_prox_<lang>/
└──… See the full description on the dataset page: https://huggingface.co/datasets/gililior/mmlu-prox-eval-predictions.Global-MMLU-Lite
Releases:
Version 3.0 (May 2026): GMMLU Lite 3.0 release with 5 new languages: Czech, Hungarian, Italian (updated), Oriya, Slovak and Tajik
Version 2.0 (Dec 2025): GMMLU Lite 2.0 release with 3 new languages: Albanian, Burmese and Welsh
Version 1.0 (Dec 2024): GMMLU Lite initial release with 15 languages.
Dataset Summary
Global-MMLU-Lite is a multilingual evaluation set spanning 23 languages, including English. It is "lite" version of the original Global-MMLU… See the full description on the dataset page: https://huggingface.co/datasets/CohereLabs/Global-MMLU-Lite.Global-MMLU
Dataset Summary
Global-MMLU 🌍 is a multilingual evaluation set spanning 42 languages, including English. This dataset combines machine translations for MMLU questions along with professional translations and crowd-sourced post-edits.
It also includes cultural sensitivity annotations for a subset of the questions (2850 questions per language) and classifies them as Culturally Sensitive (CS) 🗽 or Culturally Agnostic (CA) ⚖️. These annotations were collected as part of an open… See the full description on the dataset page: https://huggingface.co/datasets/CohereLabs/Global-MMLU.mmlu
Dataset Card for MMLU
Dataset Summary
Measuring Massive Multitask Language Understanding by Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt (ICLR 2021).
This is a massive multitask test consisting of multiple-choice questions from various branches of knowledge. The test spans subjects in the humanities, social sciences, hard sciences, and other areas that are important for some people to learn. This covers 57 tasks… See the full description on the dataset page: https://huggingface.co/datasets/lighteval/mmlu.mmlu_greek
Dataset Card for MMLU Greek
The MMLU Greek dataset is a set of 15858 examples from the MMLU dataset [available from here and here], machine-translated into Greek. The original dataset consists of multiple-choice questions from 57 tasks including elementary mathematics, US history, computer science, law, etc.
Dataset Details
Bias, Risks, and Limitations
This dataset is the result of machine translation.
Dataset Card Contact… See the full description on the dataset page: https://huggingface.co/datasets/ilsp/mmlu_greek.mmlu-redux-2.0-okThis dataset is filtered from https://huggingface.co/datasets/edinburgh-dawg/mmlu-redux-2.0 using:
from datasets import load_dataset
from pathlib import Path
CONFIGS = ['abstract_algebra', 'anatomy', 'astronomy', 'business_ethics', 'clinical_knowledge', 'college_biology', 'college_chemistry', 'college_computer_science', 'college_mathematics', 'college_medicine', 'college_physics', 'computer_security', 'conceptual_physics', 'econometrics', 'electrical_engineering', 'elementary_mathematics'… See the full description on the dataset page: https://huggingface.co/datasets/fxmarty/mmlu-redux-2.0-ok.MMLU_ChineseChinese version of MMLU dataset tranlasted by gpt-3.5-turbo.The dataset is used in the research related to MultilingualSIFT.
PromptEval_MMLU_full
MMLU Multi-Prompt Evaluation Data
Overview
This dataset contains the results of a comprehensive evaluation of various Large Language Models (LLMs) using multiple prompt templates on the Massive Multitask Language Understanding (MMLU) benchmark. The data is introduced in
Maia Polo, Felipe, Ronald Xu, Lucas Weber, Mírian Silva, Onkar Bhardwaj, Leshem Choshen, Allysson Flavio Melo de Oliveira, Yuekai Sun, and Mikhail Yurochkin. "Efficient multi-prompt evaluation of LLMs."… See the full description on the dataset page: https://huggingface.co/datasets/PromptEval/PromptEval_MMLU_full.multilingual_mmluMMLU professionally translated into 14 languages using professional human translators, sourced from OpenAI's simple-eval.
Original files:
english: https://openaipublic.blob.core.windows.net/simple-evals/mmlu.csv
multilingual: https://openaipublic.blob.core.windows.net/simple-evals/mmlu_{language}.csv where language one of "AR-XY", "BN-BD", "DE-DE", "ES-LA", "FR-FR", "HI-IN", "ID-ID", "IT-IT", "JA-JP", "KO-KR", "PT-BR", "ZH-CN", "SW-KE", "YO-NG", "EN-US"
PromptEval_MMLU_correctness
MMLU Multi-Prompt Evaluation Data (correctness scores)
Overview
This dataset contains the results of a comprehensive evaluation of various Large Language Models (LLMs) using multiple prompt templates on the Massive Multitask Language Understanding (MMLU) benchmark. The data is introduced in
Maia Polo, Felipe, Ronald Xu, Lucas Weber, Mírian Silva, Onkar Bhardwaj, Leshem Choshen, Allysson Flavio Melo de Oliveira, Yuekai Sun, and Mikhail Yurochkin. "Efficient multi-prompt… See the full description on the dataset page: https://huggingface.co/datasets/PromptEval/PromptEval_MMLU_correctness.MMLU-ProX
MMLU-ProX
MMLU-ProX is a multilingual benchmark that builds upon MMLU-Pro, extending to 29 typologically diverse languages, designed to evaluate large language models' reasoning capabilities across linguistic and cultural boundaries.
Github | Paper
News
[2025/08] 🎉 MMLU-ProX was accepted by EMNLP 2025 Main Conference!
[2025/05] MMLU-ProX now contains 29 languages, all available on Huggingface.
[2025/03] MMLU-ProX is now available on Huggingface.
[2025/03] We are still… See the full description on the dataset page: https://huggingface.co/datasets/li-lab/MMLU-ProX.mmlux
Citation Information
If you find benchmarks useful in your research, please consider citing the test and also the MMLU dataset it draws from:
@misc{thellmann2024crosslingual,
title={Towards Cross-Lingual LLM Evaluation for European Languages},
author={Klaudia Thellmann and Bernhard Stadler and Michael Fromm and Jasper Schulze Buschhoff and Alex Jude and Fabio Barth and Johannes Leveling and Nicolas Flores-Herr and Joachim Köhler and René Jäkel and Mehdi Ali}… See the full description on the dataset page: https://huggingface.co/datasets/Eurolingua/mmlux.MMLU-ProX-Lite
MMLU-ProX-Lite
MMLU-ProX is a multilingual benchmark that builds upon MMLU-Pro, extending to 29 typologically diverse languages, designed to evaluate large language models' reasoning capabilities across linguistic and cultural boundaries.
Github | Paper
News
[2025/08] 🎉 MMLU-ProX was accepted by EMNLP 2025 Main Conference!
[2025/05] MMLU-ProX now contains 29 languages, all available on Huggingface.
[2025/03] MMLU-ProX is now available on Huggingface.
[2025/03] We are… See the full description on the dataset page: https://huggingface.co/datasets/li-lab/MMLU-ProX-Lite.m_mmlu
Multilingual MMLU
Dataset Summary
This dataset is a machine translated version of the MMLU dataset.
The Icelandic (is) part was translated with Miðeind's Greynir model and Norwegian (nb) was translated with DeepL. The rest of the languages was translated using GPT-3.5-turbo by the University of Oregon, and this part of the dataset was originally uploaded to this Github repository.
loom-benchmark-mmlu-prommluMeasuring Massive Multitask Language Understanding by Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt (ICLR 2021).mmlu-redux
Dataset Card for MMLU-Redux
[!TIP]
Please consider using MMLU-Redux-2.0 which contains all 57 MMLU subjects.
MMLU-Redux is a subset of 3,000 manually re-annotated questions across 30 MMLU subjects.
News
[2025.02.08] We corrected one annotation in High School Mathematics subset, as noted in the PlatinumBench paper.
[2025.01.23] MMLU-Redux is accepted to NAACL 2025!
Dataset Details
Dataset Description
Each data point in MMLU-Redux contains… See the full description on the dataset page: https://huggingface.co/datasets/edinburgh-dawg/mmlu-redux.mmluThis is a massive multitask test consisting of multiple-choice questions from various branches of knowledge, covering 57 tasks including elementary mathematics, US history, computer science, law, and more.MMLU-Pro-CoT-Train-43KMMLU-SR
MMLU-SR Dataset
This is the dataset for the paper "MMLU-SR: A Benchmark for Stress-Testing Reasoning Capability of Large Language Models".
Dataset Structure
This dataset contains three different variants:
Question Only: Key terms in questions are replaced with dummy words and their definitions, while answer choices remain unchanged.
Answer Only: Key terms in answer choices are replaced with dummy words and their definitions, while questions remain unchanged.
Question… See the full description on the dataset page: https://huggingface.co/datasets/NiniCat/MMLU-SR.mmlu_clinical_knowledgemmlu_pro_leaderboard_submissionmmlu_professional_medicinehuman_translated_arabic_mmlummlu-random-A