mmlu-prox
mmlu-prox-eval-predictions
MMLU-ProX Multilingual Model Predictions
Raw per-sample model predictions on MMLU-ProX
across 29 languages and 25 open-weight LLMs, produced with
lm-evaluation-harness.
This dataset releases the full prediction logs (not just aggregate scores) so that
item-level responses can be re-analysed — e.g. for Item Response Theory (IRT) modelling
of multilingual benchmarks, error analysis, or per-item difficulty estimation.
Repository structure
mmlu_prox_<lang>/
└──… See the full description on the dataset page: https://huggingface.co/datasets/gililior/mmlu-prox-eval-predictions.MMLU-ProX
MMLU-ProX
MMLU-ProX is a multilingual benchmark that builds upon MMLU-Pro, extending to 29 typologically diverse languages, designed to evaluate large language models' reasoning capabilities across linguistic and cultural boundaries.
Github | Paper
News
[2025/08] 🎉 MMLU-ProX was accepted by EMNLP 2025 Main Conference!
[2025/05] MMLU-ProX now contains 29 languages, all available on Huggingface.
[2025/03] MMLU-ProX is now available on Huggingface.
[2025/03] We are still… See the full description on the dataset page: https://huggingface.co/datasets/li-lab/MMLU-ProX.MMLU-ProX-Lite
MMLU-ProX-Lite
MMLU-ProX is a multilingual benchmark that builds upon MMLU-Pro, extending to 29 typologically diverse languages, designed to evaluate large language models' reasoning capabilities across linguistic and cultural boundaries.
Github | Paper
News
[2025/08] 🎉 MMLU-ProX was accepted by EMNLP 2025 Main Conference!
[2025/05] MMLU-ProX now contains 29 languages, all available on Huggingface.
[2025/03] MMLU-ProX is now available on Huggingface.
[2025/03] We are… See the full description on the dataset page: https://huggingface.co/datasets/li-lab/MMLU-ProX-Lite.MMLU-ProX-JapaneseMMLU-ProX-EnglishMMLU-ProX
MMLU-ProX
MMLU-ProX is a multilingual benchmark that builds upon MMLU-Pro, extending to 29 typologically diverse languages, designed to evaluate large language models' reasoning capabilities across linguistic and cultural boundaries.
Github | Paper
News
[2025/08] 🎉 MMLU-ProX was accepted by EMNLP 2025 Main Conference!
[2025/05] MMLU-ProX now contains 29 languages, all available on Huggingface.
[2025/03] MMLU-ProX is now available on Huggingface.
[2025/03] We are still… See the full description on the dataset page: https://huggingface.co/datasets/JudSacr/MMLU-ProX.
