malaymmlu
MalayMMLU
MalayMMLU
Updated on August 18, 2025
Released on September 27, 2024
English |
Bahasa Melayu
📄 Paper •
Code •
📜 Poster
Introduction
MalayMMLU is the first multitask language understanding (MLU) for Malay Language. The benchmark comprises 24,213 questions spanning both primary (Year 1-6) and secondary (Form 1-5) education levels in Malaysia, encompassing 5 broad topics that further divide… See the full description on the dataset page: https://huggingface.co/datasets/UMxYTLAILabs/MalayMMLU.Speech-Instructions-MalayMMLU
Speech Instructions MalayMMLU
We filter MalayMMLU that suitable to convert to voice prompt and generate synthetic voice using mesolitica/Malaysian-Dia-1.6B also verified with Force Alignment to make sure the pronunciations almost correct.
We highly recommend to not include this dataset in training set to prevent contamination. This dataset supposely to become a speech QA benchmark.
Benchmark
gpt-4o-audio-preview-2025-06-03
Social science… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Speech-Instructions-MalayMMLU.MalayMMLU-Eval
VixeroAI — MalayMMLU Evaluation
Benchmark package for evaluating base Qwen 3.8‑27B on the MalayMMLU Malay-language multiple-choice benchmark (24,213 items, 5 categories / 22 subjects).
Headline result — zero-shot recovered accuracy 83.35% (strict first-token 82.67%).
The purpose of this repo is reproducibility and defensibility: every number in the report is derivable from the committed raw results and the committed (dependency-free) harness.
Results
Metric… See the full description on the dataset page: https://huggingface.co/datasets/VixeroAI/MalayMMLU-Eval.MalayMMLU_1MalayMMLU_0Translated-MalayMMLU
Translated-MalayMMLU
Originally from https://huggingface.co/datasets/UMxYTLAILabs/MalayMMLU, we translate to English using ChatGPT 5.1