Datasetpublic

mesolitica/Speech-Instructions-MalayMMLU

Speech Instructions MalayMMLU We filter MalayMMLU that suitable to convert to voice prompt and generate synthetic voice using mesolitica/Malaysian-Dia-1.6B also verified with Force Alignment to make sure the pronunciations almost correct. We highly recommend to not include this dataset in training set to prevent contamination. This dataset supposely to become a speech QA benchmark. Benchmark gpt-4o-audio-preview-2025-06-03 Social science… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/Speech-Instructions-MalayMMLU.

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes22downloads

Dataset Card

Speech Instructions MalayMMLU

We filter MalayMMLU that suitable to convert to voice prompt and generate synthetic voice using mesolitica/Malaysian-Dia-1.6B also verified with Force Alignment to make sure the pronunciations almost correct.

We highly recommend to not include this dataset in training set to prevent contamination. This dataset supposely to become a speech QA benchmark.

Benchmark

gpt-4o-audio-preview-2025-06-03

Social science 61.188170635959175
Language 55.92151366503153
Humanities 70.65217391304348
Others 62.21198156682027
STEM 64.76043276661515

mesolitica/Malaysian-Qwen2.5-7B-Speech-Instruct

Social science 53.3
Language 51.7
Humanities 57.4
Others 58.0
STEM 54.8

mesolitica/Malaysian-Qwen2.5-7B-Speech-Instruct + Reasoning

Social science 58.8
Language 61.5
Humanities 66.9
Others 62.7
STEM 65.5

Source code

Source code at https://github.com/mesolitica/malaysian-dataset/tree/master/speech-instruction/malaymmlu