Modelpublic

Faris-Faiz/Malaysian_Gemma3_270M_F16

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face

0likes14downloads

Model Card

MMLU Benchmark

                       Model   Accuracy   shot by_letter        category
0  Malaysian_Gemma3_270M_F16  43.143676  0shot      True            STEM
1  Malaysian_Gemma3_270M_F16  43.861323  0shot      True        Language
2  Malaysian_Gemma3_270M_F16  44.246892  0shot      True  Social science
3  Malaysian_Gemma3_270M_F16  43.367714  0shot      True          Others
4  Malaysian_Gemma3_270M_F16  49.260523  0shot      True      Humanities
{'Social science': np.int64(6918), 'Language': np.int64(6288), 'Humanities': np.int64(4395), 'Others': np.int64(4169), 'STEM': np.int64(2443)}
Model : Malaysian_Gemma3_270M_F16
Metric : first
Shot : 0shot
average accuracy 44.794118861768474
accuracy for STEM 43.143675808432256
accuracy for Language 43.86132315521628
accuracy for Social science 44.246892165365715
accuracy for Others 43.367714080115135
accuracy for Humanities 49.26052332195677

Uploaded finetuned model

—Developed by: Faris-Faiz
—License: apache-2.0
—Finetuned from model : unsloth/gemma-3-270m-it

dataset was fine-tuned on split (split="force_malay")

This gemma3_text model was trained 2x faster with Unsloth and Huggingface's TRL library.