CoolFace
Modelpublic

jaifar/qwen3-4b-oman-qlora

sourceHugging Faceapache-2.0updated 5d agoView on Hugging Face
0likes43downloads
Model Card

Qwen3-4B Oman — QLoRA Adapter | مُحوِّل Qwen3-4B العُماني

A LoRA adapter that fine-tunes Qwen/Qwen3-4B (Apache 2.0) on a bilingual Arabic/English instruction dataset about the Sultanate of Oman, built from Wikipedia by Menasaat.

مُحوِّل LoRA يُخصِّص نموذج Qwen3-4B على مجموعة بيانات تعليمية ثنائية اللغة (العربية والإنجليزية) عن سلطنة عُمان.

Model Details

  • —Base model: Qwen/Qwen3-4B (Apache 2.0)
  • —Method: QLoRA — 4-bit NF4 quantization, LoRA r=16, alpha=32, dropout 0.05, all attention + MLP projections
  • —License: Apache 2.0 (inherited from base; this adapter is a derivative work with attribution)
  • —Trained by: Menasaat | Virtual Platforms LLC on Kaggle free-tier GPUs (Tesla T4, fp16)

Training Data

menasaat/oman-wikipedia-corpus — 3,926 instruction pairs (60 held out for eval): "Tell me about X" → lead section, and "More details about X" → article passages, in Arabic and English.

Training Results

MetricValue
Training time140.1 minutes (1× Tesla T4)
Training loss6.33 → 3.91
Eval loss (held-out)4.00

Evaluation: Base vs Fine-tuned (2026-09-21)

MetricBase Qwen3-4Bmenasaat fine-tuneChange
Mean NLL, 60 held-out Oman pairs (AR/EN)3.142.01−36.1%
Train loss (3,926 pairs)—6.33 → 3.91—

Full side-by-side answers (20 questions, AR+EN) in eval_results.json. Example — "أخبرني عن ولاية صلالة." (Tell me about Salalah): the base model located Salalah in Malaysia; the fine-tune correctly places it in the Sultanate of Oman. Generations remain imperfect (verify details against primary sources), but Oman grounding is materially improved.

Usage

python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B")
model = PeftModel.from_pretrained(base, "menasaat/qwen3-4b-oman-qlora")
tok = AutoTokenizer.from_pretrained("menasaat/qwen3-4b-oman-qlora")

Limitations

Pilot run (2 epochs). Answers are grounded in Wikipedia text about Oman and reflect its coverage and biases; verify facts against primary sources.

About the publisher | عن الناشر

Menasaat | Virtual Platforms LLC is an enterprise AI company serving Oman, the GCC, and the wider MENA region. We build Arabic-first AI solutions: open data about the Sultanate of Oman, fine-tuned Arabic/English large language models (LLMs), retrieval (RAG) systems, and production AI for enterprises and the public sector. Explore our work at menasaat.com and on Hugging Face.

منصات افتراضية ش.م.م (Menasaat) هي شركة ذكاء اصطناعي مؤسسي تخدم سلطنة عُمان ودول مجلس التعاون الخليجي ومنطقة الشرق الأوسط وشمال أفريقيا. نبني حلول ذكاء اصطناعي عربية أولاً: بيانات مفتوحة عن سلطنة عُمان، ونماذج لغوية كبيرة (LLM) مُدرَّبة بالعربية والإنجليزية، وأنظمة استرجاع معزّز بالتوليد (RAG)، وحلول ذكاء اصطناعي إنتاجية للمؤسسات والقطاع العام.