Modelpublic

AliAbdelrasheed/maqa_llama_4bit_GGUF

sourceHugging Faceupdated 7mo agoView on Hugging Face

0likes42downloads

Model Card

MAQA-LLaMA-4bit-GGUF — Arabic Medical Q&A (CPU-Compatible)

⚠️ Disclaimer: This model is intended for research and informational purposes only. It is not a substitute for professional medical advice, diagnosis, or treatment. It cannot and should not be used to prescribe or recommend medications.

Model Summary

maqa_llama_4bit_GGUF is the CPU-compatible GGUF quantised variant of `maqa_llama` — a Llama 3 8B model fine-tuned on 430,000 real Arabic doctor-patient interactions across 20 medical specialisations. Designed for local deployment without a GPU.

Property	Value
Base model	`unsloth/llama-3-8b-Instruct-bnb-4bit` (Meta Llama 3 8B Instruct)
Fine-tuned from	`maqa_llama`
Quantisation	GGUF q4km
Model size	8B parameters
File size	4.92 GB (Q4KM) · 16.1 GB (F16)
Language	Arabic 🇸🇦
License	Apache 2.0
Developed by	Ali Abdelrasheed

Model Family

Model	Format	Size	Best for
`maqa_llama`	BF16 SafeTensors	~16 GB	Research / further fine-tuning
`maqa_llama_4bit`	4-bit (bitsandbytes)	~5 GB	GPU inference
`maqa_llama_4bit_GGUF` ← this model	GGUF q4km	4.92 GB	✅ CPU / local deployment

Quantisation Options

Format	Size	Notes
Q4_K_M	4.92 GB	✅ Recommended — best balance of quality and file size. Uses Q6K for key attention layers, Q4K elsewhere.
F16	16.1 GB	Higher fidelity, requires significantly more RAM

Quick Start

llama.cpp

bash

# Install llama.cpp (if not already installed)
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && make

# Download the model
huggingface-cli download AliAbdelrasheed/maqa_llama_4bit_GGUF --local-dir ./maqa_gguf

# Run inference
./llama-cli -m ./maqa_gguf/maqa_llama_4bit_GGUF-Q4_K_M.gguf \
  --color \
  -p "ما هي أسباب ارتفاع ضغط الدم وكيف يمكن علاجه؟" \
  -n 256

Ollama

bash

ollama run hf.co/AliAbdelrasheed/maqa_llama_4bit_GGUF

LM Studio

Search for AliAbdelrasheed/maqa_llama_4bit_GGUF in the LM Studio model browser and download the Q4KM variant.

Dataset — MAQA

Property	Value
Total records	430,000 question-answer pairs
Sources	altibbi.com · tbeeb.net · cura.healthcare
Specialisations	20 medical fields
Language	Modern Standard Arabic
Split used for training	70% train / 30% evaluation

Dataset: "Deep learning for Arabic healthcare: MedicalBot" — Springer (2023) Harvard Dataverse

Training Details

Fine-tuned using QLoRA (rank 16, alpha 32) via the Unsloth framework on Google Colab Pro. Full training hyperparameters available on the base model card.

GGUF conversion was performed using Unsloth's native push_to_hub_gguf method with quantization_method = "q4_k_m".

System Prompt Used During Training

أنت طبيب محترف ولديك خبرة في كل مجالات الطب.
يجيب على أسئلة المرضى حول الأمراض، باستخدام لهجة رسمية وودية،
وإجابات موجزة ومفيدة يسهل على الجميع فهمها.

Limitations

—Not a substitute for professional medical advice or clinical diagnosis
—Cannot prescribe or recommend medications
—Optimised for Modern Standard Arabic; dialectal performance may vary
—Trained on a sampled subset of MAQA
—Web-scraped data may contain noise or outdated medical information

Developed By

Ali Abdelrasheed — Graduation Project Nile University · B.Sc. Information Technology – Big Data · Class of 2024 🤗 HuggingFace Profile