theophilusowiti/Caracal_instruct

<!-- This model card has been generated according to the AfriLLMQuant pilot project telemetry report. You should probably proofread and complete it, then remove this comment. -->

Caracal_instruct (Pre-release)

Model Description

Caracal_instruct is an instruction-tuned model, produced as part of the AfriLLMQuant pilot project. It was trained via full Quantization-Aware Training (QAT).

The underlying small language model (SLM) backbone for this QAT process was Inkuba-0.4B, continued-pretrained on African-language data and then instruction-tuned to produce Caracal_instruct. The recommended use of this model is for fine-tuning on specific task.

Authors & Acknowledgments

Author

• —Theophilus Lincoln Owiti — Carnegie Mellon University Africa

Acknowledgments

We gratefully acknowledge [OpenToken](https://opentoken.global) and their infrastructure partner [Leafcloud](https://www.leaf.cloud/) for providing the GPU compute resources that made training this model possible.

Citation

If you use this model or its outputs in your research, please cite:

@misc{owiti2026caracal,
  title        = {Caracal Instruct},
  author       = {Owiti, Theophilus Lincoln},
  year         = {2026},
  publisher    = {Hugging Face},
  journal      = {Hugging Face Repository},
  howpublished = {\url{[https://huggingface.co/theophilusowiti/Caracal_instruct](https://huggingface.co/theophilusowiti/Caracal_instruct)}},
  institution  = {Carnegie Mellon University Africa},
  note         = {GPU compute provided by OpenToken and Leafcloud}
}

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "theophilusowiti/Caracal_instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

prompt = "<s><Input>\nWho is the president of Kenya?\n</Input>\n<Answer>\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Expected output

The model expects the <Input>...</Input> / <Answer>...</Answer> instruction format used during SFT, e.g.:

<Input>
Who is the president of Kenya?
</Input>
<Answer>
 William Ruto

William Ruto</Answer>

Top Performing Languages

Based on instruction following and PPL, these languages conform to instruction, especially when used with context/RAG:

East Africa: Swahili (swa), Amharic (amh), Luganda (lug), Kinyarwanda (kin)

West Africa: Hausa (hau), Yoruba (yor), Igbo (ibo)

Central Africa: Lingala (lin)

Southern Africa: Xhosa (xho)

Training Procedure

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 5e-05
• —trainbatchsize: 16
• —evalbatchsize: 16
• —seed: 42
• —gradientaccumulationsteps: 2
• —totaltrainbatch_size: 32
• —optimizer: Adam (betas=(0.9, 0.999), epsilon=1e-08)
• —lrschedulertype: cosine
• —lrschedulerwarmup_ratio: 0.03
• —num_epochs: 3

Training results

Framework versions

• —Transformers 4.45.0
• —Pytorch 2.12.0+cu126
• —Datasets 4.8.5
• —Tokenizers 0.20.3 ---