Bateesa/tiny-aya-global-lora-qa

🌍 Multilingual SLM — Ateso · Luganda · English · Runyankore · Japadhola

A lightweight, multilingual Small Language Model (SLM) fine-tuned for question-and-answer tasks across five languages spoken in Uganda and East Africa. Built on top of CohereLabs/tiny-aya-global using LoRA (PEFT), this model is optimized for low-resource, local-language understanding.

Model Details

Supported Languages

Intended Use

✅ Direct Use

This model is designed for question-and-answer inference in multilingual East African contexts. It is suitable for:

• —Building local-language chatbots and virtual assistants
• —Educational tools for Ugandan language communities
• —Research into low-resource NLP for African languages
• —Prototyping QA systems before scaling to larger datasets

🔧 Downstream Use

The model can be further fine-tuned or integrated into:

• —Mobile or web-based community knowledge bases
• —Agricultural, health, or civic information systems in local languages
• —Language learning applications

❌ Out-of-Scope Use

• —High-stakes or safety-critical applications without additional evaluation
• —Languages not covered in training (the model may produce low-quality outputs)
• —Tasks beyond question-answering (e.g., code generation, summarization) without further fine-tuning

How to Get Started

Installation

pip install transformers peft torch

Inference

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model_id = "CohereLabs/tiny-aya-global"
adapter_id = "Bateesa/tiny-aya-global-lora-qa" 

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(base_model_id)
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

def ask(question: str) -> str:
    prompt = f"Question: {question}\nAnswer:"
    inputs = tokenizer(prompt, return_tensors="pt")
    outputs = model.generate(**inputs, max_new_tokens=128)
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

# English
print(ask("What is the capital of Uganda?"))

# Luganda
print(ask("Ekibuga ekikulembera Uganda kye ki?"))

# Runyankore
print(ask("Obwakabaka bw'Uganda nibuki?"))

Training Details

Training Data

• —Dataset size: 90K custom QA pairs
• —Format: Instruction-style prompt/response pairs (Question: ... \nAnswer: ...)
• —Languages: Balanced across Ateso, Luganda, English, Runyankore, and Japadhola
• —Source: Manually curated domain-specific questions and answers relevant to East African contexts

Training Procedure

Fine-tuned using LoRA (Low-Rank Adaptation) via the HuggingFace PEFT library on top of CohereLabs/tiny-aya-global.

Training Hyperparameters

Evaluation

Testing Data

Held-out subset from the 90K custom QA samples, with manual review of responses across all five languages.

Metrics

• —Qualitative review: Human evaluation of answer relevance and fluency per language
• —BLEU / ROUGE: Planned for future evaluation with expanded dataset

Results

⚠️ This model is trained on a small dataset of 90 samples. Performance may vary across languages and domains. It is best used as a baseline or proof-of-concept. Expanding the training dataset is strongly recommended for production use.

Bias, Risks, and Limitations

• —Small dataset (90K samples): The model may hallucinate or give incorrect answers, particularly for rare or complex questions.
• —Language imbalance: If training samples were not evenly distributed, some languages may perform better than others.
• —Cultural context: The model may not capture nuanced cultural meanings or idiomatic expressions in all five languages.
• —No safety fine-tuning: This model has not been RLHF-tuned or filtered for harmful outputs.

Recommendations

Users should validate model outputs before deploying in community-facing applications. Additional data collection and evaluation is recommended, especially for Ateso and Japadhola which have fewer NLP resources available.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact Calculator.

Citation

If you use this model in your research or application, please cite:

@misc{multilingual-slm-ug,
  title     = {Multilingual SLM for Ugandan Languages: Ateso, Luganda, English, Runyankore, Japadhola},
  author    = {PhosAI},
  year      = {2025},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/Bateesa/tiny-aya-global-lora-qa}
}

Model Card Contact

For questions, feedback, or collaboration inquiries, please open an issue on the model repository or contact [your contact info].

Framework Versions

• —PEFT 0.18.1
• —Transformers ≥ 4.38.0
• —PyTorch ≥ 2.0