Modelpublic

Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2

sourceHugging Facellama3.1updated 20d agoView on Hugging Face

0likes756downloads

Model Card

🇨🇬 Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2

A Meta-Llama-3.1-8B-Instruct model adapted to Lingala through supervised fine-tuning (QLoRA/LoRA), developed by Congo Digital Services (CDS). This model is intended for conversation, text generation, summarization, translation, and classification in Lingala.

🔗 Sister model: QLoRA adapters only are available at `Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-adapters`

Model Details

Model Description

—Developed by: Congo Digital Services (CDS SARL) — congo-digital.com
—Base model: meta-llama/Meta-Llama-3.1-8B-Instruct
—Adaptation method: Supervised fine-tuning via QLoRA/LoRA
—Language: Lingala (ln)
—Model type: Causal language model, 8B parameters, merged
—License: Llama 3.1 Community License
—Functional objectives: Conversation, text generation, summarization, translation, classification, and conversational responses tailored to Lingala

Model Sources

—Organization: Congo-digital-service on Hugging Face
—Contact: info@congo-digital.com

Uses

Direct Use

This model can be used directly for conversational text generation in Lingala via the standard transformers API, or deployed behind a compatible inference server (see Deployment Infrastructure below).

Out-of-Scope Use

This model is not intended for use cases requiring critical factual accuracy (medical, legal domains) without human oversight, nor for certified professional translation tasks.

How to Get Started with the Model

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

messages = [
    {"role": "user", "content": "Loba na ngai na lingala : ndenge nini okoki kosala mombongo na Kinshasa ?"}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Training Data

A unified corpus of 4,818 examples across five stylistic categories:

Category	Examples
Urban	1,425
Educational	1,063
Summarization	1,041
Formal	889
Translation	400

Split: stratified 80/20 split → 3,854 initial training examples / 964 test examples (unaugmented, held out for independent evaluation).

Augmentation: applied only to the training set to balance categories at 1,140 examples each, resulting in a final training set of 5,700 examples.

Training Procedure

—Environment: Google Colab, with checkpoints and run logs saved to Google Drive

Training Hyperparameters

Parameter	Value
Epochs	3
Batch size	2
Gradient accumulation steps	8 (effective batch = 16)
Initial learning rate	2 × 10⁻⁵
Scheduler	Cosine
Intermediate evaluations	Every 200 steps

Training Convergence

Step	Training Loss	Validation Loss	Entropy	Mean Token Accuracy
200	1.5638	1.5747	1.6141	67.15%
400	1.3008	1.3514	1.3425	69.81%
600	1.2714	1.2901	1.3146	70.83%
800	1.2072	1.2741	1.2669	71.14%
1000	1.2211	1.2720	1.2690	71.15%
1071	1.1956	1.2720	1.2689	71.19%

Validation loss decreases steadily throughout training with no sign of overfitting.

Evaluation

Evaluation performed on the independent test set (964 examples), held out during training.

Final Results

Metric	Base Model	Fine-tuned Model	Improvement
ROUGE-L F1 (primary metric)	0.1521	0.2379	+56.4%
Mean Token Accuracy	—	71.20%	—
Token F1 (lexical)	0.1721	—	—
Perplexity	—	17.7211	—
ROUGE-1	—	0.2939	—
ROUGE-2	—	0.0789	—
BLEU (SacreBLEU for base)	2.3280	0.0459	not representative*
Exact match	0.00%	not computed	—

\ BLEU proves unrepresentative for open-ended generation tasks; progress is primarily measured via ROUGE-L.*

Summary

The QLoRA adaptation delivers a robust, measurable performance gain: the fine-tuned model handles Lingala's linguistic structures significantly better than the base model, with a +56.4% relative gain in ROUGE-L F1.

Human Evaluation

—Operational target: ≥ 85% of responses rated acceptable (per the project's Terms of Reference)
—Dimensions evaluated: Lingala correctness, coherence and meaning, instruction adherence, business relevance

Target Deployment Infrastructure

Component	Detail
Inference server	vLLM deployed on GPU pods
Gateway / auth	LiteLLM v1.87.0 + PostgreSQL 16
Endpoint	`POST ${VLLM_BASE_URL}/v1/chat/completions`
Format	OpenAI Chat Completions compatible (`choices[0].message.content`)
Access interface	LoBAI (LibreChat-based) with Keycloak / OpenID Connect

Traceability

—Merged model: Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2
—QLoRA adapters: `Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-adapters`
—Version manifest and checksums (SHA-256): centralized in the project's delivery GitHub repository

Bias, Risks, and Limitations

This model was trained on a moderately sized corpus (5,700 augmented examples) covering five stylistic registers. Performance may vary outside these registers, particularly on regional Lingala dialects not represented in the corpus, or on specialized technical domains absent from the training set. Users should validate model outputs before any high-stakes use.

Environmental Impact

Training was performed on Google Colab infrastructure. Carbon emissions can be estimated using the ML Impact calculator (Lacoste et al., 2019).

Citation

BibTeX:

bibtex

@misc{cds2026lingala,
  title={Llama-3.1-8B-Instruct-Lingala-QLoRA},
  author={Congo Digital Services},
  year={2026},
  howpublished={\url{https://huggingface.co/Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2}}
}

Contact

—Organization: Congo Digital Services (CDS SARL)
—Website: congo-digital.com
—Email: info@congo-digital.com