Modelpublic

Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-adapters

sourceHugging Facellama3.1updated 19d agoView on Hugging Face

0likes26downloads

Model Card

Llama 3.1 8B Instruct Lingala QLoRA Adapters

Model Description

These are the LoRA adapter weights obtained by fine-tuning meta-llama/Llama-3.1-8B-Instruct on Lingala instruction-following data using QLoRA (4-bit quantized base model + low-rank adapters). This repository contains adapter weights only — it must be loaded on top of the base Llama 3.1 8B Instruct model using the peft library; it is not a standalone model.

A fully merged, ready-to-use version of this same fine-tune is available at `Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-merged-v2`, for users who prefer not to manage base model + adapter loading themselves.

Built with Llama

This model was created by fine-tuning Meta-Llama-3.1-8B-Instruct using QLoRA. Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. Use of this model is subject to the Llama 3.1 Community License Agreement.

Intended Use

Instruction-following / instruction-tuning in Lingala: the model learns to follow an instruction written in Lingala and produce a Lingala response, adapted to five task categories — formal, pedagogical, summary, urban, and translation — across five public-interest domains (agriculture, governance, health, informal sector, education).

Training Data

Fine-tuned on `Congo-digital-service/Dataset-Lingala-Alpaca-LLM` (6,664 instruction/response examples in Lingala; 5,700 train / 964 test). An augmented version of this dataset is available at `Congo-digital-service/Lingala-Alpaca-Dataset-Augmented`.

How to Use

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_id = "meta-llama/Llama-3.1-8B-Instruct"
adapter_id = "Congo-digital-service/Llama-3.1-8B-Instruct-Lingala-QLoRA-adapters"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(base_model_id, device_map="auto")
model = PeftModel.from_pretrained(base_model, adapter_id)

License

The adapter weights in this repository are released under the same Llama 3.1 Community License as the base model (see meta-llama/Llama-3.1-8B-Instruct license) — use of this model requires acceptance of that license.

In addition, because this model was fine-tuned on data released under the Nwulite Obodo Open Data License (NOODL-1.0), the following attribution requirement — carried over from the training dataset's license — applies to this model as well:

This model was created by Congo Digital Services (CDS SARL) (https://www.congo-digital.com/) using training data licensed under the Nwulite Obodo Open Data License (https://licensingafricandatasets.com/nwulite-obodo-license). Users from high-income countries or commercial entities are required to publicly acknowledge and credit the Maloba Project (UNDP Republic of Congo — language digitalisation initiative) in any publication, product, model, or output derived from this model. To fulfil this requirement, contact contact@congo-digital.com or visit https://www.congo-digital.com/contact.

Considerations for Using the Model

This model was fine-tuned on a modest dataset (6,664 examples) for a low-resource language — users should expect limited lexical and stylistic coverage outside the five domains covered by the training data.