vmombo/bloom-560m-fang-translation

BLOOM-560m — French → Fang Translation

Fine-tuned version of bigscience/bloom-560m for translating French into Fang (a Bantu language spoken in Gabon, Equatorial Guinea, and Cameroon), using LoRA (Low-Rank Adaptation).

Usage

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("bigscience/bloom-560m", dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "vmombo/bloom-560m-fang-translation")
tokenizer = AutoTokenizer.from_pretrained("vmombo/bloom-560m-fang-translation")
model.eval()

prompt = "Traduis en fang: Bonjour, comment allez-vous ?\n\nFang:"
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(output[0], skip_special_tokens=True))

Prompt Format

Traduis en fang: <french sentence>

Fang: <fang translation>

Training Details

Training Objective

Only the Fang output tokens receive gradient signal — the French prompt is masked with labels=-100. This focuses 100% of learning on the target language rather than wasting capacity predicting French that BLOOM already knows.

Limitations

Fang is a very low-resource language. This model is an early-stage research prototype and may produce inaccurate translations. Contributions of additional Fang text data are welcome.

Author

Developed by vmombo.