AmplifiedAccess/Swahili-gemma-1b-it

Swahili Gemma 1B IT

A fine-tuned version of Google Gemma 3 1B, continued-pretrained and instruction-tuned for English → Swahili translation and general Swahili instruction-following.

Also available as a quantized Ollama release (ollama run amplifiedaccessorg/Swahili-gemma-1b-it).

Highlights

• —BLEU 28.86 on English→Swahili FLORES-200 translation — up from 0.41 on the base model
• —chrF++ 54.77 — up from 14.28 on the base model
• —1B parameters — small enough to run on consumer hardware
• —Trained with QLoRA (4-bit NF4 base + LoRA adapters), then merged to full weights

Results

FLORES-200 devtest, English→Swahili, beam search (beam=8), 256 max new tokens:

Fine-tuning took the base model from essentially no working Swahili translation ability to a functional, fluent translator on both metrics.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "AmplifiedAccess/Swahili-gemma-1b-it"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.float16, device_map="auto")

prompt = ("<start_of_turn>user\n"
          "Translate the following English sentence to Swahili.\n"
          "English: The children are playing outside.\nSwahili:<end_of_turn>\n"
          "<start_of_turn>model\n")

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False)

response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response.strip())

Recommended prompt format

The model was trained on this exact translation template — using it gets the most reliable output:

Translate the following English sentence to Swahili.
English: <your sentence>
Swahili:

It was also trained on general Swahili instruction data (Aya, Alpaca-Swahili), so it can follow other Swahili-language instructions, but translation quality is best with the format above.

Example translations (tested against the deployed model)

Training details

Pipeline

Two-stage training on Kaggle (T4 GPU):

1. 1.Stage 1 — Continued pretraining on Swahili data, producing an intermediate CPT-merged checkpoint (base for this stage).
2. 2.Stage 2 — Supervised fine-tuning (this release): QLoRA on top of the CPT-merged checkpoint, then merged to full weights.

Data (Stage 2 SFT)

Packed into 25,452 blocks of 512 tokens (~13.0M training tokens).

Configuration

Release format

This repo contains the merged fp16 weights. A quantized GGUF (q4KM, ~814 MB) is available via Ollama.

Limitations

• —Back-translation augmentation was planned but did not execute in this training run — the model did not benefit from that data.
• —Trained and evaluated only on English→Swahili translation; Swahili→English capability is untested and not claimed.
• —Performance outside the training domains (news, general instructions) is unverified.
• —As a 1B model, expect more errors on long, syntactically complex, or highly domain-specific sentences than from larger translation models.

Intended use

• —English → Swahili machine translation
• —Swahili instruction-following / conversational use
• —Research on low-resource African language NLP at small parameter scale

Acknowledgments

• —Masakhane for the MAFAND-MT parallel corpus
• —Sunbird AI for the SALT dataset
• —Cohere For AI for the Aya dataset
• —Google DeepMind for the Gemma model family

Framework versions

• —PyTorch 2.10.0
• —Transformers 5.0.0
• —PEFT (LoRA)

Citation

@misc{swahili-gemma-1b-2026,
  title={Swahili Gemma 1B IT: Fine-tuned Gemma 3 1B for English-Swahili Translation},
  author={Amplified Access},
  year={2026},
  publisher={HuggingFace},
  url={https://huggingface.co/AmplifiedAccess/Swahili-gemma-1b-it}
}