Modelpublic

pranavgupta/gemma4-creole-mt-2026-baseline-adapter

sourceHugging Faceupdated 4mo agoView on Hugging Face

1likes266downloads

Model Card

Gemma-4 E2B — Creole MT Baseline Adapter

LoRA adapter fine-tuned on Kreyol-MT (Robinson et al., NAACL 2024), covering 86 language pairs across 50 Creole and related languages.

Results (dev set, 10 epochs, 8× L40S)

Metric	Base Gemma-4	+ This adapter	Δ
Macro BLEU	7.19	18.48	+11.29
Macro chrF	20.90	36.94	+16.04
Corpus BLEU	5.57	18.48	+12.91
Corpus chrF	21.82	38.48	+16.66

Training details

—Base model: google/gemma-4-E2B
—Method: LoRA (r=64, α=128, target=all-linear)
—Training data: Kreyol-MT public split — 86 language pairs, temperature-sampled (T=5, NLLB-style) to 50k examples per epoch
—Epochs: 10
—Precision: bf16
—Hardware: 8× NVIDIA L40S

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

model_id = "google/gemma-4-E2B"
adapter_id = "pranavgupta/gemma4-creole-mt-2026-baseline-adapter"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16).cuda()
model = PeftModel.from_pretrained(model, adapter_id)
model = model.merge_and_unload()

prompt = "Translate this from Haitian Creole into English:\nBonjou, kijan ou rele?\n### Translation:\n"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=100, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Framework versions

—PEFT 0.19.1