kruokruo/nllb-200-kapampangan-en

Got it 👍 Thanks for sharing your Wav2Vec2 model card format — I’ll now draft your NLLB fine-tuned model card in the same style and structure for consistency.

Model Card for NLLB-200-Distilled-600M-Kapampangan-English

This model is a fine-tuned version of facebook/nllb-200-distilled-600M for machine translation between Kapampangan and English. It was trained on a custom dataset of 1,685 sentence pairs and achieves competitive results on a held-out test split. The model supports bidirectional translation between Kapampangan (pam) and English (en).

Model Details

Model Description

• —Developed by: Sean Almendral
• —Model type: Machine Translation (NMT)
• —Languages (NLP): Kapampangan (kap), English (en)
• —License: Apache-2.0
• —Finetuned from model: facebook/nllb-200-distilled-600M
• —Pipeline tag: translation

Model Sources

• —Repository: Hugging Face Model Card
• —Paper \[base model]: No Language Left Behind: Scaling Human-Centered Machine Translation

Uses

Direct Use

• —Translating between Kapampangan ↔ English.
• —Educational and cultural preservation of Kapampangan.
• —Baseline translation model for further fine-tuning.

Downstream Use

• —Integration into ASR + MT pipelines (e.g., Kapampangan speech → English text).
• —Language learning tools, chatbots, or educational apps.

Out-of-Scope Use

• —Not suited for medical, legal, or safety-critical translations.
• —May produce inaccurate outputs for low-resource domains or idiomatic expressions.

Bias, Risks, and Limitations

• —Dataset size is relatively small (1,685 pairs), so coverage is limited.
• —May not generalize well to informal speech, slang, or rare words.
• —Risk of bias toward dataset domains (conversational & general sentences).
• —Limited exposure to code-switching between Kapampangan and English.

Recommendations

• —Validate translations before sensitive use.
• —Extend training with larger and domain-specific corpora for production systems.

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("kruokruo/nllb-200-kapampangan-en")
model = AutoModelForSeq2SeqLM.from_pretrained("kruokruo/nllb-200-kapampangan-en")

text = "Ali ku balu"
inputs = tokenizer(text, return_tensors="pt")

translated_tokens = model.generate(
    **inputs,
    forced_bos_token_id=tokenizer.lang_code_to_id["eng_Latn"]
)
translation = tokenizer.decode(translated_tokens[0], skip_special_tokens=True)

print(translation)  # "I didn’t know"

Training Details

Training Data

• —Custom dataset of 1,685 parallel Kapampangan–English pairs.
• —Preprocessing included lowercasing and punctuation normalization.

Training Procedure

• —Base model: facebook/nllb-200-distilled-600M
• —Optimizer: AdamW
• —Learning rate: 1e-5
• —Epochs: 5 (with Early Stopping)
• —Train/Validation split: 80/20 (seed=42)

Training Hyperparameters

• —Precision: fp32
• —Batch size: GPU-dependent (with gradient accumulation)
• —Scheduler: Linear decay with warmup

Evaluation

Testing Data

• —20% held-out split from the dataset.

Metrics

• —Training Loss: 1.3471 → 0.0322
• —Validation Loss: ↓ 1.3589 → 1.2495 (epoch 2), ↑ 1.5447 (epoch 5)
• —SacreBLEU: ↑ 16.6973 → 29.6054

Sample Prediction

• —Source: malyari meng gisyan ing kahun ba yang ibusni.
• —Prediction: The box was about to be removed.
• —Reference: you can tear the box open.

BLEU Breakdown (test):

• —BLEU = 0.4046
• —Precisions = [61.79%, 42.34%, 35.20%, 31.20%]
• —BP = 0.9826
• —Translation length = 2,387
• —Reference length = 2,429

SacreBLEU Breakdown (test):

• —SacreBLEU = 40.46
• —Counts = [1475, 868, 603, 430]
• —Totals = [2387, 2050, 1713, 1378]
• —Precisions = [61.79%, 42.34%, 35.20%, 31.20%]
• —BP = 0.9826 ---

Environmental Impact

• —Hardware Type: NVIDIA L4 GPU (Google Colab Pro)
• —Training Duration: \[fill in if tracked]
• —Energy Consumed: \~104.87 gCO2e (estimated using ML Impact calculator)
• —Energy Needed: \~257.03 Wh
• —Cloud Provider: Google Colab Pro

Technical Specifications

Model Architecture and Objective

• —Architecture: NLLB-200-Distilled-600M (600M parameters)
• —Objective: Fine-tuned for supervised sequence-to-sequence translation

Compute Infrastructure

• —Framework: PyTorch + Hugging Face Transformers
• —Trainer: Hugging Face Seq2SeqTrainer API

Citation

@misc{nllb-kapampangan-en,
  title={NLLB-200-Distilled-600M-Kapampangan-English: Machine Translation Model},
  author={Sean Almendral},
  year={2025},
  howpublished={\url{https://huggingface.co/kruokruo/nllb-200-kapampangan-en}},
}

Model Card Authors

Sean Almendral

Model Card Contact

📧 seansimonea@gmail.com