Modelpublic

kelvinmbewe/nyanja_english_warmup

sourceHugging Facecc-by-4.0updated 4mo agoView on Hugging Face

0likes15downloads

Model Card

Nyanja → English Encoder Warm-Up Model

This is an intermediate "warm-up" checkpoint, not a finished translation system. It is `Helsinki-NLP/opus-mt-ny-en` whose encoder has undergone masked-language-model (MLM) adaptation on monolingual Nyanja (Chichewa) text, with the decoder frozen. The goal is to adapt the encoder's understanding of Nyanja before a downstream supervised fine-tuning stage on parallel data.

Important: This model has only seen monolingual Nyanja during this stage. Monolingual adaptation does not, on its own, teach the Nyanja↔English mapping, so this checkpoint should not be expected to have higher translation quality (BLEU/chrF++) than the base model. It is intended to be fine-tuned on parallel ny–en data before evaluation or use.

Model details


Base model	`Helsinki-NLP/opus-mt-ny-en` (MarianMT)
Direction	Nyanja (Chichewa) → English
Warm-up objective	Encoder masked language modeling (mask 15%, predict masked tokens only)
Trainable	Encoder only — decoder frozen
Vocabulary	59,812 (native 59,811 + one added `<mask>` token; embeddings resized)
Hidden size	512
Parameters	~75M total (~19.2M trained)

Vocab note / integration gotcha: because MLM needs a mask token (MarianMT has none), a <mask> token was added and the embeddings resized to 59,812. This model is therefore not drop-in compatible with a vanilla opus-mt-ny-en (59,811). When you fine-tune or load it downstream, use this repo's tokenizer so the sizes match.