Touqeer19/kashmiri-english-nllb-merged

Kashmiri-English Translator (NLLB-200, merged)

A fine-tuned, standalone version of facebook/nllb-200-distilled-600M for English-Kashmiri (Perso-Arabic script) translation. This is the LoRA fine-tuned adapter (Touqeer19/kashmiri-english-nllb-lora) merged into the base model into a single set of weights — no PEFT/adapter loading required to use it.

Model description

• —Base model: facebook/nllb-200-distilled-600M
• —Fine-tuning method: LoRA (r=16, alpha=32, targeting q_proj/v_proj), merged into the base weights in float32 precision
• —Training data: 5,573 pairs (90% split) from AI4Bharat's BPCC nllb-seed configuration, Kashmiri Arabic-script subset

Why a merged version exists

The original LoRA adapter (kashmiri-english-nllb-lora) requires peft to load. This merged version bakes the adapter weights directly into the base model, so it loads with plain transformers — simpler to deploy, and avoids a loading conflict specific to Hugging Face Spaces' ZeroGPU hardware and PEFT's adapter-loading path.

Note: merging was done in float32, not float16 — an earlier float16 merge introduced precision-loss artifacts that degraded output quality. If reproducing this merge, use float32.

Intended use

Experimental English-Kashmiri machine translation. Given the small fine-tuning dataset, this is best understood as a research/portfolio demonstration of a low-resource-language fine-tuning pipeline, not a production-grade translator.

Evaluation

Evaluated on a held-out test set of 310 sentence pairs, unseen during training (evaluation run on the pre-merge LoRA adapter; merging does not change model outputs).

Honest note on these results: the improvement from fine-tuning is modest. NLLB-200 already includes Kashmiri in its pretraining data, so this fine-tune refines an already-competent model on a small dataset rather than teaching it a new language from scratch. Output is fluent and coherent (confirmed via round-trip translation checks), but the quantitative gain over the strong baseline is small.

How to use

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_name = "Touqeer19/kashmiri-english-nllb-merged"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
model.eval()

def translate(text, src_lang="eng_Latn", tgt_lang="kas_Arab"):
    tokenizer.src_lang = src_lang
    inputs = tokenizer(text, return_tensors="pt")
    generated_tokens = model.generate(
        **inputs,
        forced_bos_token_id=tokenizer.convert_tokens_to_ids(tgt_lang),
        max_length=128
    )
    return tokenizer.decode(generated_tokens[0], skip_special_tokens=True)

print(translate("Hello, how are you?"))
# -> ہیلو، تم چِھ کیازِ؟

Important: forced_bos_token_id must be set explicitly to the target language token, or generation defaults to an unrelated language.

Try it live

A public demo is available at Touqeer19/kashmiri-english-translator.

Limitations

• —Small training set (5,573 pairs) drawn from a single domain (Wikipedia-style encyclopedic text via NLLB-Seed) — performance on conversational or domain-specific text is untested and likely weaker.
• —Only Perso-Arabic script (kas_Arab) was used for fine-tuning; Devanagari Kashmiri (kas_Deva) was not included.
• —Sensitive to input casing. ALL CAPS input (e.g. "I WANT TO EAT RICE") produces noticeably degraded output compared to standard sentence casing, since the training data was entirely standard-cased. The public demo Space normalizes all-caps input automatically; this raw model does not.
• —Evaluated with automatic metrics (BLEU/chrF) only; no native-speaker human evaluation has been conducted yet.

License

Apache 2.0 (matching the base NLLB-200 model's license). Training data from AI4Bharat's BPCC is used under its documented terms (CC0 / CC BY 4.0 depending on subset).