bazik-0/indictrans2-en-kas-finetuned-200M-5

indictrans2-en-kas-finetuned-200M-5

A LoRA fine-tune of IndicTrans2 (distilled 200M, en-indic) for English → Kashmiri translation, with Kashmiri output in the Perso-Arabic script (kas_Arab).

Model description

IndicTrans2 is AI4Bharat's open-source neural MT model covering English and 22 scheduled Indian languages. This checkpoint takes the distilled 200M en→indic variant and adapts it specifically to Kashmiri using LoRA adapters trained on the kas_Arab portion of the BPCC parallel corpus, merged back into the base weights.

• —Base model: ai4bharat/indictrans2-en-indic-dist-200M
• —Fine-tuning method: LoRA (PEFT), merged into base weights
• —Source language: English (eng_Latn)
• —Target language: Kashmiri, Perso-Arabic script (kas_Arab)
• —License: Apache 2.0

Intended use

Sentence-level English → Kashmiri translation. Intended for research, low-resource MT experimentation, and applications needing Kashmiri text output in the Perso-Arabic script. Not intended for high-stakes use (legal, medical) without human review — see Limitations below.

How to use

This model requires `IndicTransToolkit` for the pre/post-processing IndicTrans2 expects (language tags, script normalization).

pip install transformers accelerate sentencepiece
git clone https://github.com/VarunGumma/IndicTransToolkit.git
pip install --editable ./IndicTransToolkit

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
from IndicTransToolkit.IndicTransToolkit import IndicProcessor

MODEL_ID = "bazik-0/indictrans2-en-kas-finetuned-200M-5"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
model = AutoModelForSeq2SeqLM.from_pretrained(MODEL_ID, trust_remote_code=True, torch_dtype=torch.float32)
model.eval()

ip = IndicProcessor(inference=True)

sentences = ["Hello, how are you?", "The weather is very cold today."]
processed = ip.preprocess_batch(sentences, src_lang="eng_Latn", tgt_lang="kas_Arab")

inputs = tokenizer(processed, return_tensors="pt", padding=True, truncation=True, max_length=128)

with torch.no_grad():
    outputs = model.generate(**inputs, max_length=128, num_beams=5, early_stopping=True)

decoded = tokenizer.batch_decode(outputs, skip_special_tokens=True)
translations = ip.postprocess_batch(decoded, lang="kas_Arab")

print(translations)

Example outputs

Training details

Training data

Fine-tuned on `ai4bharat/BPCC` (bpcc-seed-latest config), kas_Arab subset — 90,000 training pairs and 8,929 held-out eval pairs, shuffled with seed 42.

Preprocessing

Source and target text is passed through IndicTransToolkit's IndicProcessor for IndicTrans2's language-tag formatting and script normalization before tokenization. Sequences are truncated to a max length of 128 tokens.

Training procedure

LoRA adapters (via PEFT) trained on the attention projections with the base model frozen, then merged into the base weights for release:

• —rank r=16, alpha=32, dropout 0.05
• —target modules: q_proj, v_proj

Hyperparameters:

Compute: single Kaggle T4 GPU, via 🤗 Seq2SeqTrainer.

Evaluation

Metric: BLEU (sacrebleu), computed on the held-out eval split.

Result: 6.07 BLEU on the target evaluation set.

Limitations

• —Kashmiri is a low-resource language in NLP — training data volume and diversity are limited compared to high-resource pairs, so translation quality varies notably by sentence complexity and domain.
• —Trained only on kas_Arab (Perso-Arabic script); it does not support Kashmiri written in Devanagari.
• —Not evaluated for bias, toxicity, or factual accuracy — apply human review for any use beyond research/experimentation.

Citation

If you use this model, please cite the base IndicTrans2 work:

@article{gala2023indictrans2,
  title   = {IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages},
  author  = {Jay Gala and Pranjal A. Chitale and A K Raghavan and Varun Gumma and Sumanth Doddapaneni and Aswanth Kumar M and Janki Atul Nawale and Anupama Sujatha and Ratish Puduppully and Vivek Raghavan and Pratyush Kumar and Mitesh M. Khapra and Raj Dabre and Anoop Kunchukuttan},
  journal = {arXiv preprint arXiv:2305.16307},
  year    = {2023}
}

Model card contact

Open a discussion on this model's Hugging Face repo for questions.