shanjivkr/catla-nllb-bn2en

catla-nllb-bn2en

A LoRA fine-tune of facebook/nllb-200-distilled-600M for Bengali→English (bn2en) translation, trained on tweet-length text as part of the CATLA Bengali↔English translation project.

Model description

Frozen NLLB-200-distilled-600M base model with a LoRA adapter (rank 16) applied to the attention and feed-forward projections, fine-tuned to translate short, informal Bengali social-media text (tweets) into English. This is one of three fine-tuned engines (alongside IndicTrans2 and BanglaT5) used in CATLA's candidate-ensemble translation pipeline.

Intended uses & limitations

Intended for short-form, informal Bengali social-media text (tweets: slang, code-mixing, emoji, hashtags). Not evaluated for formal/long-form text, domains outside social media, or languages other than Bengali↔English.

Training and evaluation data

Trained on the bn_en_translation subset of the CATLA Bengali-English tweets dataset, filtered to drop rows exceeding 500 characters (removes ~0.1% of rows identified as scraped, mismatched non-tweet content) and rows with corrupted/mojibake Bengali text.

Two training passes exist for this adapter, and this checkpoint reflects the second, incomplete one:

• —An earlier pass completed one full epoch over 150,000 rows (2026-08-15).
• —This checkpoint is from a second pass over the full 572,703-row dataset (one epoch target), which was interrupted at step 16,000 of ~17,859 (~90% complete) by a Kaggle GPU session time limit, not a training failure. The most recent saved state is what's published here.

Training procedure

Training hyperparameters

• —learning_rate: 2e-4
• —trainbatchsize: 8
• —gradientaccumulationsteps: 4 (effective batch size: 32)
• —optimizer: AdamW (torch fused), betas=(0.9, 0.999), epsilon=1e-8
• —lrschedulertype: linear
• —num_epochs (target): 1.0 — actual: ~0.90 (interrupted, see above)
• —mixedprecisiontraining: fp16 (Native AMP)
• —LoRA: r=16, alpha=32, dropout=0.05, targetmodules=[qproj, kproj, vproj, out_proj, fc1, fc2]

Training results

Not included here: this checkpoint's own step-by-step loss curve was not fully captured due to the session interruption noted above, and the loss table from the earlier 150,000-row run does not describe this checkpoint's actual training — publishing it here would misrepresent this specific adapter's provenance.

Evaluation results

Not yet independently evaluated as this specific (full-dataset, ~90%-trained) checkpoint. The earlier 150,000-row version of this adapter scored BLEU 24.68 / chrF++ 44.09 / TER 74.20 on the CATLA bn2en test set (vs. 17.55 / 39.22 / 90.95 for the untuned pretrained baseline) — re-evaluation of this checkpoint is pending.

Framework versions

• —PEFT 0.19.1
• —Transformers 5.0.0
• —Pytorch 2.10.0+cu128
• —Datasets 5.0.0
• —Tokenizers 0.22.2