Neobe/dhivehi-en-qwen3-4b-lora-paragraph

Qwen3-4B Dhivehi→English (paragraph-level)

Qwen3-4B (decoder-only) + LoRA adapter for Dhivehi→English translation, trained on the multi-granularity Dhivehi–English parallel corpus (gated).

Qwen-based paragraph model — higher quality, heavier. For a lighter, faster option, see the mT5 paragraph variant. Handles both single sentences and multi-sentence paragraphs.

Scores (chrF / chrF++ / BLEU)

chrF is the metric to trust for Thaana; BLEU is unreliable (word segmentation / morphology).

Example

Input (dv): އެއީ، މިދިޔަ އަހަރުގެ މި މުއްދަތާ ބަލާއިރު، 7.6 އިންސައްތައިގެ ކުރިއެރުމެއް ކަމަށްވާއިރު، ދުވާލަކަށް 7،778 ފަތުރުވެރިން ރާއްޖެ ޒިޔާރަތްކުރެއެވެ. Output (en): While this is a growth of 7.6 percent compared to the same period last year, an average of 7,778 tourists visit the Maldives daily.

Multi-sentence input (dv): ނާޝިދާ މުޙައްމަދަކީ ދިވެހި ފިލްމީ ތަރިއެކެވެ. އޭނާވަނީ ދިވެހި ތަފާތު އެތައް ފިލްމްތަކެއްގައި ހަރަކާތްތެރި ވެފައެވެ. މީގެ އިތުރުން ތަފާތު ވީޑިއޯ ލަވަތަކާއި، ޓީވީ ޑްރާމާ ސިލްސިލާ ތަކުން ވެސް ނާއްކޮގެ ހުނަރު ބެލުންތެރިންނަށް ފެނިގެން ގޮސްފައި ވެއެވެ. އަމިއްލަ ދިރިއުޅުން ނާޝިދާ މުޙައްމަދު އަކީ ހދ. ކުޅުދުއްފުއްޓަށް އުފަން ބަތަލާއެކެވެ. މިހާރު ޒަވާޖީ ޙަޔާތެއް ވަނީ ފަށާފައެވެ. ފިލްމީ ދާއިރާއަށް ނިކުތުން ނާޝިދާ މުޙައްމަދު ފިލްމީ ދާއިރާ އަށް ނިކުތީ ފިލްމީ ތަރި އަޙުމަދު އާޞިމް އާއެކު ކުޅުނު ވީޑިއޯ ލަވަޔަކުންނެނެވެ. އެއަށްފަހު ތަފާތު ޑްރާމާ ، ފިލްމް، ސިލްސިލާ ތަކުގައި ހަރަކާތްތެރި ވެފައި ވެއެވެ. Output (en): Nashida Mohamed is a Maldivian film star. She has performed in many different Maldivian films. In addition, Nashida's talent was seen by audiences through various video songs and TV drama series as well. Personal Life: Nashida Mohamed is a actress born in HDh. Kulhudhuffushi. She has now started her married life. Entering the Film Industry: Nashida Mohamed entered the film industry with a video song she sang for actor Ahmed Asim. After that, she acted in various dramas, films, and television series.

Real held-out sample and this model's own output.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base, adapter = "Qwen/Qwen3-4B", "Neobe/dhivehi-en-qwen3-4b-lora-paragraph"
tok = AutoTokenizer.from_pretrained(adapter)
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained(base, torch_dtype=torch.bfloat16, device_map="cuda"),
    adapter).eval()

src = "ދިވެހިރާއްޖޭގެ ރައީސް މިއަދު ކެބިނެޓާ ބައްދަލުކުރެއްވި އެވެ."
msgs = [{"role":"user","content":
    "Translate the following Dhivehi text to English. Output only the translation, "
    f"no explanations.\n\nDhivehi: {src}\nEnglish:"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inp = tok(prompt, return_tensors="pt", truncation=True, max_length=3072).to("cuda")
out = model.generate(**inp, max_new_tokens=512, num_beams=1, repetition_penalty=1.15, no_repeat_ngram_size=3, do_sample=False)
print(tok.decode(out[0][inp["input_ids"].shape[1]:], skip_special_tokens=True))

~9–10 GB VRAM (bf16). Qwen tokenizes Thaana at ~1.8 tokens/char — don't over-truncate the input.

Training

Base Qwen/Qwen3-4B; LoRA r=16, α=32, targets q/k/v/o+gate/up/down; bf16; adamwtorch LR 2e-4 cosine; maxlength 1536; 1 epoch; effective batch ~32; gradient checkpointing.

Limitations

Domain = Maldivian news / press / Wikipedia; technical or informal English is out of distribution. Non-human references are machine-generated (distillation).

Citation

@misc{neobe_dhivehi_en_qwen3_4b_lora_paragraph_2026,
  title  = {Qwen3-4B Dhivehi→English (paragraph-level)},
  author = {Neobe},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/Neobe/dhivehi-en-qwen3-4b-lora-paragraph}}
}