manishehehe/whisper-small-hindi-lora

whisper-small-hindi-lora

LoRA adapter for `openai/whisper-small`, fine-tuned for Hindi ASR on 500 FLEURS (hi_in) utterances. ~35 minutes on a free Colab T4.

Results (IndicTTS eval set, out-of-domain)

Closes 55% of the gap to Sarvam Saaras v3 (0.278 → 0.125 WER points).

Usage

from transformers import WhisperForConditionalGeneration, WhisperProcessor
from peft import PeftModel

processor = WhisperProcessor.from_pretrained("openai/whisper-small", language="Hindi", task="transcribe")
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-small")
model = PeftModel.from_pretrained(model, "manishehehe/whisper-small-hindi-lora")
model.eval()

ids = model.generate(input_features=input_features, language="hi", task="transcribe", max_new_tokens=225)
print(processor.tokenizer.batch_decode(ids, skip_special_tokens=True)[0])

Training

LoRA on q_proj/v_proj (r=32, α=64, dropout=0.05) — ~1.8M trainable params (0.7% of base). 500 steps, effective batch 32, LR 1e-3, fp16, single T4.

Code: https://github.com/manishehehe/whisper-indic-lora

Limitations

Small eval set (n=30); WER reductions are descriptive (no per-utterance significance test). FLEURS read speech differs in domain from IndicTTS studio recordings.