Modelpublic

Vengadanathan/hindi-hinglish-zipformer-ctc-streaming

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes

Model Card

Hindi/Hinglish Streaming ASR — Zipformer-M CTC

A from-scratch streaming ASR model for Hindi + code-mixed Hinglish. Architecture: Zipformer-M encoder (~65M) + pure CTC head, causal/chunked for streaming, character-level (Unicode codepoint) vocab. Trained with icefall.

Two releases:

—`model_p3_epoch15_avg8.pt` — CURRENT BEST (Phase 3): scaled up on ~337h.
—model_p1_epoch30_avg8.pt — Phase 1 baseline (~89h).

Results (CTC greedy, no LM; checkpoint-averaged)

Eval set	Phase 1 (~89h)	Phase 3 (~337h)
IndicVoices-hi (Hindi)	37.2%	25.7%
call-test (internal hinglish calls)	47.3%	39.9%
Svarah (Indian English)	50.3%	44.5%

Architecture

Zipformer2 "M" (~64.6M): num_encoder_layers=2,2,3,4,3,2, encoder_dim=192,256,384,512,384,256, downsampling_factor=1,2,4,8,4,2, causal, chunk_size=16,32,64,-1, left_context_frames=64,128,256,-1. Pure CTC head, 80-dim fbank @16kHz, subsampling 4 (~25 Hz). Vocab 103 tokens (<blk>,<sos/eos>,<unk>,▁=word-boundary, 63 Devanagari + 26 a-z + 10 digits).

Training data

—Phase 1 (base, from scratch): IndicVoices-hi 60h + keywords 15.4h + Svarah 8.4h + numbers 5.9h (~89h).
—Phase 3 (scale-up, warm-start from P1): IndicVoices-hi 307h + keywords + Svarah + numbers (~337h base → ~1017h with ×3 speed perturbation). IndicVoices text uses unsanitized_normalized (code-switch word [English] → English, tags stripped); all text NFC-normalized, punctuation/foreign scripts dropped, Latin lowercased.

Training

icefall zipformer recipe (char-CTC), PyTorch 2.4.1+cu124 / k2 / lhotse, bf16 (fp16 was unstable on a single L4), ScaledAdam + Eden. P1: from scratch, baselr 0.045, 30 ep. P3: warm-start (`--init-ckpt`), baselr 0.02, 15 ep, non-finite-loss batches skipped. Single NVIDIA L4.

Files

model_p3_epoch15_avg8.pt / model_p1_epoch30_avg8.pt — averaged weights + arch config + vocab_size. tokens.txt — vocab. scripts/ — full reproduction pipeline.

Usage (decode)

Build the icefall zipformer from the checkpoint's config, load_state_dict(ckpt["model"]), extract 80-dim lhotse fbank, run encoder_embed → encoder → ctc_output → ctc_greedy_search, map ids via tokens.txt, replace ▁→space. See scripts/eval_wer.py and scripts/server_ctc.py.

Limitations

Greedy CTC, no LM (n-gram/RNN-T would improve WER). English weaker than Hindi (English data-limited).