Modelpublic

Luigi/nemotron-3.5-asr-streaming-0.6b-zhtw

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

1likes84downloads

Model Card

Nemotron-3.5-ASR-Streaming-0.6B — zh-TW fine-tuned (v3)

Fine-tune of nvidia/nemotron-3.5-asr-streaming-0.6b specialised for Taiwan Mandarin. v3 improves zh-TW by a further 18% over v2 (Common Voice zh-TW CER 12.03 → 9.81), for small regressions on a few other languages — see the honest table below.

Results (transformers runtime, greedy, script-normalised via OpenCC)

eval	base*	v2	v3 (this)
Common Voice zh-TW (n=120, CER)	38.43	12.03	9.81 ✅
Taiwan-Tongues test (n=100, CER)	30.67	—	14.85
English LibriSpeech (n=100, WER)	2.71	2.58	2.79
Arabic FLEURS (WER)	14.59	14.80	14.48 ✅
Japanese FLEURS (CER)	16.89	16.48	16.65 ✅
Korean FLEURS (CER)	9.60	8.80	9.42 ✅
German FLEURS (WER)	11.67	10.92	11.42 ✅
Hindi FLEURS (WER)	8.08	7.91	8.80
Spanish FLEURS (WER)	5.56	6.83	6.75
French FLEURS (WER)	13.90	15.45	16.15 ⚠️

\ base = same model with the zh-TW prompt column warm-started from zh-CN (the untouched base zh-TW slot is dead, ~100% CER). Choose v2 if you need French/Hindi/English at their best; choose v3 for zh-TW.* French is the one number that exceeds NVIDIA's published FLEURS figure (15.93).

Output is Simplified Chinese — convert with OpenCC

The 13,087-token tokenizer cannot represent many common Traditional characters (點兒區說麼嗎灣黨體產 …; 6.3% of Traditional chars fail a round-trip vs 0.1% for Simplified). Training targets were Simplified on purpose. For Traditional:

python

from opencc import OpenCC
text = OpenCC("s2t").convert(text)

Any of language="auto" / "zh-CN" / "zh-TW" works (within ~0.7 CER of each other).

Training

NVIDIA's NeMo recipe (speech_to_text_finetune.py, fastconformer_transducer_bpe_streaming_prompt.yaml), 2×RTX 5090 DDP, bf16, AdamW lr=1e-5 (no Noam — the config's lr:2.0 is a pretraining setting).

Phase 1 (47.5 h): 12,077 Common Voice zh-TW + 5,846 IVOD 立法院 segments kept only where two independent teachers agreed (X-ASR and Whisper-medium disagreed by >20% on 41% of segments — those were discarded) + 4,516 LibriSpeech English + 600 ASCEND zh-CN replay. prompt_mode: unified. Phase 2 (repair, lr 4e-6, 3 epochs): English-heavy mix to undo the English drift — recovered en 2.92 → 2.79 with zero zh-TW loss.

Limitations

—A dedicated zh-en model is still better at Chinese: an X-ASR zipformer scores 6.66 CER (Common Voice zh-TW) and 6.13 (Taiwan-Tongues test) on these same clips. This model's value is covering 25 languages, not beating a zh-en specialist.
—Long-form literary Chinese is weak (FormosaSpeech 23.4 CER) — classical proper nouns and vocabulary are out of domain.
—ASR only: no diarization, no timestamps.

Provenance

Base: nvidia/nemotron-3.5-asr-streaming-0.6b (NVIDIA Open Model License). Common Voice zh-TW (CC0) via JacobLinCool/common_voice_16_1_zh_TW_clean; 立法院 IVOD open data (CC-BY-4.0) pseudo-labelled and teacher-filtered; LibriSpeech (CC-BY-4.0); ASCEND.