Modelpublic

Luigi/nemotron-asr-litert-zhtw

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

0likes90downloads

Model Card

Nemotron-3.5-ASR — LiteRT q4-mix, zh-TW fine-tuned (v2)

On-device (Android) LiteRT build of Luigi/nemotron-3.5-asr-streaming-0.6b-zhtw — the zh-TW fine-tuned Nemotron (3.2× better Taiwan Mandarin than base). q4-mix: INT4 encoder + fp16 decoder/joint + fp32 prompt-fusion. Bundle 663 MB.

Supersedes v1 (which packaged the zero-training warm-start).

Accuracy

build	CV zh-TW CER (n=120)
fp32 fine-tuned model (reference)	11.33
this q4-mix LiteRT	13.20 (+1.9 INT4 cost)
v1 q4-mix (base warm-start)	~38

Per language-prompt slot on this build — all within ~0.7 CER, pick either:

slot	q4-mix CER	fp32 CER
`zh-CN` (4)	13.20	11.57
`auto` (101)	13.79	11.33
`zh-TW` (5)	13.90	12.03

auto was the worst slot on the base model (50.58 CER) and is repaired by the fine-tune — training used prompt_mode: unified, which trains the auto path alongside the explicit language ID. Use auto when the speaker may switch languages, or an explicit slot when you know it.

Other languages (fp32 FT, vs base): ko/de/ja/hi/en all improve, ar flat, fr/es +1.5 — see the source model card.

Files

file	role	precision	size
`nemotron_encoder_q4.tflite`	FastConformer encoder	INT4 FC (blockwise-128) + fp32 convs	596 MB
`nemotron_prompt_fuse_fp32.tflite`	128-slot language-prompt fusion	fp32	18 MB
`nemotron_decoder_fp16.tflite`	RNN-T prediction net	fp16	31 MB
`nemotron_joint_fp16.tflite`	RNN-T joint (→ vocab 13088)	fp16	18 MB
`tokenizer.json` / `processor_config.json` / `config.json`	runtime	—	—

Run

Port, runner and integration note: https://github.com/vieenrose/LiteRT/tree/nemotron/litert/samples/asr/nemotron

bash

python -m nemotron.runner --models ./ --wav clip.wav --lang zh-CN --s2t --itn
# --lang auto also works well (see slot table); --s2t gives Traditional output

Output is Simplified Chinese — the tokenizer cannot represent many common Traditional characters, so apply OpenCC s2t (--s2t) for Traditional.

The INT4 encoder needs the LiteRT-Next CompiledModel runtime (Android NNAPI / XNNPACK-QD8); the classic Interpreter cannot allocate dynamic-range INT4. Set the CPU thread count explicitly (big-core count on big.LITTLE).

Provenance

Base: nvidia/nemotron-3.5-asr-streaming-0.6b (NVIDIA Open Model License), fine-tuned for zh-TW (see source model card), then exported with litert_torch 0.9.1 + ai_edge_quantizer 0.7.0 (INT4 weights pre-baked to the export grid before quantization).