Modelpublic

pantinor/parakeet-tdt-0.6b-v3-smoothquant

sourceHugging Facecc-by-4.0updated 2mo agoView on Hugging Face

0likes

Model Card

Parakeet TDT 0.6B v3 — SmoothQuant int8 (sherpa-onnx format)

nvidia/parakeet-tdt-0.6b-v3 repackaged for sherpa-onnx with a SmoothQuant int8 encoder from Olicorne/parakeet-tdt-0.6b-v3-smoothquant-onnx.

Why SmoothQuant?

The stock int8 encoder degrades badly on audio >30s (40.4% WER on 390s audio). SmoothQuant fixes this by migrating per-channel activation outliers into weights before static quantization. Result: 11.3% WER on long audio (vs fp16's 10.2%), using half the RAM.

Files

File	Size	Description
	842 MB	SmoothQuant int8 encoder (from Olicorne)
	12 MB	int8 decoder (unchanged from stock)
	6 MB	int8 joiner (unchanged from stock)
	92 KB	Tokenizer vocabulary

sherpa-onnx Configuration

Size Comparison

Variant	Encoder Size	Total Size	Long-audio WER
Stock int8 (pantinor/parakeet-tdt-0.6b-v3-int8)	622 MB	~670 MB	40.4%
SmoothQuant int8 (this repo)	842 MB	~860 MB	11.3%
fp16/fp32	~1.2 GB	~1.2 GB	10.2%

Credits

—Original model: nvidia/parakeet-tdt-0.6b-v3
—SmoothQuant encoder: Olicorne/parakeet-tdt-0.6b-v3-smoothquant-onnx
—ONNX export: istupakov/parakeet-tdt-0.6b-v3-onnx