hynt/Zipformer-30M-RNNT-6000h
Vietnamese Speech-to-Text (ASR) — ZipFormer-30M-RNNT-6000h
🔍 Overview
The Vietnamese Speech-to-Text (ASR) model is built on the ZipFormer architecture — an improved variant of the Conformer — featuring only 30 million parameters yet achieving exceptional performance in both speed and accuracy. On CPU, the model can transcribe a 12-second audio clip in just 0.3 seconds, significantly faster than most traditional ASR systems without requiring a GPU.
☕ Give Me a Coffee for Training Cost
This model was trained entirely on self-funded, personal hardware — every GPU hour and electricity bill came out of my own pocket. If this work is useful to you, a small contribution helps cover the cost of the next training run and keeps future models free and open. Every coffee is hugely appreciated 🙏
<p align="center"> <img src="qr.jpg" alt="VietQR — Give me a coffee" width="320"/> </p>
<p align="center"><i>Scan with any VietQR / Napas 247 supported banking app</i></p>
Thank you for supporting open Vietnamese ASR! ❤️
🚀 Online Demo
You can try the model directly here: 👉 https://huggingface.co/spaces/hynt/k2-automatic-speech-recognition-demo
⚙️ Model Architecture and Training strategy:
- Architecture: ZipFormer
- Parameters: ~30M
- Language: Vietnamese
- Loss Function: RNN-Transducer (RNNT Loss)
- Framework: PyTorch + k2
- Training strategy: Carefully preprocess the data, apply an augmentation strategy based on the distribution of out-of-vocabulary (OOV) tokens and refine the transcriptions using Whisper.
- Optimized for: High-speed CPU inference
🧠 Training Data
The model was trained on approximately 6000 hours of high-quality Vietnamese speech collected from various public datasets:
🧪 Evaluation Results
Lower is better (WER %)
🏆 Achievements
By training this model architecture on 4,000 hours of data, I won First Place in the Vietnamese Language Speech Processing (VLSP) competition 2025. Comprehensive details about training data, optimization strategies, architecture improvements, and evaluation methodologies are available in the paper below:
⚡ Inference Speed
⚙️ How to Run This Model
Please refer to the following guides for instructions on how to run and deploy this model:
- For Torch JIT Script: https://k2-fsa.github.io/sherpa/
- For ONNX: https://k2-fsa.github.io/sherpa/onnx/
💬 Summary
The ZipFormer-30M-RNNT-6000h model demonstrates that a lightweight architecture can still achieve state-of-the-art accuracy for Vietnamese ASR. It is designed for fast deployment on CPU-based systems, making it ideal for real-time speech recognition, callbots, and embedded speech interfaces.