hynt/Zipformer-30M-RNNT-Streaming-6000h
Vietnamese Streaming Speech-to-Text (ASR) — ZipFormer-30M-RNNT-Streaming-6000h
🔍 Overview
The Vietnamese Streaming Speech-to-Text (ASR) model is built on the ZipFormer architecture with chunk size 16,32,64 — an improved variant of the Conformer — featuring only 30 million parameters yet. On CPU, the model can transcribe a 1-second audio chunk in just 0.05 seconds, designed for streaming-based tasks with low latency requirements.
🚀 Online Demo
You can test the streaming and none-streaming model directly here: 👉 https://huggingface.co/spaces/hynt/k2-automatic-speech-recognition-demo
⚙️ Model Architecture and Training strategy:
- Architecture: ZipFormer
- Parameters: ~30M
- Language: Vietnamese
- Loss Function: RNN-Transducer (RNNT Loss)
- Chunk Size: 16, 32, 64
- Framework: PyTorch + k2
- Training strategy: Carefully preprocess the data, apply an augmentation strategy based on the distribution of out-of-vocabulary (OOV) tokens and refine the transcriptions using Whisper.
- Optimized for: High-speed CPU inference
🧠 Training Data
The model was trained on approximately 6000 hours of high-quality Vietnamese speech collected from various public datasets:
🧪 Evaluation Results
Lower is better (WER %)
🏆 Achievements
By training this none-streaming model architecture on 4,000 hours of data, I won First Place in the Vietnamese Language Speech Processing (VLSP) competition 2025. Comprehensive details about training data, optimization strategies, architecture improvements, and evaluation methodologies are available in the paper below:
⚡ Inference Speed
⚙️ How to Run This Model
Please refer to the following guides for instructions on how to run and deploy this model:
- For Torch JIT Script: https://k2-fsa.github.io/sherpa/
- For ONNX: https://k2-fsa.github.io/sherpa/onnx/
- For Streaming Web Test:: https://github.com/k2-fsa/sherpa/tree/master/sherpa/bin
💬 Summary
The ZipFormer-30M-RNNT-6000h and ZipFormer-30M-RNNT-Streaming-6000h model demonstrates that a lightweight architecture can still achieve state-of-the-art accuracy for Vietnamese ASR. It is designed for fast deployment on CPU-based systems, making it ideal for real-time speech recognition, callbots, and embedded speech interfaces.