microsoft/VibeVoice-ASR-BitNet
20044k
1---2language:3- en4- zh5- fr6- it7- ko8- pt9- vi10license: mit11pipeline_tag: automatic-speech-recognition12tags:13- ASR14- quantization15- cpu-inference16- gguf17- bitnet18- multilingual19library_name: ggml20---21 22## VibeVoice-ASR-BitNet23 24[](https://github.com/microsoft/VibeASR.cpp)25[](https://arxiv.org/abs/2607.21075)26[](https://opensource.org/licenses/MIT)27 28**VibeVoice-ASR-BitNet** is a compressed variant of [VibeVoice-ASR](https://huggingface.co/microsoft/VibeVoice-ASR) optimized for **real-time inference on edge CPUs** — no GPU required. Through heterogeneous quantization, the model is compressed from 4.62 GB to **1.58 GB** while achieving **1.6–2.3× faster** inference than Whisper.cpp with real-time capability (RTF < 1) on as few as 3 CPU threads.29 30➡️ **Code:** [microsoft/VibeASR.cpp](https://github.com/microsoft/VibeASR.cpp)<br>31➡️ **Report:** [VibeVoice-ASR-BitNet Technical Report](https://arxiv.org/abs/2607.21075)<br>32➡️ **Base Model:** [microsoft/VibeVoice-ASR](https://huggingface.co/microsoft/VibeVoice-ASR)<br>33 34<p align="center">35 <img src="figures/report_overview.png" width="90%"/>36</p>37 38---39 40## 🔥 Key Features41 42- **⚡ Real-time on CPU** — RTF < 1 with 3+ threads on commodity x86 (AVX2) and ARM (NEON) hardware43- **📦 Compact** — 1.58 GB total (2.9× compression from FP16), fits in edge device memory44- **🌍 Multilingual** — English, Chinese, French, Italian, Korean, Portuguese, Vietnamese, and more45- **🔧 Custom SIMD Kernels** — Fused operators within the ggml framework for both ARM and x86 platforms46 47---48 49## Quantization Strategy50 51<div align="center">52 53| Component | FP16 | Quantized | Method | Compression |54|:-:|:-:|:-:|:-:|:-:|55| VAE Tokenizer | 1.31 GB | 0.65 GB | I8\_S | 2.0× |56| LM Decoder | 3.32 GB | 0.92 GB | I2\_S + Q6\_K | 3.6× |57| **Total** | **4.62 GB** | **1.58 GB** | — | **2.9×** |58 59</div>60 61---62 63## Evaluation64 65### Inference Speed66 67<div align="center">68 69| Threads | 1 | 2 | 3 | 4 | 6 | 8 |70|:-:|:-:|:-:|:-:|:-:|:-:|:-:|71| RTF | 1.98 | 1.08 | **0.77** | **0.63** | **0.49** | **0.42** |72| vs. Whisper.cpp | 2.28× | 2.12× | 1.86× | 1.86× | 1.71× | 1.55× |73 74</div>75 76> Benchmarked on AMD EPYC 7V13 (AVX2+FMA) with 20s audio. **Bold** = RTF < 1 (real-time).77 78### Accuracy (WER%)79 80<div align="center">81 82| Benchmark | VibeVoice-ASR-7B | VibeVoice-ASR-BitNet | Parakeet | Whisper | SenseVoice | FunASR |83|:-:|:-:|:-:|:-:|:-:|:-:|:-:|84| MLC-EN | 7.82 | **8.25** | 8.40 | 13.57 | 12.39 | 11.36 |85| MLC-FR | 16.03 | 17.41 | — | — | — | — |86| MLC-IT | 15.67 | 17.23 | — | — | — | — |87| MLC-KO | 9.83 | 11.15 | — | — | — | — |88| MLC-PT | 22.41 | 24.87 | — | — | — | — |89| MLC-VI | 20.15 | 22.38 | — | — | — | — |90| AISHELL4 | 19.83 | 27.45 | — | — | 22.52 | **20.41** |91| AMI-ihm | 17.42 | **21.36** | 21.92 | 27.07 | 30.81 | 32.07 |92| AMI-sdm | 24.18 | **25.87** | 26.33 | 36.92 | 48.11 | 40.17 |93| AliMeeting | 36.21 | 40.58 | — | — | **38.75** | 39.27 |94| Fleurs-en | 4.73 | 5.21 | 4.09 | **3.99** | 6.84 | 4.93 |95| Fleurs-zh | 7.92 | 8.35 | — | — | **5.56** | 7.00 |96| Libri-clean | 2.17 | 2.41 | **1.49** | 1.98 | 2.78 | 1.58 |97| Libri-other | 5.84 | 6.27 | **3.13** | 3.60 | 6.81 | 4.01 |98| VoxPopuli | 4.92 | **5.18** | 5.26 | 7.19 | 8.63 | 6.46 |99 100</div>101 102---103 104## Model Files105 106<div align="center">107 108| File | Size | Description |109|:-:|:-:|:-:|110| `vibeasr-vae-encoder-i8_s.gguf` | 0.65 GB | VAE tokenizer, I8\_S quantized (ready to use) |111| `vibeasr-lm-i2_s-embed-q6_k.gguf` | 0.92 GB | LM decoder, I2\_S quantized (ready to use) |112| `model-*.safetensors` | 10.7 GB | Original SafeTensors (for conversion) |113 114</div>115 116---117 118## License119 120This project is licensed under the MIT License.121 122## Contact123 124This project was conducted by members of Microsoft Research. If you have suggestions, questions, or observe unexpected behavior, please contact us at VibeVoice@microsoft.com.125