Modelpublic

Ar4ikov/VibeVoice-ASR-AWQ-W4A16-ASYM-DFlash2

sourceHugging Facemitupdated 20h agoView on Hugging Face

0likes16downloads

Model Card

VibeVoice-ASR-AWQ-W4A16-ASYM-DFlash2

VibeVoice-ASR (7B), AWQ W4A16 (asymmetric, groups of 128), with its [DFlash 2](https://inco.ai/blog/dflash2/) drafter bundled in drafter/: one download, and vibevoice.c decodes with speculative decoding -- the drafter proposes 8 tokens in one pass, the model checks them in one pass and keeps the ones it agrees with. The check is exact: every checked row is computed with the arithmetic of the model's own decode step, so the transcript is byte-for-byte the one without the drafter. (With a drafter vibevoice.c attends with flashinfer, which checks rows together; the transcript is then --attn flashinfer's.)

Use

Needs vibevoice.c with DFlash 2 support: branch dflash2 (PR #48), in the next release. A model directory's drafter/ is used without asking:

bash

vv_cli --model ./VibeVoice-ASR-AWQ-W4A16-ASYM-DFlash2 --audio talk.wav                  # with the drafter
vv_cli --model ./VibeVoice-ASR-AWQ-W4A16-ASYM-DFlash2 --audio talk.wav --draft none     # plain decoding
vv_cli serve --model ./VibeVoice-ASR-AWQ-W4A16-ASYM-DFlash2 --slots 4

Results

vibevoice.c b72be15 (branch dflash2), RTX 3090, greedy decoding, decode tokens per second:

	plain	drafted	speedup	tokens per block	same transcript
20 held-out clips (AMI, VoxConverse, earnings calls, LibriSpeech, FLEURS)	148 tok/s	565 tok/s	3.82x	5.32	20/20
2-minute file, 2 speakers	151 tok/s	523 tok/s	3.47x	4.64	yes
32-minute file	123 tok/s	296 tok/s	2.41x	3.63	yes

Plain = the same binary without --draft, both with flashinfer attention, which --draft picks by default; --draft-check exact with the check width it measures. Against the default without a drafter (fa2): 3.65x on the 2-minute file, 2.78x on the 32-minute file.

Inside

—The model: the files of Ar4ikov/VibeVoice-ASR-AWQ-W4A16-ASYM at revision 22b44c85, unchanged (7.02 GB) -- its card has the quantization, the calibration and the WER.
—drafter/: Ar4ikov/VibeVoice-ASR-DFlash2-Drafter-AWQ-W4A16-ASYM at revision cff4a1db (0.55 GB): 5 Qwen3-style layers reading the model's layers 1/7/13/19/25, a candidate selector, a 32768-id draft vocabulary; its projections stored as INT4 (compressed-tensors pack-quantized). Its card has the architecture and the training.

License

MIT, like VibeVoice.