CoolFace
Modelpublic

christopherthompson81/vibevoice-asr-streaming-7b-onnx

sourceHugging Facemitupdated 18d agoView on Hugging Face
0likes98downloads
Model Card

VibeVoice-ASR-Streaming 7B — ONNX export for Vernacula

An ONNX export of `microsoft/VibeVoice-ASR-Streaming-7B`, Microsoft's streaming speaker-attributed ASR model (Qwen2.5-7B decoder over a pair of causal audio tokenizers), for use as an ASR backend in Vernacula.

The model transcribes who said what as audio arrives, with no separate diarizer: it emits one text chunk per 2.93 s of audio and marks speaker turns inline.

Files

filewhat it is
audio_encoder.onnx (+ .data)acoustic + semantic tokenizer encoders and both connectors, float16. One fixed 83,200-sample window (3.47 s) in, 26 frames out.
decoder_gqa.onnx (+ .data)Qwen2.5-7B decoder, INT8 weight-only, attention as com.microsoft::GroupQueryAttention over a shared float16 KV cache.
export-report.jsonshapes, the streaming constants, and the prompt/special token ids the runtime needs.
tokenizer.json, config.json, preprocessor_config.json, tokenizer_config.jsonmetadata carried through from the source checkpoint.

How it differs from the source checkpoint

  • —Deterministic audio encoding. Upstream samples the acoustic latents with Gaussian noise; this export uses the latent mean, which is bit-repeatable and sits inside the seed-to-seed variation of the original.
  • —INT8 decoder weights. Measured indistinguishable from float16 on transcript accuracy, while halving the weights.
  • —Shared, pre-allocated KV cache. VRAM is flat in recording length rather than growing, and throughput does not decay over a long file.

Requirements and limits

  • —ONNX Runtime with the CUDA execution provider. GroupQueryAttention and the float16 graphs are not supported on the CPU provider here.
  • —Peak VRAM about 15.7 GB; real-time factor about 0.142 on an RTX 3090.
  • —The KV cache ceiling in export-report.json bounds recording length and is set to the checkpoint's trained context (131,072 positions, roughly 2 hours of audio). The runtime refuses a longer recording rather than failing partway. Attention cost grows with the filled cache, so decoding slows as a long recording proceeds; VRAM does not.
  • —Speaker attribution at this size is reliable — it separated two speakers correctly across a 30-minute file.

Licence

MIT, matching the source checkpoint.