Modelpublic

ct03/omnivoice-onnx-int8hq

sourceHugging Faceotherupdated 4mo agoView on Hugging Face

3likes46downloads

Model Card

OmniVoice ONNX (INT8-HQ) — CPU inference

INT8-HQ ONNX export of k2-fsa/OmniVoice, optimized to run on CPU with onnxruntime (no GPU, no PyTorch needed at inference time).

—Multilingual TTS (646 languages incl. Vietnamese) + zero-shot voice cloning + voice design.
—24 kHz output, 8 codebooks, frame rate 75.
—INT8-HQ: cosine > 0.9999 vs FP32, perceptually almost indistinguishable.

Components

Folder	Role	Size
`omnivoice_lm_int8_hq/`	Diffusion LM (INT8-HQ, audio head kept FP32)	~611 MB
`audio_tokenizer_decoder_int8/`	tokens → 24 kHz waveform	~82 MB
`audio_tokenizer_encoder_int8/`	reference audio → codec tokens (voice cloning)	~378 MB
`tokenizer.json`, `config.json`	text tokenizer + config	small

Each model is model.onnx + model.onnx_data (ONNX external data) — download both files.

ONNX I/O

Model	Input	Output
LM	`input_ids[batch,8,seq]` int64; `audio_mask[batch,seq]` bool; `attention_mask[batch,1,seq,seq]` bool; `position_ids[batch,seq]` int64	`logits[batch,8,seq,1025]` f32
Encoder	`audio[batch,1,num_samples]` f32 @24kHz	`audio_codes[batch,8,num_frames]` int64
Decoder	`audio_codes[batch,8,num_frames]` int64	`audio[batch,1,num_samples]` f32

Inference

The sampling loop (diffusion + CFG) runs in Python; the LM / encoder / decoder run via ONNX. Reference inference code: AFun9/Omnivoice-onnx (infer_onnx.py). Example:

bash

python infer_onnx.py --variant int8hq --num-step 16 --only demo_voice_clone

Performance (CPU)

RTF ~3–7 depending on the CPU (i9-14900KF ~2.8–6.7; slower CPUs are slower). Suitable for batch / offline use, not real-time. Increase --num-step for higher quality (slower).

License

This export bundles two components under different licenses — you must comply with both:

—Diffusion LM (omnivoice_lm_int8_hq/): derived from Qwen3-0.6B / OmniVoice — Apache-2.0.
—Audio tokenizer (audio_tokenizer_*_int8/): derived from Higgs Audio 2 (Boson AI) — Boson Higgs Audio 2 Community License (based on the Meta Llama 3 Community License); see `LICENSE`. It includes an acceptable-use policy and Llama-style terms.

Because it contains the Higgs Audio 2 component, the repo license is declared as other. This is an ONNX export + quantization only — weights are unchanged; all rights belong to the original authors (k2-fsa/OmniVoice, Boson AI, Qwen).