Modelpublic

preference1/OmniVoice

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face

0likes17downloads

Model Card

OmniVoice 🌍

OmniVoice is a massively multilingual zero-shot text-to-speech (TTS) model supporting over 600 languages. Built on a novel diffusion language model-style architecture, it delivers high-quality speech with superior inference speed, supporting voice cloning and voice design.

—Paper: OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models
—Repository: GitHub
—Demo: Hugging Face Space

Key Features

—600+ Languages Supported: The broadest language coverage among zero-shot TTS models.
—Voice Cloning: State-of-the-art voice cloning quality from a short reference audio.
—Voice Design: Control voices via assigned speaker attributes (gender, age, pitch, dialect/accent, whisper, etc.).
—Fine-grained Control: Non-verbal symbols (e.g., [laughter]) and pronunciation correction via pinyin or phonemes.
—Fast Inference: RTF as low as 0.025 (40x faster than real-time).
—Diffusion Language Model-style Architecture: A clean, streamlined, and scalable design that delivers both quality and speed.

Sample Usage

To get started, install the omnivoice library:

We recommend using a fresh virtual environment (e.g., conda, venv, etc.) to avoid conflicts.

Step 1: Install PyTorch

<details> <summary>NVIDIA GPU</summary>

bash

# Install pytorch with your CUDA version, e.g.
pip install torch==2.8.0+cu128 torchaudio==2.8.0+cu128 --extra-index-url https://download.pytorch.org/whl/cu128

See PyTorch official site for other versions installation.

</details>

<details> <summary>Apple Silicon</summary>

bash

pip install torch==2.8.0 torchaudio==2.8.0

</details>

Step 2: Install OmniVoice

bash

pip install omnivoice

Python API

You can use OmniVoice for zero-shot voice cloning as follows:

python

from omnivoice import OmniVoice
import torch
import torchaudio

# Load the model
model = OmniVoice.from_pretrained(
    "k2-fsa/OmniVoice",
    device_map="cuda:0",
    dtype=torch.float16
)

# Generate audio
audio = model.generate(
    text="Hello, this is a test of zero-shot voice cloning.",
    ref_audio="ref.wav",
    ref_text="Transcription of the reference audio.",
) # audio is a list of `torch.Tensor` with shape (1, T) at 24 kHz.

torchaudio.save("out.wav", audio[0], 24000)

For more generation modes (e.g., voice design), functions (e.g., non-verbal symbols, pronunciation correction) and comprehensive usage instructions, see our GitHub Repository.

Discussion & Communication

You can directly discuss on GitHub Issues.

You can also scan the QR code to join our wechat group or follow our wechat official account.

Wechat Group	Wechat Official Account

Citation

bibtex

@article{zhu2026omnivoice,
      title={OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models},
      author={Zhu, Han and Ye, Lingxuan and Kang, Wei and Yao, Zengwei and Guo, Liyong and Kuang, Fangjun and Han, Zhifeng and Zhuang, Weiji and Lin, Long and Povey, Daniel},
      journal={arXiv preprint arXiv:2604.00688},
      year={2026}
}