niobures/MOSS-Audio-Tokenizer-ONNX

MOSS-Audio-Tokenizer-ONNX

This repository provides the ONNX exports of MOSS-Audio-Tokenizer (encoder & decoder), enabling torch-free audio encoding/decoding for the MOSS-TTS family.

Overview

MOSS-Audio-Tokenizer is the unified discrete audio interface for the entire MOSS-TTS Family, based on the Cat (Causal Audio Tokenizer with Transformer) architecture — a 1.6B-parameter, pure Causal Transformer audio tokenizer trained on 3M hours of diverse audio.

This ONNX repository is designed for lightweight, torch-free deployment scenarios. It serves as the audio tokenizer component in the MOSS-TTS llama.cpp inference backend, which combines llama.cpp (for the Qwen3 backbone) with ONNX Runtime or TensorRT (for the audio tokenizer) to achieve fully PyTorch-free TTS inference.

Supported Backends

Note: We do not provide pre-built TensorRT engines, as they are tied to your specific GPU architecture and TensorRT version. To use TRT, build engines from the ONNX models yourself — see moss_audio_tokenizer/trt/build_engine.sh in the main repository.

Repository Contents

Quick Start

# Download
huggingface-cli download OpenMOSS-Team/MOSS-Audio-Tokenizer-ONNX \
    --local-dir weights/MOSS-Audio-Tokenizer-ONNX

This is typically used together with MOSS-TTS-GGUF for the llama.cpp inference pipeline. See the llama.cpp Backend documentation for the full end-to-end setup.

Main Repositories

About MOSS-Audio-Tokenizer

MOSS-Audio-Tokenizer compresses 24kHz raw audio into a 12.5Hz frame rate using a 32-layer Residual Vector Quantizer (RVQ), supporting high-fidelity reconstruction from 0.125kbps to 4kbps. It is trained from scratch on 3 million hours of speech, sound effects, and music, achieving state-of-the-art reconstruction quality among open-source audio tokenizers.

For the full model description, architecture details, and evaluation metrics, please refer to:

• —MOSS-Audio-Tokenizer GitHub Repository
• —MOSS-TTS README — Audio Tokenizer Section

Evaluation Metrics

The table below compares the reconstruction quality of open-source audio tokenizers with MossAudioTokenizer on speech and audio/music data.

• —Speech metrics are evaluated on LibriSpeech test-clean (English) and AISHELL-2 (Chinese), reported as EN/ZH.
• —Audio metrics are evaluated on the AudioSet evaluation subset, while music metrics are evaluated on MUSDB, reported as audio/music.
• —STFT-Dist. denotes the STFT distance.
• —Higher is better for speech metrics, while lower is better for audio/music metrics (Mel-Loss, STFT-Dist.).
• —Nq denotes the number of quantizers.

LibriSpeech Speech Metrics (MOSS Audio Tokenizer vs. Open-source Tokenizers)

The plots below compare our MOSS Audio Tokenizer model with other open-source speech tokenizers on the LibriSpeech dataset, evaluated with SIM, STOI, PESQ-NB, and PESQ-WB (higher is better). We control the bps of the same model by adjusting the number of RVQ codebooks used during inference.

<table> <tr> <td align="center"><b>SIM</b><br><img src="images/sim.png" width="100%"></td> <td align="center"><b>STOI</b><br><img src="images/stoi.png" width="100%"></td> </tr> <tr> <td align="center"><b>PESQ-NB</b><br><img src="images/pesq-nb.png" width="100%"></td> <td align="center"><b>PESQ-WB</b><br><img src="images/pesq-wb.png" width="100%"></td> </tr> </table>

Citation

If you use this code or result in your paper, please cite our work as: