cstr/chatterbox-GGUF

Chatterbox TTS — GGUF (ggml-quantised)

GGUF / ggml conversion of `ResembleAI/chatterbox` for use with [CrispStrobe/CrispASR](https://github.com/CrispStrobe/CrispASR).

Chatterbox is a full TTS pipeline: character tokenizer → T3 (30-layer Llama AR, 520M) → speech tokens → S3Gen (Conformer encoder + UNet1D CFM denoiser, 10 Euler steps) → HiFTGenerator vocoder (conv chains + Snake activations + iSTFT) → 24 kHz WAV. Distributed under MIT license.

This is the multilingual Chatterbox (23 languages: Arabic, Danish, German, Greek, English, Spanish, Finnish, French, Hebrew, Hindi, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese, Russian, Swedish, Swahili, Turkish, Chinese) — the T3 GGUF carries the 2454-token multilingual text tokenizer. Select the language with -l <code> (e.g. -l de, -l ja); English is the default.

Two GGUF files are needed: the T3 model (text → speech tokens) and the S3Gen model (speech tokens → audio).

Files

Note: vocoder weights (convpre, resblocks, convpost, source fusion) are kept at F32 in all quant levels for audio quality. Quantization applies to the Conformer encoder, UNet decoder, and T3 Llama layers.

Quick start

# 1. Build CrispASR
git clone https://github.com/CrispStrobe/CrispASR
cd CrispASR
cmake -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=OFF
cmake --build build -j --target chatterbox

# 2. Pull both model files
huggingface-cli download cstr/chatterbox-GGUF chatterbox-t3-q8_0.gguf --local-dir .
huggingface-cli download cstr/chatterbox-GGUF chatterbox-s3gen-q8_0.gguf --local-dir .

# 3. Synthesise with the built-in default voice
./build/bin/crispasr --backend chatterbox \
    -m chatterbox-t3-q8_0.gguf \
    --codec-model chatterbox-s3gen-q8_0.gguf \
    --tts "Hello there, this is chatterbox speaking." \
    --tts-output out.wav

# 4. (Optional) clone a different speaker — bake a small voice GGUF
# from a reference WAV, then pass it via --voice. Requires the upstream
# python pkg: pip install chatterbox-tts
python models/bake-chatterbox-voice-from-wav.py \
    --input /path/to/reference.wav \
    --output my_voice.gguf

./build/bin/crispasr --backend chatterbox \
    -m chatterbox-t3-q8_0.gguf \
    --codec-model chatterbox-s3gen-q8_0.gguf \
    --voice my_voice.gguf \
    --tts "Cloned voice synthesising arbitrary text." \
    --tts-output cloned.wav

See `docs/tts.md` for the full Chatterbox voice-clone reference, including the per-call cache used by --server mode.

Architecture

Text → Character tokenizer (704 tokens)
     → T3 Llama AR (30 layers, 1024D, 16 heads, RoPE, SwiGLU, CFG)
     → 25 Hz speech tokens (6561 codebook)
     → Conformer encoder (6 pre + 4 post upsample, 512D, 8 heads)
     → 80-channel mel spectrogram
     → UNet1D CFM denoiser (1 down + 12 mid + 1 up, 256 ch, 10 Euler steps)
     → HiFTGenerator vocoder (3× ConvTranspose1d + 9 ResBlocks + Snake + iSTFT)
     → 24 kHz mono WAV

Quality verification

ASR roundtrip on Python reference mel (no source fusion, deterministic):

All quantization levels (F16/Q80/Q4K) produce ASR-identical output on the reference mel.

Conversion

python models/convert-chatterbox-to-gguf.py \
    --input ResembleAI/chatterbox \
    --output-dir .

Requires pip install gguf safetensors torch huggingface_hub.

Related models

• —`cstr/lahgtna-chatterbox-v1-GGUF` — Arabic T3 variant (MIT, shares S3Gen)
• —`cstr/orpheus-3b-0.1-ft-GGUF` — Llama-3.2 + SNAC TTS
• —`cstr/qwen3-tts-0.6b-customvoice-GGUF` — Qwen3-TTS with fixed speakers

License

MIT — same as the upstream ResembleAI/chatterbox.

Provenance and EU AI Act Art. 53 note

• —Upstream model: ResembleAI/chatterbox — published by ResembleAI.
• —Upstream licence: mit. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• —What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• —Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• —Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.