BricksDisplay/Chatterbox-Multilingual-TTS-GGUF

Chatterbox-Multilingual T3 GGUF

End-to-end GGUF conversion of ResembleAI's Chatterbox multilingual T3 (t3_mtl23ls_v3.safetensors — 23-language coverage).

This release adopts the native codec_lm split used by CSM / Qwen3-TTS / MOSS-TTSD:

• —Backbone (`chatterbox-mtl-t3-<quant>.gguf`) — stock llama arch GGUF of Chatterbox T3's Llama-520M tfmr.* weights (30 layers, hidden 1024, 16 heads, headdim=64, MLP 4096, llama3 RoPE scaling, ropetheta=500000, vocab placeholder = tokenizer.ggml.model = "none"). Runs in stock llama.cpp with embeddings=true.
• —Codec + codec_lm (`chatterbox-mtl-codec-<quant>.gguf`) — Chatterbox S3G (flow-matching decoder + HiFi-GAN vocoder) bundled with the T3 LM-adaptor side (audio embed table, speech head, text embed/head, learned positional embeddings, cond encoder weights). Runs in codec.cpp as a parallel_heads_delay codeclm with `ncb=1`.
• —S3T tokenizer (`chatterbox-mtl-s3t.gguf`) — speech tokenizer needed when registering a custom voice from a reference WAV.

Compared to the earlier release in this repo, the old t3-*.gguf (custom shape) and t3-extras.gguf are gone — everything the host runtime needs to drive the LM adaptor is now bundled into chatterbox-mtl-codec-*.gguf and exposed through codec.cpp's codec_lm API.

Files

Backbone (chatterbox-mtl-t3-<quant>.gguf)

Codec + codec_lm (chatterbox-mtl-codec-<quant>.gguf)

S3T speech tokenizer

chatterbox-mtl-s3t.gguf (F16, 237 MB) — needed for voice-clone, encodes a reference WAV into the speech token IDs that the codec_lm consumes when registering a custom speaker. Same weights as English Chatterbox.

Inference shape

Per-frame AR loop, single-codebook parallel_heads_delay:

backbone (Llama-520M, embeddings=true) hidden h
    → codec_lm_step_begin(state, h)
    → codec_lm_step_logits(0) → sample speech-token → codec_lm_step_push_code
    → codec_lm_step_finish → codes[1]
    → codec_lm_compose_audio_embd(codes) + speech_pos_emb[step] → next-step embedding
    → feed via b.embd; loop until stop_speech_token (6562)

Prompt prefix assembly (text + cond) lives inside codec.cpp's chatterbox path; the host application doesn't directly touch lm.chatterbox.* tensors.

Sources

• —Upstream model: `ResembleAI/chatterbox` (t3_mtl23ls_v3.safetensors)
• —Conversion tooling: `mybigday/codec.cpp` (prep_chatterbox_t3 + lm_adaptor/chatterbox.py + ChatterboxS3GConverter with lm_source)
• —Inference runtime: `mybigday/llama.rn`

Supported languages

Arabic, Danish, German, Greek, English, Spanish, Finnish, French, Hebrew, Hindi, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese, Russian, Swedish, Swahili, Turkish, Chinese (23 total).