EvoAwaken-Workshop/Breeze-TTS-2-gguf

Breeze-TTS-2 GGUF

GGUF conversion of `BreezeBlue/Breeze-TTS-2` for the native Rust implementation in `Liyulingyue/rust-model-inference`.

Breeze TTS 2 is a 3.5B bilingual (English / Chinese) text-to-speech model that supports three inference modes through a single Python CLI:

• —Plain TTS — synthesize speech from a text prompt
• —Voice Design — drive voice characteristics via a natural-language --instruction (e.g. "温柔地说。") and a CFG scale (--cfg-scale)
• —Voice Clone — clone a speaker's timbre from a reference 24 kHz WAV (--ref-audio) plus its transcript (--ref-text)

The model is a multi-module TTS stack: a Qwen3 backbone (1.4B) + T5Gemma2 text encoder (1B) + depth decoder (0.4B) + Mimi codec (0.1B), totalling ~3.5B parameters. The audio tokenizer is based on Qwen3-TTS.

Files

Each row holds a precision variant of the same Breeze-TTS-2 main model. All variants share the same breeze-tts-2-mmproj-F32.gguf audio codec projection. Files are independent — pick one main GGUF plus the codec.

The main model has 28 backbone layers + 26 text encoder layers + 12 depth decoder layers, a hidden size of 2048 (backbone) / 1152 (text encoder) / 1024 (depth decoder), and a vocabulary of 262158 tokens (text) + 2051 audio codes. The codec projector uses the custom breeze_audio architecture and expects 24 kHz mono PCM16 WAV input.

Compatibility

The validated runtime is rust-model-inference after merged PR #64 (Breeze TTS 2 native inference). Built against commit 2f224d2 and later (the path that landed rope + activation SIMD plus the per-precision Breeze routing).

The model is not compatible with stock llama.cpp. The breeze and breeze_audio architectures are specific to the Rust implementation above. The audio codec projector requires the breeze_audio mmproj path which has no llama.cpp equivalent.

Rust Usage

git clone https://github.com/Liyulingyue/rust-model-inference.git
cd rust-model-inference
cargo build --release --bin rust-model-inference
BIN=target/release/rust-model-inference
MODEL=/path/to/breeze-tts-2-Q8_0.gguf
CODEC=/path/to/breeze-tts-2-mmproj-F32.gguf

Plain TTS

"$BIN" --tts \
  --model "$MODEL" \
  --mmproj "$CODEC" \
  --prompt "你好，这是一个声音合成测试。" \
  --out plain.wav --seed 42

Voice Design (instruction + CFG)

--instruction is a natural-language voice description (e.g. "温柔地说。", "愤怒地说。"). CFG defaults to 3 with a non-empty instruction, 1 otherwise. Explicit --cfg-scale 1 disables CFG.

"$BIN" --tts \
  --model "$MODEL" \
  --mmproj "$CODEC" \
  --prompt "你好。" \
  --instruction "温柔地说。" \
  --cfg-scale 3 \
  --out instruction.wav --seed 42

Voice Clone (reference audio + transcript)

--ref-audio (24 kHz mono PCM16 WAV) and --ref-text (the reference transcript, required) must be supplied together. Multi-channel input is mixed down automatically; non-24 kHz sample rates are resampled.

"$BIN" --tts \
  --model "$MODEL" \
  --mmproj "$CODEC" \
  --prompt "再见。" \
  --ref-audio plain.wav --ref-text "你好，这是一个声音合成测试。" \
  --out clone.wav --seed 42

Sampling defaults

--temperature 0.9 --top-k 50 --top-p 1 --seed 42; --temperature 0 selects greedy decoding. --threads N controls the rayon worker pool (default = physical cores). With seed fixed, sampling is reproducible.

Conversion

The converter is torch-free — it reads sharded BF16 / F32 safetensors through mmap and writes GGUF directly:

python3 tools/breeze/convert_breeze.py \
  /path/to/Breeze-TTS-2 \
  --out-dir /path/to/output \
  --quant q8_0 \
  --codec-quant f32

--quant accepts bf16, f16, f32, q8_0, q4_0. --codec-quant accepts f32 or q8_0. The script preserves BF16 / F32 source bytes for tensors that the Rust loader pins to F32 / BF16 (norms, codebook initialisation flags, eoi embeddings, codec_model.* snapshots) — those never participate in the chosen precision mode.

Alignment

End-to-end timing per main-model precision on a single prompt ("天气真好", seed 42, 4 threads, --max-tokens 64):

BF16 / F16 / F32 outputs are bit-exact identical (md5 c19502ff...) — the underlying matmul kernels differ only in reduction-order ULP drift that the sampler absorbs. Q8_0 drifts by 1 ULP (v3 baseline md5 b051f3c1...), tracked as the maintainable reference in docs/TODO.md (TODO-006).

The BF16 main model round-trips byte-for-byte through the converter (bytes.chunks_exact(2).flat_map(|c| bf16->f32) is a lossless decode), so a model output loaded from breeze-tts-2-BF16.gguf is bit-exact with the original PyTorch BF16 checkpoint after a single forward pass.

Checksums

bcb6e27cc31949754bfd1a48ad914c41e5204b6259720ca1c469f7cd4ed91313  breeze-tts-2-BF16.gguf
b1bff9eb99d52a2438404d4bb4f3e20a61a6bcff1dfeea41ae5eec51860e1531  breeze-tts-2-F16.gguf
1c0b4a736091f78dd43fc61846e481194482f20418d275e815b0adfc9514d39b  breeze-tts-2-F32.gguf
be226657da01b6089445fbc338d7aa46b1b70af56f159b60e4e4a5a589243440  breeze-tts-2-mmproj-F32.gguf
7e4c2f1a30b6d25746eda88ecfdbb0272f58195c98f14b0dcf524cd72d8c61f4  breeze-tts-2-Q4_0.gguf
ace2a8dbd1a2047ebfeb0f3f8a99fc426b13531cf5872e8c90e10a017e0df54a  breeze-tts-2-Q8_0.gguf

Limitations

• —Q4_0 output quality degrades: 4-bit quantisation noise on the bulk attn / mlp weights propagates through the TTS frame counter, producing ~120 frames where BF16 produces ~35 (the model keeps generating audio tokens because the noisy hidden states don't fall below the stop threshold). Q4K and per-tensor mixed-precision paths are tracked in `docs/TODO.md` TODO-004. Until they ship, use Q80 for any real deployment.
• —CPU only: the validated Rust build runs CPU-only. Vulkan paths exist for other models but are not exercised against Breeze.
• —No streaming API: the Rust CLI is one-shot per WAV. No WebSocket or microphone capture.
• —Voice direction (instruction + reference together) is not routed: with both --instruction and --ref-audio set, the reference is consumed but CFG still follows the instruction; the reference-aware CFG dual path is not implemented. Use either instruction OR reference, not both.
• —Vocal events ((laugh), [笑], etc.) are not parsed; the text encoder treats them as plain BPE tokens.
• —Quantization may change output relative to the BF16 reference. The --parity-trace feature in the Rust CLI emits per-step JSONL traces that can be diffed against the upstream PyTorch reference for a given prompt; the BF16 main model output is bit-exact for the validated fixtures.

License and Attribution

The source model is provided by BreezeBlue under the BreezeBlue Research and Non-Commercial License. The model weights, checkpoints, adapters, derivative models, and self-hosted outputs are governed by that license. A BreezeBlue paid subscription does not grant commercial rights to the open-weight model or to self-hosted outputs — see the original license text.

The audio tokenizer is derived from Qwen3-TTS (Apache License 2.0). The Rust inference runtime in this repository is Apache License 2.0. This conversion redistributes GGUF weight files only; it does not modify the model weights themselves.