EvoAwaken-Workshop/dots-tts-base-gguf

dots.tts Base - GGUF

This repository contains a GGUF conversion of the dots.tts-base checkpoint. The files are intended to be used with a runtime that supports the custom dotstts audio-generation components, such as `rust-model-inference`.

References

• —Original checkpoint: `dots-studio/dots.tts-base`
• —Official dots.tts implementation: `studio-dots-ai/dots.tts`
• —GGUF exporter and Rust runtime: `Liyulingyue/rust-model-inference`
• —Exporter source used for this conversion: `tools/dots/convert_dots_tts.py`
• —Exporter contract tests: `tools/dots/test_convert_dots_tts.py`
• —Rust dots.tts runtime: `src/models/dots`
• —Technical report: `dots.tts Technical Report`

This repository contains two files because the model is split into an LLM file and an audio pipeline file:

These files are not Q8 or Q4 quantizations. The BF16 suffix identifies the primary model-weight precision. In the mmproj file, the generation core uses BF16 while the speaker encoder and AudioVAE vocoder use F32.

The LLM file uses standard Qwen2 GGUF tensor names. The second file uses GGUF general.architecture = clip metadata together with the custom dotstts.* metadata required by the Rust runtime. A generic text-only GGUF runner is not sufficient to run the complete TTS pipeline.

What Was Exported

The exporter converts the original dots.tts-base source directory into two GGUF v3 files:

dots.tts-base/
|-- model.safetensors
|-- speaker_encoder.safetensors
|-- vocoder.safetensors
|-- llm_config.json
|-- config.json
|-- tokenizer_config.json
|-- vocab.json
|-- added_tokens.json
|-- merges.txt
`-- latent_stats.pt

The source checkpoint is read directly from these files. The exporter does not require PyTorch or the safetensors Python package. It uses a small Python reader for the Safetensors headers and payloads, standard-library parsers for the configuration/tokenizer files, and NumPy only for decoding the array data stored in latent_stats.pt.

The exporter supports both base and edit variants, but this repository contains the base output only.

Export Command

Clone the `rust-model-inference` repository and download the `dots-studio/dots.tts-base` source checkpoint:

git clone https://github.com/Liyulingyue/rust-model-inference.git
cd rust-model-inference

hf download dots-studio/dots.tts-base \
  --local-dir models/dots.tts-base

Install the exporter's only Python dependency, then run the `convert_dots_tts.py` script:

python3 -m pip install numpy

python3 tools/dots/convert_dots_tts.py \
  models/dots.tts-base \
  --variant base \
  --out-dir models

The command creates:

models/dots-tts-base-BF16.gguf
models/dots-tts-base-mmproj-BF16.gguf

The variant is restricted to base or edit. If --variant is omitted, the exporter infers it from the source directory name. Existing output files are not overwritten unless --overwrite is passed:

python3 tools/dots/convert_dots_tts.py \
  models/dots.tts-base \
  --variant base \
  --out-dir models \
  --overwrite

Conversion Pipeline

1. Validate the source checkpoint

Before reading model weights, the exporter checks that all required source files exist. It also validates the selected variant and refuses to continue when an output already exists without an explicit overwrite request.

The source tensors are checked for both dtype and shape. Dimensions are taken from llm_config.json and config.json, rather than being guessed from the destination names.

2. Read configuration and tokenizer data

The exporter reads:

• —llm_config.json for the Qwen2 LLM dimensions and attention settings;
• —config.json for the patch encoder, DiT, speaker, and vocoder settings;
• —tokenizer_config.json for BOS/EOS IDs;
• —vocab.json, added_tokens.json, and merges.txt for the tokenizer;
• —latent_stats.pt for the latent mean and variance used by the runtime.

The tokenizer vocabulary and added control tokens are merged into GGUF metadata. Missing vocabulary IDs are represented by reserved token names so that the output vocabulary remains contiguous.

3. Build the LLM GGUF

The LLM output is written as GGUF v3 with the Qwen2 architecture metadata:

general.architecture         = qwen2
general.name                 = dots.tts-base
general.file_type            = 32
general.quantization_version = 2

The exporter also writes the Qwen2 block count, context length, embedding length, feed-forward length, attention head counts, RMS norm epsilon, RoPE settings, vocabulary size, and tokenizer metadata.

The main tensor conversion is:

The input and output embeddings are tied: the same source embedding payload is written to both token_embd.weight and output.weight.

Most LLM weight tensors are preserved as BF16. The final normalization tensor is converted from BF16 to F32 because the runtime expects the norm weights in that representation. This is a storage conversion, not a Q4 or Q8 quantization pass.

4. Build the audio mmproj GGUF

The second output is also GGUF v3. It is marked as an audio-capable CLIP-style projector container and carries the custom model metadata:

general.architecture          = clip
clip.has_vision_encoder      = false
clip.has_audio_encoder       = true
clip.has_gen_audio_encoder   = true
clip.audio.projector_type    = dotstts_spkenc
clip.gen.audio.projector_type = dotstts_gen

For the current dots.tts runtime contract, the generated metadata includes:

The mmproj contains the following model parts:

1. 1.Latent mean and variance tensors from latent_stats.pt.
2. 2.Projection heads connecting the LLM hidden state, latent state, speaker x-vector, and EOS prediction.
3. 3.The patch encoder and its transformer blocks.
4. 4.The flow-matching DiT, including time embedding, attention, feed-forward, AdaLN modulation, and output layers.
5. 5.The CAM++-style speaker encoder and its resampling kernel.
6. 6.The AudioVAE/vocoder tensors used to decode latent frames into waveform samples.

The BF16 tensors in the core model are kept as BF16. Speaker and vocoder tensors are emitted as F32. Required integer tensors, including batch normalization counters, are retained as I64.

5. Normalize tensor layout and derived weights

The exporter reverses tensor dimensions when writing GGUF because the source checkpoint uses the PyTorch dimension order while GGUF stores dimensions in the engine's tensor order.

Vocoder layers using legacy weight normalization are materialized before they are written. For each output-channel row, the exporter computes the F32 norm of weight_v, then emits the plain weight:

weight = (weight_g / norm(weight_v_row)) * weight_v_row

This removes the need for the inference runtime to reconstruct weight normalization at load time. Fixed resampling/filter tensors are emitted as part of the GGUF payload as well.

6. Write and read back the GGUF files

The custom writer uses the following layout:

1. 1.GGUF v3 header.
2. 2.Metadata key/value records.
3. 3.Tensor directory records.
4. 4.32-byte-aligned tensor data.

Each output is first written to a temporary file. The exporter then reads the file back and checks:

• —all metadata values;
• —tensor names, GGML types, dimensions, and byte lengths;
• —the raw byte payload of every emitted tensor.

Duplicate metadata keys, duplicate tensor names, invalid tensor dimensions, unexpected source dtypes, and configuration shape mismatches are rejected.

Verification

The exporter has `focused contract tests`:

cd tools/dots
python3 -m unittest -v test_convert_dots_tts.py

The tests cover:

• —source component discovery and output pairing;
• —BF16 payload preservation and GGUF metadata readback;
• —exact F32 weight-normalization folding fixtures;
• —duplicate metadata and tensor rejection;
• —explicit base/edit variant validation;
• —source dtype validation;
• —source tensor shape validation.

The conversion command itself also performs full output-file and tensor payload readback validation before publishing each file.

Usage With rust-model-inference

Build the `rust-model-inference` engine:

cargo build --release

Run Base text-to-speech:

cargo run --release --bin rust-model-inference -- \
  --model dots-tts-base-BF16.gguf \
  --mmproj dots-tts-base-mmproj-BF16.gguf \
  --tts \
  --prompt "Hello, this is a dots.tts test." \
  --language en \
  --out output.wav

The generated WAV is mono PCM16 at 48 kHz.

Reference-audio conditioning

Reference audio can be used for speaker conditioning. When reference text is also supplied, the reference audio can provide prompt conditioning in addition to speaker conditioning:

cargo run --release --bin rust-model-inference -- \
  --model dots-tts-base-BF16.gguf \
  --mmproj dots-tts-base-mmproj-BF16.gguf \
  --tts \
  --prompt "This sentence uses the reference voice." \
  --language en \
  --ref-audio reference.wav \
  --ref-text "Text spoken in the reference audio." \
  --out output.wav

Reference audio must be a PCM16 WAV. The runtime mixes multi-channel input to mono and resamples input that is not already 48 kHz. --ref-text requires --ref-audio.

Generation controls

Useful controls include:

The model metadata provides defaults for NFE, classifier-free guidance, speaker scale, and EOS threshold. The runtime reads these values from the mmproj file instead of duplicating them in the command line.

Limitations

• —The two GGUF files must be used together for full TTS generation.
• —The custom dotstts.* audio pipeline is runtime-specific; a generic text-generation GGUF loader will not provide complete speech synthesis.
• —This repository is the base variant. The separate edit variant uses the same exporter but has different generation inputs and outputs.
• —No Q4/Q8 quantized version is included here. The exported tensors primarily use BF16 and F32 storage.
• —Quality, speed, and memory use depend on the runtime, CPU architecture, and generation settings.

Provenance and License

The source checkpoint is `dots-studio/dots.tts-base`, published by the dots.tts team under Apache-2.0. The upstream implementation is available in `studio-dots-ai/dots.tts`. This model card documents the GGUF conversion and does not replace the usage terms of the original checkpoint.

Before redistributing or using the files commercially, review the original checkpoint's `license and model card` and any applicable terms for its tokenizer and audio components.