UrocyonF/Qwen3-TTS-12Hz-1.7B-NVFP4

Qwen3-TTS-12Hz-1.7B-CustomVoice - NVFP4A16 Quantized

Model Overview

This repository contains a pre-quantized NVFP4A16 derivative of `Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice`.

It is a weight-only post-training quantization of the model's autoregressive speech-generation decoder, stored in the compressed-tensors format. The rest of the model remains in BF16.

Qwen3-TTS CustomVoice model supports:

• —10 languages: Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, and Italian
• —9 preset voices in the CustomVoice variant
• —instruction-aware style control

This derivative is intended for users who want a smaller, faster checkpoint while staying close to the original model's behavior.

Quantization Summary

• —Base model: Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice
• —Quantization type: NVFP4A16, weight-only
• —Storage format: compressed-tensors
• —Quantized scope: autoregressive decoder layers (talker.model.layers.*)
• —Kept in BF16: speech tokenizer, embeddings, text projection, codec/output layers, and other non-decoder components

Approximate repository size: 2.4 GB

Practical Gains

Compared with the BF16 base model, this checkpoint delivered:

• —1.4x faster generation at median RTF (30.6% lower)
• —1.3x lower first-audio latency (24.8% lower)
• —2.1x lower decoder weight memory (52.8% lower)

Evaluation

Benchmark setup

The benchmark numbers below come from local measurements collected on an NVIDIA DGX Spark system.

• —Corpus size: 100 fixed multilingual prompts
• —Language distribution: Japanese 24, English 20, French 14, Chinese 14, German 6, Spanish 6, Italian 6, Korean 6, Russian 2, Portuguese 2
• —Latency metrics: real-time factor (RTF) for standard request generation; first-audio latency and uninterrupted-playback start delay for streaming generation
• —Quality metric: character error rate (CER) computed by transcribing generated audio back with `Qwen3-ASR-1.7B`

Standard request generation

Streaming generation

Relative to the BF16 baseline, this checkpoint measured:

• —30.6% lower median RTF for standard requests and streaming requests
• —24.8% lower median first-audio latency
• —35.1% lower median gapless-start delay
• —no median CER degradation on the saved benchmark set

Repository Contents

• —model.safetensors - quantized checkpoint
• —config.json - model config including quantization metadata
• —generation_config.json - generation defaults
• —preprocessor_config.json - preprocessing metadata
• —tokenizer_config.json, vocab.json, merges.txt - tokenizer assets
• —speech_tokenizer/ - speech tokenizer assets required by the upstream architecture
• —recipe.yaml - quantization scope metadata

Compatibility Notes

This repository contains weights only. Use software that already supports:

• —the upstream Qwen3-TTS architecture
• —pre-quantized compressed-tensors checkpoints
• —NVFP4-style decoder quantization

If you are evaluating software stacks, a recent release of vLLM / vLLM-Omni with audio-model and compressed-tensors support is a reasonable starting point.

Sources

• —Original model: `Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice`
• —Upstream technical report: Qwen3-TTS Technical Report (arXiv:2601.15621)
• —Quantization tooling: `llm-compressor`
• —Quantized checkpoint format: `compressed-tensors`

License Reminder

This repository is a derivative of an upstream model released under Apache-2.0.

Please review and comply with the original model license, notices, and any upstream usage guidance before redistributing, serving, or incorporating this checkpoint into another product or repository.