acul3/Qwen3-TTS-1.7B-Base-ExecuTorch

Qwen3-TTS Voice Clone Pipeline (1.9B params total)

Input: text + reference audio (3-5s voice sample)
                    │
        ┌───────────┴───────────┐
        ▼                       ▼
  Speaker Encoder (12M)    Speech Tokenizer
  TDNN → AttPool → FC     (encode ref audio
  ref_audio → x_vector     to codec codes)
  [1, 2048]                [T, 16]
        │                       │
        └───────┬───────────────┘
                ▼
         Talker LM (1.7B)
         Qwen3, 28 layers, GQA 16/8
         dim=2048, audio vocab=3072
         Autoregressive codec generation
                │
                ▼
         Code Predictor (175M)
         Predict 15 additional codebooks
         per token (residual VQ)
                │
                ▼
         Vocoder (154M)
         Codec tokens → 24kHz PCM audio
                │
                ▼
         Output: speech waveform (.wav)

1. ref_audio (24kHz, 3-5s) → mel_spectrogram → speaker_encoder.pte → x_vector [1, 2048]
2. ref_audio → speech_tokenizer.encode() → ref_codes [T, 16]  (runs on CPU, not exported)
3. text → Qwen2 tokenizer → input_ids
4. Embed: text_embedding(input_ids) + codec_embedding(ref_codes) + x_vector → inputs_embeds
5. talker.pte(inputs_embeds, kv_cache, ...) → codec logits  (autoregressive loop)
6. code_predictor.pte(hidden_states) → codebooks 2-16  (per step)
7. All codec codes → vocoder.pte → 24kHz PCM waveform

[BOS] [text tokens...] [speaker x-vector] [ref audio codes...] [SEP] [generated audio codes...]

from huggingface_hub import hf_hub_download
from executorch.runtime import Runtime
import torch
import numpy as np

REPO = "acul3/Qwen3-TTS-1.7B-Base-ExecuTorch"

# Download INT8 models
spk_path = hf_hub_download(REPO, "speaker_encoder_int8.pte")
talker_path = hf_hub_download(REPO, "talker_int8.pte")
cp_path = hf_hub_download(REPO, "code_predictor_int8.pte")
voc_path = hf_hub_download(REPO, "vocoder_int8.pte")
emb_path = hf_hub_download(REPO, "talker_embeddings.pt")
cp_extras_path = hf_hub_download(REPO, "code_predictor_extras.pt")

# Load ExecuTorch runtime
runtime = Runtime.get()
speaker_enc = runtime.load_program(spk_path).load_method("forward")
vocoder = runtime.load_program(voc_path).load_method("forward")

# Load embeddings (used in Python orchestration)
embeddings = torch.load(emb_path, weights_only=True)
cp_extras = torch.load(cp_extras_path, weights_only=True)

# For full generation, see scripts/test_e2e.py

import org.pytorch.executorch.Module

// Load all 4 modules
val speakerEnc = Module.load("speaker_encoder_int8.pte")
val talker = Module.load("talker_int8.pte")
val codePred = Module.load("code_predictor_int8.pte")
val vocoder = Module.load("vocoder_int8.pte")

// Pipeline:
// 1. Process ref audio → mel → speaker_enc.forward() → x_vector
// 2. Build input embeddings (text + speaker + ref codes)
// 3. Autoregressive loop: talker.forward() → codec token → code_pred.forward() → all codebooks
// 4. vocoder.forward(all_codes) → PCM audio
// See scripts/ for full implementation details

# Setup
git clone https://huggingface.co/acul3/Qwen3-TTS-1.7B-Base-ExecuTorch
cd Qwen3-TTS-1.7B-Base-ExecuTorch
pip install executorch torchao qwen-tts

# Export all modules
python scripts/export_speaker_encoder.py
python scripts/export_talker.py
python scripts/export_code_predictor.py
python scripts/export_vocoder.py

# Quantize
python scripts/quantize_all.py

# Validate
python scripts/test_e2e.py

@misc{qwen3tts_executorch_2026,
  title = {Qwen3-TTS-1.7B-Base-ExecuTorch: On-Device Voice Clone TTS},
  author = {Samsul Rahmadani},
  year = {2026},
  url = {https://huggingface.co/acul3/Qwen3-TTS-1.7B-Base-ExecuTorch},
  note = {Converted from Qwen/Qwen3-TTS-1.7B-Base}
}