litert-community/Qwen3-TTS-12Hz-0.6B-Base

LiteRT is Google's on-device runtime, the new name for TensorFlow Lite (Android: com.google.ai.edge.litert:litert), and litert-torch, the renamed ai-edge-torch, is its PyTorch converter: a PyTorch model converted unmodified with litert_torch.convert matched the original to 4e-7 on a Galaxy S26 (measured, LiteRT 2.2.0, Android 16, 2026-09-05).

Qwen3-TTS-12Hz-0.6B-Base — LiteRT

Qwen3-TTS-12Hz-0.6B-Base (Apache-2.0) converted to LiteRT (.tflite) for fully on-device text-to-speech with 3-second voice cloning, in 10 languages at 24 kHz.

Qwen3-TTS is a speech LM: a Qwen3-style talker predicts 12.5 Hz frames of 16 codec tokens (first codebook by the talker, 15 residual codebooks by an inner "MTP" transformer), and a neural codec decoder renders PCM. LiteRT-LM's Engine decode loop does not support this generation structure yet, so the model runs as three LiteRT graphs driven by a host-side loop (LiteRT Compiled Model pattern). A complete Python reference pipeline and all conversion scripts live in the litert-samples sample: `compiled_model_api/text_to_speech_lm`.

Quick start (Python, desktop)

git clone -b qwen3-tts-sample https://github.com/john-rocky/litert-samples.git
cd litert-samples/compiled_model_api/text_to_speech_lm/python
pip install -r requirements.txt
python synthesize.py --text "Hello from LiteRT running fully on device." --output hello.wav

The script downloads this repository automatically (~1.4 GB for the default int4 configuration) and speaks in the bundled demo voice. Enroll your own voice from ~3 s of audio with the sample's conversion/extract_speaker_embedding.py, then pass --speaker my_voice.npy.

Android app

The same sample ships an Android app (Kotlin, Compiled Model API, CPU) under compiled_model_api/text_to_speech_lm/kotlin_cpu/android/: build with Android Studio or ./gradlew :app:installDebug, then run ./install_to_device.sh to download the model files from this repository and push them to the device. Device-verified on Pixel 8a. With the reference mtp_fp32 + codec_decoder_fp32 graphs: RTF ≈ 6.7. The app auto-selects the fast graphs when present: mtp_folded_int8 drops the MTP from ≈333 to ≈68 ms/frame (~5×), and the split codec_partA/codec_partB drops the codec from ≈114 to ≈40 ms/frame (~2.5×). Together the end-to-end RTF falls to ≈2.06 (~3.2× vs the reference graphs), ASR-lossless.

Files

Accuracy

• —Each graph is numerically verified against the PyTorch reference: talker bit-exact at torch level and correlation 1.0 / top-1 100% as .tflite; MTP 15/15 greedy tokens; codec decoder correlation 1.0 (max abs diff 1.8e-5).
• —End to end with talker_fp32 + greedy: token-for-token identical codes to the reference implementation, waveform correlation 1.000000, ASR round-trip returns the input sentence.
• —talker_int4 (data-free blockwise-32 OCTAV) produces a different but valid sampling trajectory; outputs transcribe identically under ASR round-trip. Channelwise int8/int4 quantization (the tooling default) degenerates on this model family — use blockwise granularity.

Performance (Apple M4 Max, CPU/XNNPACK)

The MTP inner loop dominates (a 78M-parameter transformer streams its weights 17 times per frame). mtp_folded_int8.tflite folds those 17 invokes into one graph and quantizes it: on the M4 Max the MTP drops to ~41 ms/frame, and on a Pixel 8a from ~333 to ~68 ms/frame. The fold is token-identical to the reference; the dynamic-int8 weights give a different-but-intelligible trajectory (ASR round-trip exact). The codec then dominates, and codec_partA/codec_partB split it so the conv-heavy back half runs in fp16 (~2.5× on device). Together the end-to-end RTF drops from ≈6.7 to ≈2.06 on a Pixel 8a (≈1.44 on M4 Max), ASR-lossless. Conversion scripts: `export_mtp_folded.py` / `gptq_mtp_folded.py` / `export_codec_split.py`. Remaining lever: the talker (now ~52 ms/frame).

Android (Pixel 8a)

Android figures use the standard TFLite `benchmark_model` on a Pixel 8a (Tensor G3, Android 16) — 5 warm-up runs then 20 timed runs, CPU at 4 threads.

Nothing here is faster on the GPU on the Pixel 8a, so run this pipeline on the CPU on that phone. On a Snapdragon phone the picture differs — the Hexagon NPU runs mtp_fp32.tflite at 4.33 ms and codec_partA.tflite at 2.29 ms, and the GPU runs talker_fp32.tflite; see the Snapdragon NPU (Hexagon) section below.

Snapdragon NPU (Hexagon)

• —codec_decoder_fp32.tflite — neither accelerator produced a usable row on the S26. NPU: the benchmark process was killed, most likely out of memory. GPU: LiteRtException: Failed to compile model.
• —codec_partA.tflite — the NPU runs it at 2.29 ms. The GPU does not — LiteRtException: Failed to compile model.
• —codec_partB.tflite — every attempt ran while the phone was already warm, so no row meets the sweep's thermal condition. Not reported.
• —mtp_folded_int8.tflite — neither accelerator produced a usable row on the S26. Both ended the same way: LiteRtException: Failed to compile model.
• —mtp_fp32.tflite — the NPU runs it at 4.33 ms. The GPU does not — LiteRtException: Failed to compile model.
• —talker_fp32.tflite — the GPU runs it at 127.6 ms. The NPU does not — LiteRtException: Failed to compile model.
• —talker_int4.tflite — neither accelerator produced a usable row on the S26. NPU: the request fell back to the CPU (XNNPACK) without saying so, which leaves no number attributable to the NPU. GPU: the benchmark process was killed, most likely out of memory.

Measured on a Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status NONE throughout. Headroom 0.62–0.79, where 1.0 is the throttling threshold.

The NPU rows ran the published file unchanged. LiteRT compiled it for the Hexagon on the device at first load. Those first compiles took 2.7 s to 5.7 s here. The load column above is the cached load every later run pays. Recipe and the runtime libraries it needs: NPU guide.

GPU wiring: GPU guide.

Limitations

• —Voice cloning is x-vector mode only (speaker embedding). ICL-mode cloning (reference transcript + codec encoding of the reference audio) additionally needs the codec encoder, which is kept off-device (enrollment-time PyTorch).
• —The prompt prefill is capped at 32 positions (the x-vector prompt is always 10); the KV cache is 1024 (~80 s of audio), generation is capped at 512 frames (~41 s) in the sample.
• —Streaming synthesis (the model's dual-track design supports it) is not implemented in the sample loop yet.

Raspberry Pi 5 (CPU)

Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with the LiteRT `benchmark_model` tool from litert-cli-nightly 0.2.0.dev20260805: CPU inference (XNNPACK, 4 threads), 3 invocations per file of 10 warm-up plus 50 timed runs (the tool caps a phase at 150 s, so very slow graphs run fewer — the Runs column is the actual timed total). The latency is the median across invocations; the spread is the min–max over all timed runs. No thermal throttling occurred during these runs (vcgencmd get_throttled stayed 0x0).

License

Apache-2.0, inherited from the base model by the Qwen team, Alibaba Group.