q3146dq4/Supertonic-3-LiteRT-Static-MultiPreset-GELU-WI8-AFP32

Supertonic-3 LiteRT Static MultiPreset GELU — WI8 / AFP32

Quantized LiteRT/TFLite build of Supertonic-3 based on the Static MultiPreset GELU-fused model.

This repository keeps the same static multi-preset signature layout as the FP32 model while reducing model size with post-training Dynamic WI8 / AFP32 quantization.

Final Model Profile

Total model size is reduced by approximately 44.3% compared with the FP32 Static MultiPreset bundle.

The Text Encoder was also quantized and tested, but the WI8 candidate failed the quality A/B test. The final release therefore intentionally keeps the Text Encoder in FP32.

Quantization Method

The models were quantized using Post-Training Quantization (PTQ) with Google AI Edge Quantizer.

Target format:

Weights     : INT8
Activations : FP32
Inputs      : FP32
Outputs     : FP32
Runtime     : LiteRT / TFLite

This is Dynamic WI8 / AFP32.

It is not W8A8 and does not use QAT.

Graph Details

Duration Predictor

The Duration Predictor uses full Dynamic WI8 / AFP32 quantization.

FP32 size      : 5.07 MiB
WI8 size       : 4.03 MiB
INT8 tensors   : 259
Signatures     : 7

All seven static T signatures are preserved.

Text Encoder

A Dynamic WI8 / AFP32 Text Encoder candidate was successfully generated.

FP32 size           : 37.94 MiB
WI8 candidate size  : 18.74 MiB
INT8 tensors        : 406

However, end-to-end quality testing showed unacceptable degradation.

The WI8 candidate was therefore rejected and the final bundle uses the original FP32 Text Encoder.

Final Text Encoder : FP32
Final size         : 37.94 MiB

Vector Estimator

The Vector Estimator uses full Dynamic WI8 / AFP32 quantization.

FP32 size      : 276.67 MiB
WI8 size       : 150.88 MiB
INT8 tensors   : 6,076
Signatures     : 49

All T × L signatures are preserved.

The Vector Estimator provides the largest model-size reduction in the bundle.

Because the VE is executed once per diffusion step, its runtime performance has the largest effect on total TTS inference time.

Vocoder

The fully quantized Dynamic WI8 Vocoder was successfully generated:

Full Dynamic WI8 size : 28.00 MiB
INT8 tensors          : 231

However, fully quantizing all Vocoder layers caused excessive quality degradation.

A selective PTQ search was therefore performed over 33 logical Vocoder layers, shared across all seven L signatures.

Final selective configuration:

Logical layers total : 33

WI8 logical layers   : 27
FP32 logical layers  : 6

Physical scopes:
INT8                  : 189
FP32                  : 42
Total                 : 231

Final size            : 39.67 MiB

The six FP32 logical layers are intentionally retained because quantizing them caused the measured quality metric to exceed the configured acceptance threshold.

The final Vocoder is therefore:

Selective WI8 / AFP32

Static MultiPreset Layout

The model uses fixed static presets instead of one fully dynamic graph.

Presets

T = 32, 48, 64, 80, 96, 112, 128
L = 32, 48, 64, 80, 96, 112, 128

Signatures

Examples:

Duration Predictor:
T32
T48
T64
T80
T96
T112
T128

Text Encoder:
T32
T48
T64
T80
T96
T112
T128

Vector Estimator:
T32_L32
T32_L48
...
T128_L112
T128_L128

Vocoder:
L32
L48
L64
L80
L96
L112
L128

All signatures were verified to load correctly with the tested LiteRT runtime, including LiteRT 2.2.0.

Automatic Bucket Selection

At runtime, select the smallest preset that can contain the requested input.

Text bucket

T = smallest preset >= token_count

Latent bucket

L = smallest preset >= ceil(duration_seconds × 44100 / 3072)

Audio / latent configuration:

Sample rate              : 44.1 kHz
Samples per latent frame : 3072
Latent channels          : 144

CPU Benchmark

Benchmark configuration:

FP32 Static MultiPreset
vs
Final WI8 / AFP32 Static MultiPreset

Backend       : XNNPACK CPU
Threads       : 8
Warmup        : 1
Measured runs : 5
Result        : Median

Steps:
4 / 8 / 16 / 32 / 64

The WI8 model is not universally faster on desktop XNNPACK.

Dynamic WI8 overhead is significant on small buckets, while larger buckets benefit more from reduced weight bandwidth and computation.

Representative Results

General trend on the tested desktop CPU:

T32–T64   : usually slower than FP32
T80       : around break-even
T96–T128  : generally faster than FP32

The exact crossover point is hardware-dependent.

Mobile ARM/XNNPACK behavior may differ significantly from desktop results, so testing on the target device is recommended.

Quality Validation

The final selective Vocoder passed the configured baseline-relative quality gate.

The search baseline used:

Duration Predictor : WI8
Text Encoder       : FP32
Vector Estimator   : WI8
Vocoder            : FP32

Search threshold:

Baseline log-spectrum RMSE + 1.5 dB

Final selective Vocoder result:

FP32 logical layers : 6 / 33
WI8 logical layers  : 27 / 33

Final size          : 39.67 MiB
INT8 tensors        : 189

Quality gate        : PASS

The automatic metric is intended as a regression guard and does not replace listening tests.

For real deployment, listening tests on the target device are still recommended.

Files

duration_predictor.tflite
text_encoder.tflite
vector_estimator.tflite
vocoder.tflite
multi_preset_manifest.json

Final precision profile:

Duration Predictor : WI8 / AFP32
Text Encoder       : FP32
Vector Estimator   : WI8 / AFP32
Vocoder            : Selective WI8 / AFP32

Model Size Comparison

FP32 Static MultiPreset
--------------------------------
Duration Predictor :   5.07 MiB
Text Encoder       :  37.94 MiB
Vector Estimator   : 276.67 MiB
Vocoder            :  97.66 MiB
--------------------------------
Total              : 417.33 MiB

WI8 / AFP32 Static MultiPreset
--------------------------------
Duration Predictor :   4.03 MiB
Text Encoder       :  37.94 MiB
Vector Estimator   : 150.88 MiB
Vocoder            :  39.67 MiB
--------------------------------
Total              : 232.52 MiB

Reduction:
417.33 MiB
→ 232.52 MiB

-184.81 MiB
-44.3%

Model Provenance

This model is derived from the official Supertonic-3 model and converted into a Static MultiPreset LiteRT/TFLite graph with GELU fusion before quantization.

It is not a quantization or modification of another third-party TFLite model.

FP32 Static MultiPreset GELU model:

https://huggingface.co/q3146dq4/Supertonic-3-LiteRT-Static-MultiPreset-GELU

This repository contains the corresponding WI8 / AFP32 quantized variant.

Notes

• —All four graphs were included in the quantization evaluation.
• —The final precision of each graph is based on measured acceptance/rejection results.
• —The Text Encoder WI8 candidate was rejected because of quality degradation.
• —The Vocoder uses selective quantization because full Dynamic WI8 significantly degraded quality.
• —The Vector Estimator provides most of the quantization benefit.
• —Small buckets can be slower with Dynamic WI8 because quantization/runtime overhead can outweigh the reduced weight cost.
• —Larger buckets generally benefit more from WI8 execution.
• —The Vector Estimator is executed repeatedly for every diffusion step, so its performance matters much more to total inference time than one-shot stages such as the Vocoder.
• —Device-specific benchmarking is strongly recommended before choosing FP32 or WI8 for production deployment.