litert-community/parakeet-tdt_ctc-0.6b-ja

Measured on device (edge-compat, parakeettdtctc0.6bja5sf32): Galaxy S26 · LiteRT 2.2.0 · GPU (ML Drift) · 63.6 ms p50 (2026-08-27); Galaxy S26 · LiteRT 2.2.0 · NPU (QNN/HTP) · 30.3 ms p50 (2026-08-27); browser · Chromium 151 on M4 Max · LiteRT.js 2.5.3 · WebGPU · did not run (2026-08-13). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/parakeet-tdtctc-0.6b-japarakeettdtctc0.6bja5s_f32/CARD.md

Measured on device (edge-compat, parakeettdtctc0.6bja5sf32stateful): Galaxy S26 · LiteRT 2.2.0 · GPU (ML Drift) · 63.0 ms p50 (2026-08-27); Galaxy S26 · LiteRT 2.2.0 · NPU (QNN/HTP) · 30.3 ms p50 (2026-08-27). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/parakeet-tdtctc-0.6b-ja_parakeettdtctc0.6bja5sf32stateful/CARD.md

Measured on device (edge-compat, parakeettdtctc0.6bja5si8): Galaxy S26 · LiteRT 2.2.0 · GPU (ML Drift) · did not run: compilefailed (2026-08-27); Galaxy S26 · LiteRT 2.2.0 · NPU (QNN/HTP) · did not run: aotcompilefailed (2026-08-27); Raspberry Pi 5 · LiteRT 2.2.0.dev20260804 · CPU/XNNPACK, 4 threads · 110 ms p50 (2026-08-31); browser · Chromium 151 on M4 Max · LiteRT.js 2.5.3 · WebGPU · did not run (2026-08-13). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/parakeet-tdtctc-0.6b-ja_parakeettdtctc0.6bja5s_i8/CARD.md

Measured on device (edge-compat, parakeettdtctc0.6bja5si8stateful): Galaxy S26 · LiteRT 2.2.0 · GPU (ML Drift) · did not run: compilefailed (2026-08-27); Galaxy S26 · LiteRT 2.2.0 · NPU (QNN/HTP) · did not run: aotcompilefailed (2026-08-27); Raspberry Pi 5 · LiteRT 2.2.0.dev20260804 · CPU/XNNPACK, 4 threads · 6.89 ms p50 (2026-08-31). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/parakeet-tdtctc-0.6b-japarakeettdtctc0.6bja5si8stateful/CARD.md

Parakeet TDT-CTC 0.6B (ja) — LiteRT

On-device Japanese speech recognition, fully on the mobile GPU.

`nvidia/parakeet-tdt_ctc-0.6b-ja` (FastConformer hybrid TDT-CTC, 0.6B params, Japanese with punctuation) converted to LiteRT .tflite for the CompiledModel API. The TDT (Token-and-Duration Transducer) branch is exported as two signatures — encode (FastConformer encoder) and decode (LSTM prediction network + joint) — the same layout the official LiteRT speech recognition sample uses for parakeet-tdt-0.6b-v3, so the sample's TdtDecoder runs it unchanged.

[image]

Files

Both models take a 5-second window: log-mel features [1, 80, 500] → encode → [1, 1024, 63] → decode (with [1, 64] int32 token ids and two [2, 1, 640] LSTM states) → logits [1, 63, 64, 3078] (3072 tokens + blank + 5 TDT durations).

Signatures

Preprocessing (same as NeMo): 16 kHz mono, preemphasis 0.97, n_fft 512, win 25 ms, hop 10 ms, 80 mel bins, per-feature mean/var normalization. TDT decoding constants: blank id 3072, durations [0, 1, 2, 3, 4].

Measured on device (Pixel 8a, Tensor G3 / Mali-G715, LiteRT 2.1.5)

A 5 s window typically needs ~20 decode calls with the stateless loop, so f32 on the Pixel 8a GPU transcribes 5 s of audio in roughly 2.1 s end-to-end. Both signatures delegate fully (1862/1862 and 2083/2083 nodes on LITERT_CL).

⚠ Known limitation: the i8 variant fails to compile on Mali GPUs (Pixel 8a, Unable to parse bc coord for BATCH axis in ML Drift OpenCL codegen, reproduced on LiteRT 2.1.3 and 2.1.5). On the Pixel 8a the i8 model runs on CPU (verified, numbers above). The f32 variant compiles and runs on the Pixel 8a GPU. Devices from the official parakeet-tdt-0.6b-v3 validation list (Adreno / Tensor G5) use the same graph layout and quant recipe. One Adreno device has since been measured here: on a Galaxy S26 (Snapdragon 8 Elite Gen 5) the i8 files also fail to compile on the GPU, while the f32 files run on that GPU — and on the Hexagon NPU after an ahead-of-time compile. See Snapdragon NPU (Hexagon) below. Tensor G5 was not tested by this conversion.

Conversion fidelity

Measured on a 137 s CC0 Japanese test read (日本国憲法前文, Wikimedia Commons), processed as 28 × 5 s windows:

On-device (Pixel 8a GPU, f32): greedy TDT token ids reproduce the desktop/NeMo decode exactly; encoder output correlation 0.9999 (max abs diff 0.07, fp16 GPU arithmetic). Desktop f32 vs NeMo fp32: encode correlation 1.000, max abs diff 1.6e-05.

Reference quality of the base model (NeMo, from the upstream card): CER 6.4 (JSUT basic5000), 7.1 (Common Voice 8), 9.0 (TEDxJP-10K).

Stateful decoding (decode_1)

The _stateful files add a single-step decode_1 signature (LSTM states carried between calls) next to a 4-token decode, the same layout as parakeet-tdt-0.6b-v3's stateful files. Measured on the Pixel 8a GPU (f32): compile 7.6 s, decode 16 ms, decode_1 8 ms per call — the 28-window sweep that takes 18.2 s with the stateless file finishes in 3.0 s on a desktop CPU (~6×), and a 5 s window on the Pixel 8a GPU drops from ~2.1 s to ~0.45 s of decode+encode work.

Accuracy note: with correct RNN-T stateful semantics the f32 stateful file reproduces NeMo transcribe() 28/28 windows (CER 0.0000) — identical to the stateless file. The TdtDecoder.kt shipped in the litert-samples app today has two stateful-path defects (state adoption on blank steps; switching to decode_1 one token early) that cost accuracy (16/28, CER 6.25% on the same sweep); a fix is proposed in litert-samples#278. Until it merges, prefer the stateless files with the stock app, or apply the two-line fix. (The minimal-usage snippet below uses the stateless file; for stateful decoding, advance the LSTM states only on non-blank emissions and fill the 4-token array before switching to decode_1.)

Minimal usage — Python

import numpy as np
from ai_edge_litert.compiled_model import CompiledModel
from ai_edge_litert.hardware_accelerator import HardwareAccelerator
from tokenizers import Tokenizer

model = CompiledModel.from_file(
    "parakeet_tdt_ctc_0.6b_ja_5s_i8.tflite",
    hardware_accel=HardwareAccelerator.CPU,  # or GPU on device
)
tok = Tokenizer.from_file("tokenizer.json")
BLANK, V, DUR = 3072, 3073, 5  # blank id, tokens+blank, TDT durations

# features: [1, 80, 500] float32 log-mel of a 5 s / 16 kHz clip
# (NeMo AudioToMelSpectrogramPreprocessor: preemph 0.97, n_fft 512,
#  win 25 ms, hop 10 ms, 80 mels, per-feature normalization)
enc_idx = model.get_signature_index("encode")
enc_in = model.create_input_buffers(enc_idx)
enc_out = model.create_output_buffers(enc_idx)
enc_in[0].write(features.ravel().astype(np.float32))
model.run_by_index(enc_idx, enc_in, enc_out)
enc = enc_out[0].read(1024 * 63, np.float32)

dec_idx = model.get_signature_index("decode")
dec_in = model.create_input_buffers(dec_idx)
dec_out = model.create_output_buffers(dec_idx)
tokens = np.zeros(64, np.int32); tokens[0] = BLANK
emitted, t, k = [], 0, 0
while t < 63:
    dec_in[0].write(enc); dec_in[1].write(tokens)
    dec_in[2].write(np.zeros(2 * 640, np.float32))
    dec_in[3].write(np.zeros(2 * 640, np.float32))
    model.run_by_index(dec_idx, dec_in, dec_out)
    lg = dec_out[0].read(63 * 64 * (V + DUR), np.float32).reshape(63, 64, -1)
    vec = lg[t, k]
    tok_id, dur = int(vec[:V].argmax()), int(vec[V:].argmax())
    if tok_id != BLANK:
        emitted.append(tok_id); k += 1
        if k >= 63: break
        tokens[k] = tok_id
    t += 1 if (dur == 0 and tok_id == BLANK) else dur

print(tok.decode(emitted, skip_special_tokens=True))

Minimal usage — Kotlin (Android)

import com.google.ai.edge.litert.Accelerator
import com.google.ai.edge.litert.CompiledModel

val model = CompiledModel.create(
    modelPath, CompiledModel.Options(Accelerator.GPU), null)

val encIn = model.createInputBuffers("encode")
val encOut = model.createOutputBuffers("encode")
encIn[0].writeFloat(logMel)          // [1, 80, 500] as FloatArray(40000)
model.run(encIn, encOut, "encode")

val decIn = model.createInputBuffers("decode")
val decOut = model.createOutputBuffers("decode")
val tokens = IntArray(64).also { it[0] = 3072 }  // blank start
// Greedy TDT loop: write tokens + zero LSTM states, run "decode",
// read [63, 64, 3078] logits, argmax tokens [0..3072] and durations
// [3073..3077] — identical to TdtDecoder.kt in the official
// litert-samples speech_recognition app, with blank id 3072.
decIn[1].writeInt(tokens)
decIn[2].writeFloat(FloatArray(2 * 640))
decIn[3].writeFloat(FloatArray(2 * 640))
model.run(listOf(encOut[0], decIn[1], decIn[2], decIn[3]), decOut, "decode")
val logits = decOut[0].readFloat()

Conversion

Converted with the official litert-samples `speech_recognition/convert` pipeline (litert-torch), through the ParakeetTDT path with the -ja model's constants (vocab 3072, blank 3072, durations [0,1,2,3,4], 80 mel bins). The int8 variant uses the pipeline's dynamic-range recipe (int8 channelwise weights, float32 activations).

Snapdragon NPU (Hexagon)

• —parakeet_tdt_ctc_0.6b_ja_5s_f32.tflite — the NPU is 2.10x faster than the GPU (30.33 ms against 63.61 ms) and loads 14.26x faster (420 ms against 5984 ms).
• —parakeet_tdt_ctc_0.6b_ja_5s_f32_stateful.tflite — the NPU is 2.08x faster than the GPU (30.32 ms against 63.03 ms) and loads 15.17x faster (394 ms against 5977 ms).
• —parakeet_tdt_ctc_0.6b_ja_5s_i8.tflite — neither accelerator produced a usable row on the S26. NPU: the ahead-of-time compile for SM8850 failed on the host. GPU: LiteRtException: Failed to compile model.
• —parakeet_tdt_ctc_0.6b_ja_5s_i8_stateful.tflite — neither accelerator produced a usable row on the S26. NPU: the ahead-of-time compile for SM8850 failed on the host. GPU: LiteRtException: Failed to compile model.

Measured on a Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status NONE throughout. Headroom 0.70–0.73, where 1.0 is the throttling threshold.

The NPU rows marked AOT ran an artifact compiled ahead of time for SM8850 (ai-edge-litert 2.2.0 + QAIRT 2.47.0), not the published file. That artifact is not distributed here; the compile is one command in the NPU guide.

GPU wiring: GPU guide.

Raspberry Pi 5 (CPU)

Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with the LiteRT `benchmark_model` tool from litert-cli-nightly 0.2.0.dev20260805: CPU inference (XNNPACK, 4 threads), 3 invocations per file of 10 warm-up plus 50 timed runs (the tool caps a phase at 150 s, so very slow graphs run fewer — the Runs column is the actual timed total). The latency is the median across invocations; the spread is the min–max over all timed runs. No thermal throttling occurred during these runs (vcgencmd get_throttled stayed 0x0).

License

CC-BY-4.0, inherited from `nvidia/parakeet-tdt_ctc-0.6b-ja`.