litert-community/wav2vec2-base-960h-LiteRT

Measured on device (edge-compat, w2v2asrfrontendfp16): Raspberry Pi 5 · LiteRT 2.2.0.dev20260804 · CPU/XNNPACK, 4 threads · 2011 ms p50 (2026-08-31); browser · Chromium 151 on M4 Max · LiteRT.js 2.5.3 · WebGPU · 37.1 ms p50 (2026-08-20). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/wav2vec2-base-960hw2v2asrfrontendfp16/CARD.md

Measured on device (edge-compat, w2v2asrheadfp16): Raspberry Pi 5 · LiteRT 2.2.0.dev20260804 · CPU/XNNPACK, 4 threads · 2820 ms p50 (2026-08-31); browser · Chromium 151 on M4 Max · LiteRT.js 2.5.3 · WebGPU · 51.3 ms p50 (2026-08-20). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/wav2vec2-base-960hw2v2asrheadfp16/CARD.md

wav2vec2-base-960h CTC — LiteRT (GPU)

English speech recognition with wav2vec2-base-960h running fully on the LiteRT `CompiledModel` GPU (ML Drift) — and with zero FFT anywhere: the raw 16 kHz waveform goes straight into the 1D-conv feature extractor, so there is no mel/fbank step even on the host. Character-level CTC (29 chars + specials), greedy decode, no language model.

[image] Real model output: char-CTC word onsets for J.F. Kennedy's 1961 inaugural address (U.S. National Archives recording, public domain).

Ships as two GPU graphs — the fused graph exceeds the Mali whole-graph shader-compile limit (a graph can be op-clean and still fail to compile when fused; each half compiles and runs fully delegated):

Pipeline

16 kHz mono PCM in [-1, 1], zero-padded to the fixed 16 s window → [GPU] conv frontend → [GPU] 12-layer transformer + lm_head → host greedy-CTC over the valid frames

• —Valid frames for n samples: run L=(L-k)//s+1 over the conv stack (10,5)(3,2)(3,2)(3,2)(3,2)(2,2)(2,2) — 50 Hz frames (16 s → 799).
• —Blank id 0 (<pad>), | = word delimiter (tokens.txt, index-ordered).
• —Greedy char-CTC without an LM has the model's known spelling quirks on hard words (e.g. GRAVED/GRAVE); add a beam+LM host-side if you need the last WER point.

Minimal usage — Python

import numpy as np, torch, torchaudio
from ai_edge_litert.interpreter import Interpreter

wave, sr = torchaudio.load("speech.wav")             # 16 kHz mono, [-1,1]
x = torch.zeros(1, 256000); n = min(wave.shape[1], 256000)
x[0, :n] = wave[0, :n]

def run(path, inp):
    it = Interpreter(model_path=path); it.allocate_tensors()
    d = it.get_input_details()[0]
    it.set_tensor(d["index"], inp.astype(np.float32)); it.invoke()
    return it.get_tensor(it.get_output_details()[0]["index"])

feat = run("w2v2_asr_frontend_fp16.tflite", x.numpy())
logits = run("w2v2_asr_head_fp16.tflite", feat)[0]    # [799, 32]

L = n
for k, s in [(10,5),(3,2),(3,2),(3,2),(3,2),(2,2),(2,2)]:
    L = (L - k) // s + 1
tokens = open("tokens.txt").read().splitlines()
out, prev = [], -1
for i in logits[:L].argmax(-1):
    if i != prev and i != 0: out.append(tokens[int(i)])
    prev = i
print("".join(out).replace("|", " ").strip())

Minimal usage — Kotlin (Android)

val frontend = CompiledModel.create(frontendPath, CompiledModel.Options(Accelerator.GPU), null)
val head = CompiledModel.create(headPath, CompiledModel.Options(Accelerator.GPU), null)
val fIn = frontend.createInputBuffers(); val fOut = frontend.createOutputBuffers()
val hIn = head.createInputBuffers(); val hOut = head.createOutputBuffers()

fIn[0].writeFloat(pcm)                        // [-1,1] floats, zero-padded to 256000
frontend.run(fIn, fOut)
hIn[0].writeFloat(fOut[0].readFloat())        // features [1,799,768]
head.run(hIn, hOut)
val logits = hOut[0].readFloat()              // [799 * 32], readback syncs the GPU
// greedy CTC over the valid frames: argmax per frame, drop blanks (id 0) + repeats,
// map through tokens.txt, '|' -> space

On-device performance (Pixel 8a, CompiledModel GPU)

• —frontend 448 ms + head 391 ms per 16 s window (RTF ≈ 0.05); GPU compile 0.7 s + 1.5 s.
• —Device logits vs desktop float reference: corr 0.9928 (valid region), per-frame argmax agreement 97.0 %; transcript matches the desktop reference.

Conversion notes

Converted with litert-torch, numerically exact (tflite vs PyTorch: corr 1.000000): GELU → tanh-GELU; frontend GroupNorm → 4D-reshape group-norm (avoids GATHERND); posconv weight-norm folded to a static weight; the all-valid bidirectional attention mask removed (fixed window → plain SDPA). The CTC head is a plain Linear — logits come out raw.

Snapdragon NPU (Hexagon)

• —w2v2_asr_frontend_fp16.tflite — the NPU compiles this graph and then fails to run it: LiteRtException: Failed to invoke the compiled model. The GPU row below is the only S26 figure for it. A clean compile is not evidence that a model runs.

• —w2v2_asr_head_fp16.tflite — the NPU is 1.37x faster than the GPU (76.18 ms against 104.1 ms) and loads 5.84x faster (226 ms against 1322 ms).

Measured on a Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16), LiteRT CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status NONE throughout except w2v2_asr_frontend_fp16.tflite on the GPU LIGHT->LIGHT. Headroom 0.65-0.83, where 1.0 is the throttling threshold.

The NPU rows here ran artifacts compiled ahead of time for SM8850 with QAIRT 2.47.0; the GPU rows ran the published files as they are. LiteRT can also compile for the NPU on the device at first load, which is what lets you ship the published file unchanged — that path and the ten runtime libraries it needs are in the NPU recipe, and we did not measure it here. GPU wiring is in the GPU recipe.

Raspberry Pi 5 (CPU)

Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with the LiteRT `benchmark_model` tool from litert-cli-nightly 0.2.0.dev20260805: CPU inference (XNNPACK, 4 threads), 3 invocations per file of 10 warm-up plus 50 timed runs (the tool caps a phase at 150 s, so very slow graphs run fewer — the Runs column is the actual timed total). The latency is the median across invocations; the spread is the min–max over all timed runs. No thermal throttling occurred during these runs (vcgencmd get_throttled stayed 0x0).

Sources & license

• —Model: facebook/wav2vec2-base-960h — Apache-2.0.
• —Paper: wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations.
• —Hero audio: J.F. Kennedy inaugural address (1961), U.S. National Archives — public domain.