larryliu0820/parakeet-tdt-0.6b-v3-executorch

Parakeet TDT (0.6B v3) — ExecuTorch Exports

This repository contains ExecuTorch .pte exports of the Hugging Face ASR model `nvidia/parakeet-tdt-0.6b-v3` for:

• —XNNPACK (CPU): fp32 and INT4 weights (8da4w)
• —Metal (macOS): bf16

Each export directory contains:

• —model.pte — ExecuTorch program
• —tokenizer.model — SentencePiece tokenizer (needed by the runner)

Artifacts

Tokenizers:

• —xnnpack/fp32/tokenizer.model
• —xnnpack/int4/tokenizer.model
• —metal/bf16/tokenizer.model

How to run (C++ runner)

Build the Parakeet runner from an ExecuTorch checkout:

./install_executorch.sh
make parakeet-cpu
make parakeet-metal  # macOS only

Then run from the ExecuTorch repo root:

CPU / XNNPACK

./cmake-out/examples/models/parakeet/parakeet_runner \
  --model_path /path/to/model.pte \
  --audio_path /path/to/audio.wav \
  --tokenizer_path /path/to/tokenizer.model

Metal (macOS)

DYLD_LIBRARY_PATH=/usr/lib ./cmake-out/examples/models/parakeet/parakeet_runner \
  --model_path /path/to/model.pte \
  --audio_path /path/to/audio.wav \
  --tokenizer_path /path/to/tokenizer.model

Export recipe (ExecuTorch)

Exports were produced with examples/models/parakeet/export_parakeet_tdt.py from ExecuTorch commit efe4f0cce36c4b95ac70eac969c0d55163f23c78 using:

• —PyTorch 2.11.0.dev20251222
• —TORCHAO_FORCE_SKIP_LOADING_SO_FILES=1 (export-only workaround for torchao C++ extension loading)

Commands:

# XNNPACK fp32
TORCHAO_FORCE_SKIP_LOADING_SO_FILES=1 python examples/models/parakeet/export_parakeet_tdt.py \
  --backend xnnpack \
  --output-dir ./parakeet_xnnpack_fp32

# XNNPACK int4 (8da4w)
TORCHAO_FORCE_SKIP_LOADING_SO_FILES=1 python examples/models/parakeet/export_parakeet_tdt.py \
  --backend xnnpack \
  --qlinear_encoder 8da4w --qlinear_encoder_group_size 32 \
  --qlinear 8da4w --qlinear_group_size 32 \
  --output-dir ./parakeet_xnnpack_int4

# Metal bf16
TORCHAO_FORCE_SKIP_LOADING_SO_FILES=1 python examples/models/parakeet/export_parakeet_tdt.py \
  --backend metal --dtype bf16 \
  --output-dir ./parakeet_metal_bf16

Verification

All variants were verified to transcribe the same test audio correctly (and match eager PyTorch output for the XNNPACK exports).

• —Audio sample: hf-internal-testing/librispeech_asr_dummy (clean, validation[0])
• —Reference transcript (dataset):
• —MISTER QUILTER IS THE APOSTLE OF THE MIDDLE CLASSES AND WE ARE GLAD TO WELCOME HIS GOSPEL
• —Model output:
• —mister Quilter is the apostle of the middle classes, and we are glad to welcome his gospel.

License / Attribution

• —Upstream model license: CC-BY-4.0 (see nvidia/parakeet-tdt-0.6b-v3).
• —Please attribute NVIDIA when using these exported artifacts.