cstr/parakeet-tdt-0.6b-v3-GGUF

Parakeet TDT 0.6B v3 — GGUF (ggml-quantised)

GGUF / ggml conversions of `nvidia/parakeet-tdt-0.6b-v3` for use with the parakeet-main CLI from [CrispStrobe/CrispASR](https://github.com/CrispStrobe/CrispASR/).

Parakeet TDT 0.6B v3 is NVIDIA's 600 M-parameter multilingual ASR model:

• —25 European languages with automatic language detection (no prompt prefix needed)
• —Built-in word-level timestamps from the TDT (Token-and-Duration Transducer) decoder — no separate CTC alignment model required
• —6.34 % avg WER on the HuggingFace Open ASR Leaderboard
• —CC-BY-4.0 licence (friendlier than most ASR models)

This repo provides four quantisations, all converted from the same .nemo checkpoint via the streaming convert-parakeet-to-gguf.py script and quantised with cohere-quantize.

Files

All quantisations produce identical text on samples/jfk.wav:

And so my fellow Americans. Ask not what your country can do for you. Ask what you can do for your country.

Quick Start

# 1. Build the runtime
git clone -b parakeet https://github.com/CrispStrobe/CrispASR
cd CrispASR
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc) --target parakeet-main

# 2. Download a quantisation
huggingface-cli download cstr/parakeet-tdt-0.6b-v3-GGUF \
    parakeet-tdt-0.6b-v3-q4_k.gguf --local-dir .

# 3. Transcribe
./build/bin/parakeet-main \
    -m parakeet-tdt-0.6b-v3-q4_k.gguf \
    -f your-audio.wav -t 8

Word-level timestamps for free

Pass -v to dump per-token timestamps from the TDT duration head. Each token spans one or more encoder frames; one frame = 80 ms. No separate alignment model is required.

$ ./build/bin/parakeet-main -m parakeet-tdt-0.6b-v3-q4_k.gguf -f samples/jfk.wav -t 8 -v
  [    0.32s →     0.64s]  ' And'
  [    0.64s →     0.88s]  ' so'
  [    1.04s →     1.28s]  ' my'
  [    1.28s →     1.76s]  ' fellow'        ← f + ell + ow grouped
  [    1.76s →     2.56s]  ' Americans'
  [    2.96s →     3.28s]  '.'
  [    3.28s →     3.84s]  ' Ask'
  [    4.08s →     4.40s]  ' not'
  [    5.28s →     5.92s]  ' what your'
  ...

This is roughly 10× tighter than the cross-attention DTW path used for Cohere Transcribe word timestamps (~360 ms MAE), and comparable to running a separate wav2vec2 + CTC forced alignment model — but at zero extra cost.

Model architecture

The mel filterbank and Hann window are baked directly into the GGUF (preprocessor.fb and preprocessor.window from the original .nemo checkpoint), so there is no recomputation at runtime. BatchNorm in the convolution module is folded into the depthwise conv weights at load time.

How this was made

1. 1.The .nemo checkpoint was unpacked, NeMo state-dict keys were remapped to ggml-friendly names, and weights were written to GGUF F16 (matmul tensors) + F32 (norms / biases / mel filterbank). A synthetic zero conv.dw.bias is added per encoder layer so the runtime BN-fold pass has somewhere to write the absorbed bias shift.
2. 2.Quantised variants are produced by cohere-quantize (the same llama.cpp-style quantiser used for the Cohere Transcribe GGUFs).
3. 3.Inference is implemented in src/parakeet.{h,cpp}: the FastConformer encoder runs as a single ggml graph (BN folded out), the LSTM predictor and joint head run as manual F32 CPU loops, and the TDT greedy decode loop alternates "advance encoder frame" / "emit token + advance predictor" using the duration head's argmax.

Supported languages

bg cs da de el en es et fi fr hr hu it lt lv mt nl pl pt ro ru sk sl sv uk

The model auto-detects the language at inference time. No prompt prefix or -l flag is needed.

Attribution

• —Original model: `nvidia/parakeet-tdt-0.6b-v3` (CC-BY-4.0). NVIDIA NeMo team.
• —GGUF conversion + ggml runtime: `CrispStrobe/CrispASR@parakeet` — community contribution. Encoder graph borrows the dw_striding subsampling + Conformer block patterns from the same fork's cohere.cpp.
• —Reference inference: `istupakov/onnx-asr` was the cross-check for the joint head + TDT greedy loop.

Related

• —C++ runtime: [CrispStrobe/CrispASR@parakeet](https://github.com/CrispStrobe/CrispASR/tree/parakeet)
• —Sister repo (Cohere Transcribe): `cstr/cohere-transcribe-03-2026-GGUF`

License

CC-BY-4.0, inherited from the base model. Use of these GGUF files must comply with the CC-BY-4.0 license including attribution.

Provenance and EU AI Act Art. 53 note

• —Upstream model: nvidia/parakeet-tdt-0.6b-v3 — published by nvidia.
• —Upstream licence: cc-by-4.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• —What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• —Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• —Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.