Nairod785/parakeet-ultra-gguf

Parakeet Ultra 0.6B — GGUF for transcribe.cpp

GGUF quantizations of moondream/parakeet-ultra for transcribe.cpp.

Parakeet Ultra is Moondream's full-precision post-training of NVIDIA's parakeet-tdt-0.6b-v3: the same FastConformer encoder (24 layers, d_model 1024) and TDT transducer decoder, the same 25-language 8192-piece vocabulary, with better accuracy than v3 across English, European languages, noisy audio and long-form speech (upstream figures below). It takes 16 kHz mono audio and returns a punctuated, cased transcript with optional token/word timestamps. It is not a streaming model and does not translate.

• —Parent model: moondream/parakeet-ultra @ 73175eb7aeb0d82f1e2a6b53b3aabc10a90bcd0b
• —Architecture: FastConformer-TDT, 627 M parameters (parakeet in transcribe.cpp)
• —Languages (25): bg, hr, cs, da, nl, en, et, fi, fr, de, el, hu, it, lv, lt, mt, pl, pt, ro, ru, sk, sl, es, sv, uk — automatic language detection
• —License: CC-BY-4.0 (inherited from the parent model; see Attribution)
• —How these files were made and how to reproduce them: QUANTIZATION.md

Files

Every file is at most the size of the parent checkpoint (model.safetensors, 1.26 GB). Pick one; they are alternatives, not parts.

The F16 file is 516 KB larger than the parent's model.safetensors because the GGUF also embeds the tokenizer (upstream ships it separately as tokenizer.json); it is smaller than the parent's weights + tokenizer together. SHA-256 sums are listed in QUANTIZATION.md.

Usage

# build transcribe.cpp (CPU; add -DTRANSCRIBE_CUDA=ON / -DTRANSCRIBE_VULKAN=ON for GPUs)
cmake -B build && cmake --build build --config Release --target transcribe-cli

# transcribe (language is auto-detected; -l de etc. pins it)
build/bin/transcribe-cli -m parakeet-ultra-0.6b-Q8_0.gguf audio.wav
build/bin/transcribe-cli -m parakeet-ultra-0.6b-Q8_0.gguf -l de --timestamps word audio.wav

The model runs on the stock transcribe.cpp parakeet runtime (CPU, CUDA, Vulkan, Metal, Android/ARM64); no special build flags are needed. The GGUF is structurally identical to NVIDIA's v3 GGUF (same 697 tensors, tokenizer and hyper-parameters), so any application that already runs parakeet-tdt-0.6b-v3 through transcribe.cpp can load it.

Validation

• —Numerical parity against the reference implementation (Hugging Face transformers ParakeetForTDT, fp32, eager attention) on samples/jfk.wav: 18/18 dumped tensors within tolerance (encoder output max |Δ| 3.0e-4, joint log-probs max |Δ| 1.1e-2), transcript identical.
• —Every file (F16 → Q4KM) produces the reference transcript on English (JFK) and German test clips; batch decoding (batch 2/4/8) is byte-identical to serial decoding.
• —Accuracy (measured here):

FLEURS French test split, all 676 utterances, greedy decoding, no LM, CUDA backend, batch 1, language hint fr; bootstrap 95 % confidence intervals. F16 → Q5KM are indistinguishable; Q4KM costs ≈ 0.3 pp.

Upstream-reported accuracy (Moondream's own evaluation, not re-measured here): Open ASR Leaderboard 7-set average 5.80 % WER (v3: 6.26 %); FLEURS 25-language average 9.55 % (v3: 11.62 %); MUSAN noise 5.82 % (v3: 6.72 %); TED-LIUM long-form 1.94 % (v3: 2.71 %).

Speed

Measured with transcribe-bench on a 29.3 s clip (samples/german.wav), warm, mean of 3 iterations, RTX 4070 Laptop GPU (8 GB) and its laptop x86 CPU (AVX2), transcribe.cpp after the 2026-09-26 optimization round (see docs/porting/parakeet-optimization-2026-09-26.md in the repo). "×" = times faster than realtime.

On CPU, K-quants and Q40 use ggml's repacked GEMM kernels (x86 AVX2: Q4K; ARM dotprod/i8mm: also Q5K, Q6K, Q80), which is why Q4KM is the fastest CPU file here. Accuracy of the optimized paths was re-measured (FLEURS-fr: Q80 on CUDA 4.63 %, Q4KM on CPU 4.99 %) and is unchanged.

Differences from the parent checkpoint

• —The 6 vad_head.* tensors (≈213 K parameters) are omitted. They belong to Moondream's Photon runtime's voice-activity head and are not part of speech recognition.
• —Decoding is greedy TDT with at most 10 symbols per frame, exactly as in NeMo / transformers. No external language model.

Attribution

Model weights: © Moondream, released under CC-BY-4.0 as moondream/parakeet-ultra, a post-training of NVIDIA's parakeet-tdt-0.6b-v3 (CC-BY-4.0). This repository changes only the file format (GGUF) and the numeric precision of the weights as described in QUANTIZATION.md.

Correction (2026-09-26): WER figures published earlier were ~1.6–1.8 pp too high. 61 of the FLEURS-fr reference transcripts in the manifest had been stored mojibake-encoded (é → é, Windows cp1252 default in the manifest builder), so correct hypotheses scored as errors. The references were repaired and every report re-scored; the numbers here are the corrected ones. The scripts now always pass an explicit encoding (scripts/ci/check_text_encoding.py).