cstr/parakeet-tdt-0.6b-ja-GGUF

Parakeet TDT-CTC 0.6B (Japanese) — GGUF

GGUF / ggml conversions of `nvidia/parakeet-tdt_ctc-0.6b-ja` for use with the crispasr CLI from [CrispStrobe/CrispASR](https://github.com/CrispStrobe/CrispASR).

A 600 M-parameter Japanese ASR model with punctuation:

• —Hybrid FastConformer-TDT-CTC: TDT (Token-and-Duration Transducer) decoder by default; the CTC head is included in these GGUFs and selectable at runtime with --parakeet-decoder ctc.
• —Built-in word-level timestamps from the TDT duration head — no separate CTC alignment.
• —6.4 % CER on JSUT basic5000.
• —CC-BY-4.0 licence.

Files

All files include both the TDT decoder and the CTC head (added 2026-07; earlier uploads lacked the CTC tensors, so --parakeet-decoder ctc silently fell back to TDT).

Recommended: F16

Verified on a JSUT-basic5000 sample at F16:

NeMo (PyTorch): '水をマレーシアから買わなくてはならないのです。'
crispasr (F16): '水をマレーシアから買わなくてはならないのです。'

The F16 GGUF produces an identical transcript to the official NeMo Python pipeline.

About the Q4_K variant

The Japanese model uses an 80-mel preprocessor (vs. 128 for the multilingual v3) and a smaller, more sensitive encoder distribution. With our default Q4K quantisation, two of the most logit-shaping tensors (`joint.pred.weight`, `decoder.embed.weight`) fall back to `q40 because their dimensions don't tile cleanly for q4_k blocks. This is enough quantisation noise that the TDT decoder enters a fixed-point loop after the first ~8 tokens. Two clean options: prefer the **Q8_0** (TDT output identical to F16 in our tests), or keep the Q4_K and decode with the **CTC head** (--parakeet-decoder ctc`) — CTC has no autoregressive feedback, so the quantisation noise doesn't compound, and its output matches the F16 CTC transcript.

Quick start

# 1. Build the runtime
git clone https://github.com/CrispStrobe/CrispASR
cd CrispASR
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc) --target crispasr-lib

# 2. Download the F16
huggingface-cli download cstr/parakeet-tdt-0.6b-ja-GGUF \
    parakeet-tdt-0.6b-ja.gguf --local-dir .

# 3. Transcribe a 16 kHz mono WAV
./build/bin/crispasr --backend parakeet \
    -m parakeet-tdt-0.6b-ja.gguf -f your-japanese-audio.wav -t 8

You can also let crispasr auto-download the model:

./build/bin/crispasr --backend parakeet -m auto --auto-download \
    -f your-japanese-audio.wav --model-name parakeet-ja

Long-form audio (v0.8.8+)

The encoder is numerically fragile past ~12 s of context on real speech — single-pass decoding of long audio silently drops content (upstream NeMo behaves the same on the same clips: its plain, local-attention, and buffered long-form modes score 1–51 % content recall on our reference clip). CrispASR ≥ 3a8141e3 handles this automatically: audio > 30 s is VAD-segmented, slices are capped at 12 s (split at energy minima), each slice decodes in one NeMo-exact pass, and a gap-fill second pass re-transcribes any span the first pass left empty. Measured on the issue #89 reporter's clips (phonetic char-bigram recall vs whisper-large-v3-turbo): 97.2 % (60 s), 96.9 % (120 s), 95.9 % (300 s) — at the inter-model agreement ceiling (an independent SenseVoice-small run scores the same recall on the same audio). No flags needed; --vad, --chunk-seconds N, and CRISPASR_PARAKEET_* env vars override the defaults (see the CrispASR CLI docs).

Word-level timestamps for free

Pass -v to dump per-token timestamps from the TDT duration head. Each token spans one or more encoder frames; one frame = 80 ms. No separate alignment model required.

Model architecture

xscaling=True is the most important architectural detail vs. the multilingual v3 model — nvidia/parakeet-tdt-0.6b-v3 uses xscaling=False. Both settings are stored in the GGUF metadata (parakeet.xscaling) and read by the runtime, so the same code path serves both variants without per-model branches.

How this was made

1. 1.The .nemo checkpoint is unpacked; every architecture hyperparameter (dmodel, nlayers, ffdim, predhidden, jointhidden, xscaling, …) is read from `modelconfig.yaml and cross-checked against the actual tensor shapes. The mel filterbank and Hann window are baked directly into the GGUF (preprocessor.fb, preprocessor.window`).
2. 2.NeMo state-dict keys are remapped to ggml-friendly names. Weights are written as F16 for matmul tensors and F32 for norms / biases / mel filterbank. A synthetic zero conv.dw.bias is added per encoder layer when the checkpoint omits it (older NeMo BN-only convs).
3. 3.Inference is implemented in src/parakeet.{h,cpp}: the FastConformer encoder runs as a single ggml graph (BN folded into the depthwise conv weights at load time, xscaling applied between the pre-encode and the first block when parakeet.xscaling=true), the LSTM predictor and joint head run as manual F32 CPU loops, and the TDT greedy decode loop alternates "advance encoder frame" / "emit token + advance predictor" using the duration head's argmax.

Attribution

• —Original model: `nvidia/parakeet-tdt_ctc-0.6b-ja` (CC-BY-4.0). NVIDIA NeMo team.
• —GGUF conversion + ggml runtime: `CrispStrobe/CrispASR`.

Related

• —Multilingual sibling (25 EU languages): `cstr/parakeet-tdt-0.6b-v3-GGUF`
• —C++ runtime: `CrispStrobe/CrispASR`

License

CC-BY-4.0, inherited from the base model. Use of these GGUF files must comply with the CC-BY-4.0 license including attribution.

Provenance and EU AI Act Art. 53 note

• —Upstream model: nvidia/parakeet-tdt_ctc-0.6b-ja — published by nvidia.
• —Upstream licence: cc-by-4.0. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• —What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• —Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• —Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.