Modelpublic

amandabenson/parakeet-cpp-gguf-fork

sourceHugging Facecc-by-4.0updated 1mo agoView on Hugging Face

0likes562downloads

Model Card

Parakeet GGUF — models for parakeet.cpp

GGUF-format weights for parakeet.cpp, a C++/ggml port of NVIDIA NeMo Parakeet that matches the upstream PyTorch models on CPU. This single repo collects every supported model × quantization as a flat set of .gguf files — download just the one you need.

F16 is the recommended default — same accuracy as F32, ~1.7× smaller, and typically the fastest on modern CPUs via ggml's F32×F16 matmul fast path.

Models

tdt_ctc-110m

Source: nvidia/parakeet-tdt_ctc-110m · Hybrid TDT+CTC (FastConformer) · heads: TDT + CTC

File	Variant	Size	WER vs NeMo
`tdt_ctc-110m-f16.gguf` ← recommended	F16	267.5 MB	0.0000
`tdt_ctc-110m-q8_0.gguf`	Q8_0	177.8 MB	0.0000
`tdt_ctc-110m-q6_k.gguf`	Q6_K	155.9 MB	not measured
`tdt_ctc-110m-q5_k.gguf`	Q5_K	143.3 MB	not measured
`tdt_ctc-110m-q4_k.gguf`	Q4_K	131.4 MB	0.0000

realtimeeou120m-v1

Source: nvidia/parakeet_realtime_eou_120m-v1 · Cache-aware streaming RNNT (FastConformer, EOU/EOB) · heads: RNNT (streaming)

File	Variant	Size	WER vs NeMo
`realtime_eou_120m-v1-f16.gguf` ← recommended	F16	266.5 MB	not measured
`realtime_eou_120m-v1-q8_0.gguf`	Q8_0	176.0 MB	not measured
`realtime_eou_120m-v1-q6_k.gguf`	Q6_K	153.9 MB	not measured
`realtime_eou_120m-v1-q5_k.gguf`	Q5_K	141.2 MB	not measured
`realtime_eou_120m-v1-q4_k.gguf`	Q4_K	129.1 MB	not measured

ctc-0.6b

Source: nvidia/parakeet-ctc-0.6b · CTC (FastConformer) · heads: CTC

File	Variant	Size	WER vs NeMo
`ctc-0.6b-f16.gguf` ← recommended	F16	1373.4 MB	0.0000
`ctc-0.6b-q8_0.gguf`	Q8_0	875.4 MB	0.0000
`ctc-0.6b-q6_k.gguf`	Q6_K	746.8 MB	not measured
`ctc-0.6b-q5_k.gguf`	Q5_K	676.3 MB	not measured
`ctc-0.6b-q4_k.gguf`	Q4_K	609.9 MB	not measured

rnnt-0.6b

Source: nvidia/parakeet-rnnt-0.6b · RNNT transducer (FastConformer) · heads: RNNT

File	Variant	Size	WER vs NeMo
`rnnt-0.6b-f16.gguf` ← recommended	F16	1402.8 MB	0.0000
`rnnt-0.6b-q8_0.gguf`	Q8_0	903.9 MB	0.0000
`rnnt-0.6b-q6_k.gguf`	Q6_K	776.3 MB	not measured
`rnnt-0.6b-q5_k.gguf`	Q5_K	705.7 MB	not measured
`rnnt-0.6b-q4_k.gguf`	Q4_K	639.2 MB	not measured

tdt-0.6b-v2

Source: nvidia/parakeet-tdt-0.6b-v2 · TDT transducer (FastConformer) · heads: TDT

File	Variant	Size	WER vs NeMo
`tdt-0.6b-v2-f16.gguf` ← recommended	F16	1404.2 MB	0.0000
`tdt-0.6b-v2-q8_0.gguf`	Q8_0	903.8 MB	0.0000
`tdt-0.6b-v2-q6_k.gguf`	Q6_K	775.9 MB	not measured
`tdt-0.6b-v2-q5_k.gguf`	Q5_K	705.0 MB	not measured
`tdt-0.6b-v2-q4_k.gguf`	Q4_K	638.4 MB	not measured

tdt-0.6b-v3

Source: nvidia/parakeet-tdt-0.6b-v3 · TDT transducer (FastConformer) · heads: TDT

File	Variant	Size	WER vs NeMo
`tdt-0.6b-v3-f16.gguf` ← recommended	F16	1441.0 MB	0.0000
`tdt-0.6b-v3-q8_0.gguf`	Q8_0	940.7 MB	0.0000
`tdt-0.6b-v3-q6_k.gguf`	Q6_K	812.7 MB	not measured
`tdt-0.6b-v3-q5_k.gguf`	Q5_K	741.9 MB	not measured
`tdt-0.6b-v3-q4_k.gguf`	Q4_K	675.2 MB	not measured

ctc-1.1b

Source: nvidia/parakeet-ctc-1.1b · CTC (FastConformer) · heads: CTC

File	Variant	Size	WER vs NeMo
`ctc-1.1b-f16.gguf` ← recommended	F16	2395.8 MB	0.0000
`ctc-1.1b-q8_0.gguf`	Q8_0	1526.3 MB	0.0000
`ctc-1.1b-q6_k.gguf`	Q6_K	1301.7 MB	not measured
`ctc-1.1b-q5_k.gguf`	Q5_K	1178.5 MB	not measured
`ctc-1.1b-q4_k.gguf`	Q4_K	1062.6 MB	not measured

rnnt-1.1b

Source: nvidia/parakeet-rnnt-1.1b · RNNT transducer (FastConformer) · heads: RNNT

File	Variant	Size	WER vs NeMo
`rnnt-1.1b-f16.gguf` ← recommended	F16	2425.2 MB	0.0000
`rnnt-1.1b-q8_0.gguf`	Q8_0	1554.7 MB	0.0000
`rnnt-1.1b-q6_k.gguf`	Q6_K	1331.2 MB	not measured
`rnnt-1.1b-q5_k.gguf`	Q5_K	1207.9 MB	not measured
`rnnt-1.1b-q4_k.gguf`	Q4_K	1091.9 MB	not measured

tdt-1.1b

Source: nvidia/parakeet-tdt-1.1b · TDT transducer (FastConformer) · heads: TDT

File	Variant	Size	WER vs NeMo
`tdt-1.1b-f16.gguf` ← recommended	F16	2425.3 MB	0.0000
`tdt-1.1b-q8_0.gguf`	Q8_0	1554.8 MB	0.0000
`tdt-1.1b-q6_k.gguf`	Q6_K	1331.2 MB	not measured
`tdt-1.1b-q5_k.gguf`	Q5_K	1207.9 MB	not measured
`tdt-1.1b-q4_k.gguf`	Q4_K	1091.9 MB	not measured

tdt_ctc-1.1b

Source: nvidia/parakeet-tdt_ctc-1.1b · Hybrid TDT+CTC (FastConformer) · heads: TDT + CTC

File	Variant	Size	WER vs NeMo
`tdt_ctc-1.1b-f16.gguf` ← recommended	F16	2429.5 MB	0.0000
`tdt_ctc-1.1b-q8_0.gguf`	Q8_0	1559.0 MB	0.0000
`tdt_ctc-1.1b-q6_k.gguf`	Q6_K	1335.4 MB	not measured
`tdt_ctc-1.1b-q5_k.gguf`	Q5_K	1212.1 MB	not measured
`tdt_ctc-1.1b-q4_k.gguf`	Q4_K	1096.1 MB	not measured

WER (word error rate) is computed against the upstream NeMo reference on tests/fixtures/speech.wav (LibriSpeech 2086-149220-0033, ~7.4 s, English). 0.0 = byte-for-byte identical transcript. See parity.md and quantization.md.

Quantization notes

Quantization is applied only to the large linear weights fed directly into ggml_mul_mat (encoder FFN + attention projections, subsampling output projection, joint enc/pred projections). All other tensors (mel filterbank, LSTM prediction net, conv kernels, batch_norm stats, norms, biases, embeddings) stay F32.

Usage

bash

# 1. Clone + build parakeet.cpp
git clone https://github.com/mudler/parakeet.cpp
cd parakeet.cpp
cmake -B build -DPARAKEET_BUILD_CLI=ON && cmake --build build -j

# 2. Download one quant (F16 recommended)
huggingface-cli download mudler/parakeet-cpp-gguf tdt_ctc-110m-f16.gguf --local-dir models/

# 3. Transcribe
build/examples/cli/parakeet-cli transcribe \
    --model models/tdt_ctc-110m-f16.gguf \
    --input audio.wav

License

The GGUF weights are derived from the NVIDIA NeMo Parakeet checkpoints, released under the CC-BY-4.0 license. The parakeet.cpp runtime is MIT-licensed.