MarkChen1214/parakeet-unified-en-0.6b-MLX-Mixed-3bit4bit6bit

Parakeet Unified English 0.6B — MLX Mixed 3-bit/4-bit/6-bit

A sensitivity-guided mixed-precision MLX conversion of `nvidia/parakeet-unified-en-0.6b`, built from the full-precision MLX conversion `littlebearlabs/parakeet-unified-en-0.6b-mlx`.

The checkpoint uses group-64 affine quantization: 200 eligible matrices at 3-bit, 64 at 4-bit, and nine sensitive frontend, predictor, and joint matrices at 6-bit. Normalization, depthwise convolution, bias, and other ineligible weights remain at their source precision.

At 0.352 GB of safetensor weights, it is 9.89% smaller than the 0.391 GB external uniform-INT4 checkpoint. On the full 6,206-utterance English gate it scores 2.78% macro WER, compared with 2.66% for its FP32 teacher.

Full three-collection evaluation

Measured locally on Apple Silicon with MLX and public parakeet-mlx RNNT inference. The gate uses the complete LibriSpeech test-clean (2,620), complete LibriSpeech test-other (2,939), and complete FLEURS en_us test (647) splits. Every artifact scored all 6,206 utterances, totaling 12.51 audio hours, with zero skipped references. Macro WER gives each collection equal weight.

The useful result is the size/quality point: 85.76% less weight storage than full precision for a 0.12-point macro-WER increase. Among the two compressed releases, Unified is the cautious quality choice by 0.06 point; TDT is the throughput choice, measuring 51.3% faster and 1.04% smaller in this run.

Secondary meeting and financial-speech panel

An earlier seed-2026 comparison sampled up to 100 examples from LibriSpeech clean, LibriSpeech other, AMI, and Earnings-22. AMI contributed 87 valid rows, for 387 matched utterances. It remains useful domain evidence, but is not the headline quality gate.

The experiment also rejected a uniform 3-bit checkpoint. Although it reached 0.314 GB, its matched fast-screen macro WER increased from 7.09% for the dense teacher to 16.90%. Sensitivity scans identified the RNN-T joint as the largest isolated hotspot; simply lowering every eligible matrix to one bit width was not a viable compression strategy.

Provenance and reproducibility

The NVIDIA source .nemo is pinned at revision fe53cd885760c96b6a5f51a0bfd362cb4584a98b. Its model config and tokenizer are byte-identical to the dense MLX teacher. All 989 tensor keys match: 965 tensors are bit-identical, while the remaining 24 are zero-valued num_batches_tracked counters represented as FP32 rather than INT64.

The dense MLX source is pinned at revision 9e77fe17aaad040ae27d6eb8fb27e58febe4eb34. The final checkpoint was rebuilt from that revision in a clean output directory and reproduced the evaluated weights byte-for-byte:

model.safetensors SHA-256
c0843b8d113c7ba73dcf67472c65be3ed5bcde589ca39f90daefc941a65abbf7

The exact per-module allocation is included as quantization_recipe.json. The compatibility loader is included as parakeet_unified_mlx.py; conversion code, rejected profiles, and benchmark evidence are maintained in `yuting1214/model-lab`.

Usage

Install the public MLX runtime and the loader dependencies:

pip install parakeet-mlx huggingface_hub pyyaml librosa soundfile

from huggingface_hub import snapshot_download
from parakeet_unified_mlx import load_parakeet_unified

model_dir = snapshot_download(
    "MarkChen1214/parakeet-unified-en-0.6b-MLX-Mixed-3bit4bit6bit"
)
model = load_parakeet_unified(model_dir)
result = model.generate("speech.wav")
print(result.text)

Run the script from the downloaded repository directory, or place parakeet_unified_mlx.py on PYTHONPATH before importing it.

Scope and limitations

• —The published quality measurements use offline, full-context RNNT inference.
• —The upstream architecture supports offline and buffered streaming modes, but the open runtime used here does not yet reproduce the Unified model's trained dynamic-chunk streaming path. No streaming-latency or streaming-WER claim is made for this checkpoint.
• —The external INT4/INT8 cards target a separate runtime. This comparison loads all checkpoints through the same public compatibility path instead.
• —Four LSTM matrices are stored at 6-bit but dequantized during load because the current public MLX LSTM implementation has no quantized LSTM kernel.
• —Accuracy varies with accents, domains, noise, and recording conditions. Evaluate on your own target distribution before deployment.

License

The original NVIDIA model is released under the NVIDIA Open Model License. This quantized derivative inherits those terms. Review the license for your intended use and retain the required attribution.