CoolFace
Modelpublic

mlx-community/granite-speech-4.1-2b-nar-mlx-5bit

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes21downloads
Model Card

Granite Speech 4.1 2B NAR — MLX

MLX port of `ibm-granite/granite-speech-4.1-2b-nar` for Apple Silicon. Runs via mlx-audio.

Architecture

Non-autoregressive ASR via CTC + bidirectional LM editing:

  1. 1.16-layer Conformer encoder (543M params) produces an initial BPE CTC hypothesis.
  2. 2.2-layer windowed Q-Former projector (80M params) converts multi-layer encoder states into audio embeddings.
  3. 3.40-layer bidirectional Granite editor (1.6B params) takes [audio | hypothesis_tokens] and emits edited logits in a single forward pass — no autoregression, no KV cache.
  4. 4.Final CTC collapse on text-position logits yields the transcript.

Total: ~2.25B params, bf16.

Quickstart

python
from pathlib import Path
from mlx_audio.stt.utils import load_model

model = load_model(Path("mlx-community/granite-speech-4.1-2b-nar-mlx"))
out = model.generate("audio.wav")
print(out.text)

Limitations

  • —Batch size 1.
  • —bf16 baseline only — no quantized variants yet.
  • —No streaming inference.
  • —macOS 14+, Apple Silicon.

Reference

Upstream model card: https://huggingface.co/ibm-granite/granite-speech-4.1-2b-nar

Validated against the upstream PyTorch reference: exact 44-token match and exact transcript string on the example wav.

License

Apache-2.0, matching the upstream model.