fredchu/breeze-asr-26-mlx-4bit

Breeze-ASR-26 — MLX (4-bit palette quantized)

Apple MLX port of `MediaTek-Research/Breeze-ASR-26`, the Taiwanese Hokkien (Taigi) ASR model from MediaTek Research's MR Breeze 3 series, 4-bit palette quantized (group size 64) for compact on-device inference.

Runs on Apple Silicon Macs (M1/M2/M3/M4) via the `mlx-whisper` package — same API as mlx-community/whisper-* checkpoints.

Files

72% smaller than the fp16 variant (3.1 GB).

Usage

import mlx_whisper

result = mlx_whisper.transcribe(
    "audio.wav",
    path_or_hf_repo="fredchu/breeze-asr-26-mlx-4bit",
    language="zh",
)
print(result["text"])

CLI:

mlx_whisper audio.wav --model fredchu/breeze-asr-26-mlx-4bit --language zh

Performance (M1 Max, mlx-whisper)

Surprising Quality Result

In our 2026-04-30 head-to-head against the fp16 variant on a real Taigi sample (a Mandarin-speaking creator using the Taigi word 漂泊 pio-pôa), the 4-bit variant transcribed `漂泊` correctly while fp16 produced `瀟灑` instead. Both runs identical otherwise (same audio, same mlx-whisper version, language=zh).

This is counterintuitive — quantization usually degrades quality. One possible explanation: 4-bit palette quantization (mapping weights to 16 representative values per group) may re-calibrate outlier weights in a way that better generalizes to underrepresented Taigi tokens. Reproducible on M1 Max; worth verifying on your own samples.

For Taigi-heavy use cases, try this 4-bit variant first. For pure Mandarin or read-speech benchmarks, the fp16 variant remains the safer default.

Limitations (inherited from base model)

• —Outputs Mandarin Chinese characters, not Taigi orthography (台語正字 / 台羅)
• —Trained on ~10,000 hours of synthetic Taigi speech — distribution gap with real spontaneous speech
• —English brand/proper nouns are aggressively transliterated: in our Mandarin test, Hello became 哈囉, Austin became Alstin, Netflix became Nathalie 的時事. ASR-25 (MediaTek-Research/Breeze-ASR-25) handles these correctly. Do not use this model for content with frequent English code-switching.
• —All segments come back as one ~30-second block regardless of audio content (model training behaviour, not framework setting). Post-process if you need finer subtitle granularity.

Quantization Details

Performed via mlx.nn.quantize after weight conversion from HuggingFace transformers safetensors → MLX Whisper format.

Conversion

Built with a custom wrapper around `mlx-examples/whisper/convert.py` that adds sharded-safetensors loader support (the source repo ships weights as 5 GB + 1 GB shards, which the upstream converter doesn't handle).

License

Apache 2.0 — inherits from the base model.

Acknowledgments

• —MediaTek Research for the original Breeze-ASR-26 weights
• —Apple MLX team for the framework, quantization API, and conversion tooling