mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit-REAP-14B

mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit-REAP-14B

Built with [mlx-optiq](https://mlx-optiq.com), the MLX-native toolkit to quantize, prune, fine-tune, and serve LLMs locally on Apple Silicon. All OptiQ models · Docs

10.4 GB instead of 17.5 GB. 9.4 GB of memory to run instead of 16.4 GB. Capability Score 68.13 against the parent's 75.75.

50% of the routed experts are removed from mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit; active parameters per token are unchanged, since top-8 routing is preserved and only the stored expert bank shrinks. That is why it gets smaller without getting slower.

Retained experts are copied bit-for-bit from the parent quant. Nothing is dequantized, re-quantized, merged, or retrained.

What pruning costs

Gemma-4 gives up more than the Qwen models do at the same 50 % retention. It routes over 128 experts where they route over 256, and the divergence from the unpruned parent measures KL 0.56 against their 0.07–0.13. The cost shows up in MMLU first, as it does everywhere, but here HumanEval moves too — on the Qwens procedural ability is essentially untouched.

If that trade is wrong for your workload, the model is not the only option: optiq prune-experts <parent> --retain 96 keeps three quarters of the bank and measures KL 0.33, at a correspondingly smaller saving.

Details

Use it

pip install mlx-optiq
optiq serve --model mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit-REAP-14B

from mlx_lm import load, generate
model, tok = load("mlx-community/gemma-4-26B-A4B-it-OptiQ-4bit-REAP-14B")
print(generate(model, tok, prompt="Hello", max_tokens=64))

Method

Expert pruning follows REAP (Cerebras Research, ICLR 2026 — *REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression*). Experts are ranked by the conditional mean of router weight × expert output norm over calibration data; the lowest-ranked are removed and the router is sliced to match.

OptiQ applies it in the quantized domain — directly on a quantized checkpoint, with no BF16 parent and no dequantization of survivors — via optiq prune-experts. See the pruning docs.