CoolFace
Modelpublic

mlx-community/Macaw-OptiQ-4bit

sourceHugging Faceotherupdated 11d agoView on Hugging Face
2likes204downloads
Model Card

mlx-community/Macaw-OptiQ-4bit

Built with [mlx-optiq](https://mlx-optiq.com), the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. All OptiQ quants · Docs · LFM2.5 family

An OptiQ mixed-precision quant of badtheorylabs/Macaw, an on-device Mac assistant built on LFM2.5-2.6B. 1.93 GB on disk, down from 5.2 GB at bf16.

Macaw is a tool-calling agent, so the quant is aimed at keeping tool calls well formed rather than at raw benchmark scores.

What it is

PropertyValue
Basebadtheorylabs/Macaw (LFM2.5-2.6B derivative)
Architecturelfm2 — hybrid conv + full attention, 30 layers
MethodOptiQ mixed-precision, per-layer bit allocation reused from the base family
On disk1.93 GB (bf16: 5.2 GB)
Context128k

Macaw keeps its base architecture, so which layers tolerate fewer bits is unchanged. The per-layer allocation comes from LFM2.5-2.6B-OptiQ-4bit rather than a fresh sensitivity sweep: 167 of 167 layers matched, 80 kept at 8-bit and 87 at 4-bit.

Run it

bash
pip install mlx-optiq
optiq serve --model mlx-community/Macaw-OptiQ-4bit

That gives you an OpenAI and Anthropic compatible endpoint with mixed-precision KV cache, tool-call healing and prompt caching. The base model's recommended sampling ships in generation_config.json and optiq serve applies it without any flags.

Links