CoolFace
Modelpublic

npario/Qwen3.8-27B-OBLITERATED-OptiQ-4bit

sourceHugging Faceapache-2.0updated 22d agoView on Hugging Face
0likes271downloads
Model Card

mlx-community/Qwen3.8-27B-OBLITERATED-OptiQ-4bit

Built with [mlx-optiq](https://mlx-optiq.com), the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. All OptiQ quants · Docs

OptiQ mixed-precision quant of OBLITERATUS/Qwen3.8-27B-OBLITERATED, a Qwen3.8-family vision-language model with a bundled MTP speculation head. 21 GB on disk.

What it is

PropertyValue
BaseOBLITERATUS/Qwen3.8-27B-OBLITERATED (Qwen3.8, 27B)
MethodOptiQ mixed-precision, per-layer 4/8-bit
Bit allocationReused from the Qwen3.8-27B OptiQ recipe: the architecture is identical, so the per-layer sensitivity ranking transfers directly and no per-model sweep is needed
Layer split237 components at 4-bit, 261 at 8-bit
Group size64
On disk21 GB
MTPSpeculation head preserved in optiq/mtp.safetensors
Visionbf16 vision tower kept in optiq/optiq_vision.safetensors for image input

Following the naming llama.cpp uses for its mixed quants, the "4bit" label denotes the family, not the weighted average.

Run it

Qwen3.8 and the MTP/vision sidecars register through OptiQ, so import optiq once before loading:

bash
pip install "mlx-optiq>=0.4.27"
python
import optiq  # registers the arch + MTP/vision sidecars
from mlx_lm import load, generate

model, tok = load("mlx-community/Qwen3.8-27B-OBLITERATED-OptiQ-4bit")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Explain mixed-precision quantization in two sentences."}],
    tokenize=False, add_generation_prompt=True,
)
print(generate(model, tok, prompt=prompt, max_tokens=400))

For image input plus an OpenAI- and Anthropic-compatible endpoint with mixed-precision KV cache:

bash
optiq serve --model mlx-community/Qwen3.8-27B-OBLITERATED-OptiQ-4bit

This is a reasoning model, so give it a generous token budget.

Links