Modelpublic

underlotus/Qwen3.6-27B-Abliterated-Heretic-Uncensored-oQ4

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes146downloads

Model Card

Qwen3.6-27B Abliterated Heretic Uncensored — oQ4

Mixed-precision quant of Youssofal/Qwen3.6-27B-Abliterated-Heretic-Uncensored-BF16, produced with oQ (oMLX v0.5.4). Vision tower intact — image + video input still works. Standard MLX safetensors — compatible with oMLX, mlx-lm, LM Studio, and any MLX-capable app.

What is oQ?

Unlike uniform 4-bit quantization, oQ is a data-driven mixed-precision quantizer that calibrates per-layer sensitivity and allocates bits where they matter most. Critical layers (embeddings, LM head, the most sensitive transformer layers) are automatically promoted to 8-bit, while less sensitive layers stay at 4-bit. Typical result: ~4.6 bits-per-weight.

Benchmarked on Qwen3.5-35B-A3B (oMLX project):

Benchmark	mlx-lm 4-bit	oQ4
MMLU (300)	79.7%	83.3%
TruthfulQA (300)	87.7%	88.0%
HumanEval (full)	87.2%	85.4%
MBPP (300)	71.7%	74.3%

Performance (oMLX on M4 10-core)

Context	PP tok/s	TG tok/s	Peak Mem
1k	63.6	6.5	16.4 GB
4k	59.9	6.3	17.8 GB

Batch	TG tok/s	Speedup
1×	6.5	1.00×
2×	11.6	1.78×
4×	24.4	3.75×

Full benchmark →

Why this quant

The original BF16 weights require ~55 GB. This oQ4 quant runs in ~16–18 GB on Apple Silicon.

Quick start

bash

# oMLX
omlx serve --model underlotus/Qwen3.6-27B-Abliterated-Heretic-Uncensored-oQ4

python

# mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("underlotus/Qwen3.6-27B-Abliterated-Heretic-Uncensored-oQ4")
response = generate(model, tokenizer, prompt="Hello!", max_tokens=256)
print(response)

Original model

—Base: Qwen/Qwen3.6-27B
—Abliterated: Heretic-style two-stage MPOA pipeline, magnitude-preserving refusal attenuation (KL 0.0282)
—Vision: Full multimodal — image + video input supported

License

Apache 2.0, inherited from base model.