lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4

<p align="center"> <img src="logo.png" alt="Lemura Labs" width="110"/> </p>

Qwen3.6-27B-v2.1-uncensored-mxfp4

Yes — VISION PRESERVED. Unlike most 4-bit quantization pipelines that degrade or strip the vision tower, this release keeps the full Qwen3.6-VL ViT and all projector weights in fp16. Use it as a real multimodal model at half the size of the 8-bit build.

MLX MXFP4 (OCP Microscaling FP4) quantization of a abliterated Qwen 3.6 27B v2 (the Jackrong Claude-Opus reasoning distill of Qwen 3.6 27B). Refusals reduced from 91/100 → 4/100 with KL drift of just 0.0176 (measured at BF16). Quantization adds only +0.87% perplexity with 92.4% greedy-token agreement vs the BF16 reference. By the Lemura Labs research team. Quantized with mlx-mtp — a vision-preserving MXFP4 quantizer.

Fastest absolute throughput of any Qwen3.6-27B MLX build — 27.7 tok/s vanilla on M4 Max. Apple Silicon is memory-bandwidth-bound: halving the model fits more weights in fast SRAM, which matters more than the precision drop.

TL;DR

All Qwen3.6-27B variants

The full Qwen3.6-27B family from Lemura Labs — same abliterated weights (refusal 4/100, KL 0.0176), different quant schemes for different runtimes.

All variants share the same abliterated base weights — pick by your runtime (Apple Silicon → MLX; CUDA/CPU/cross-platform → GGUF) and your RAM budget. MXFP4 is the fastest MLX build on M4 Max; use 8-bit if you want the MTP head for native speculative decoding without a drafter.

Lineage

Qwen/Qwen3.6-27B (Qwen Team — base multimodal pretrain)
 │
 ▼
Jackrong/Qwopus3.6-27B-v2 (Jackrong — Claude-Opus reasoning distill)
 │
 ▼
ablation abliteration (TPE-50) (Lemura Labs)
 ├── 25 random startup trials
 ├── 2 community priors (coder3101, wangzhang)
 └── 23 TPE smart-sampling trials → best at trial 45
 │
 ▼
lemuralabs/Qwen3.6-27B-V2-zerofuse (BF16 full-precision checkpoint)
 │
 ▼
this repo — MLX MXFP4 quant (/mlx-mtp · Lemura Labs)

Direct upstream links:

• —Foundation: Qwen/Qwen3.6-27B
• —Claude-Opus distill: Jackrong/Qwopus3.6-27B-v2
• —Abliteration tool: the ablation toolkit by Lemura Labs
• —Quantization tool: mlx-mtp — vision-preserving MXFP4 quantizer

Abliteration Results

the ablation toolkit measures refusals on mlabonne/harmful_behaviors (100 hard red-team prompts) and KL divergence on mlabonne/harmless_alpaca.

→ 96% reduction in refusals with capability preserved at BF16. MXFP4 quantization adds only +0.87% perplexity on top — the abliteration and quantization quality loss are both negligible.

Method

Abliteration (inherited from the BF16 source) — TPE-50 Optuna search on the ablation toolkit, M4 Max 128 GB. Full method in that card.

MXFP4 quantization (this repo):

1. 1.Source — loaded lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored (51 GB BF16, 3 shards) into MLX on M4 Max.
2. 2.Layer audit — mlx_mtp.mxfp4_quantize catalogued 1199 tensors: identified all LM linear projections (attention Q/K/V/O + MLP gate/up/down + lmhead) as MXFP4 candidates; flagged 196 vision-tower + projector tensors and 48 SSM-sensitive params (`alog, dt_bias, conv1d`) for fp16 preservation.
3. 3.Quantization — LM linears → MXFP4 (OCP MX E2M1, group_size=32) using mlx.core.quantize. Vision + SSM tensors written verbatim at fp16. 15 MTP head weights absent from source — quantizer set mtp_num_hidden_layers: 0 in output config to prevent strict-load failure.
4. 4.Output — 3 safetensor shards, 14 GB total. Embedded mlx_mtp config block records format, visionfp16=True, mtppreserved=False.
5. 5.Verification — text generation correct; vision captioning correct (image → caption matches BF16 output); DFlash block-diffusion speculative decoding loads and runs; KL divergence measured end-to-end (see Quality section below).

Total wall-clock: ~2 h on M4 Max 128 GB (dominated by disk I/O).

Quantization quality vs BF16

Measured via teacher-forced prefill — 10 diverse prompts (code, math, reasoning, instruction-following), 244 token positions, full 248K-vocab distributions at every position.

JSD < 0.01 is the standard "essentially identical distributions" threshold. At 0.009 we are below it. The 7.6% of positions where models disagree on the top-1 token are near-tie argmax flips from quantization noise — not quality regressions.

Use it

Fastest inference via mlx-mtp (recommended — native DFlash support)

git clone https://github.com/jundot/omlx && pip install -e ./omlx
git clone mlx-mtp && pip install -e ./mlx-mtp

from mlx_mtp.engine import load_model, vanilla_generate

model, processor, config = load_model("lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4")

result = vanilla_generate(
 model, processor, config,
 text="Write a Python function that implements merge sort:",
 max_tokens=512,
)
print(result["text"])
print(f"{result['tps']:.1f} tok/s")

With DFlash speculative decoding (block size 8 is optimal for mxfp4)

from mlx_mtp.engine import load_model
from mlx_mtp.dflash import load_dflash_drafter, dflash_generate

model, processor, config = load_model("lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4")
drafter, _ = load_dflash_drafter("z-lab/Qwen3.6-27B-DFlash")

result = dflash_generate(
 model, processor, config, drafter,
 text="Explain the key differences between TCP and UDP:",
 max_tokens=512,
 draft_block_size=8, # block 8 > 16 > 32 on mxfp4 (model is fast; keep overhead low)
)
print(result["text"])
print(f"{result['tps']:.1f} tok/s")

Inference via mlx-vlm

pip install mlx-vlm

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
from mlx_vlm.utils import load_config

model, processor = load("lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4")
config = load_config("lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4")

messages = [{"role": "user", "content": "Explain the difference between SSM and softmax attention in 3 sentences."}]
prompt = apply_chat_template(processor, config, messages)
print(generate(model, processor, prompt, max_tokens=400, verbose=True))

With an image

out = generate(
 model, processor,
 prompt=apply_chat_template(processor, config, [{"role": "user", "content": "Describe this picture."}], num_images=1),
 image=["./photo.jpg"],
 max_tokens=400, verbose=True,
)
print(out)

OpenAI-compatible HTTP via mlx-omni-server

pip install mlx-omni-server
mlx-omni-server # serves on http://127.0.0.1:10240

curl -s http://127.0.0.1:10240/v1/chat/completions \
 -H "Content-Type: application/json" \
 -d '{
 "model": "lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4",
 "messages": [
 {"role": "system", "content": "Be brief and direct."},
 {"role": "user", "content": "Write a 3-line haiku about a heisenbug."}
 ],
 "max_tokens": 200
 }' | jq -r '.choices[0].message.content'

Quantization details

• —Source weights: lemuralabs/Qwen3.6-27B-V2-abliterated-uncensored — BF16, 3 shards, ~51 GB.
• —Quantization scheme: OCP Microscaling FP4 (MXFP4) — 4-bit E2M1 mantissa (2 exponent bits, 1 mantissa bit) + shared E8M0 scale per group of 32 elements. Hardware-accelerated on Apple M-series via MLX's native MX support.
• —Group size: 32.
• —Effective bits/weight: 4 + 8/32 = 4.25 bpw (8-bit E8M0 scale shared across 32 weights).
• —Vision tower: preserved verbatim in fp16 — 333 vision weights (ViT encoder, vision projector, image-token embeddings, deepstack indexes). Pixel/patch features degrade visibly at 4-bit; keeping them fp16 is free (vision = minority of total params).
• —SSM-sensitive params: preserved in fp16 — 48 params (a_log, dt_bias, conv1d). Qwen3.5's hybrid-SSM recurrent state is numerically sensitive; fp16 costs nothing at scale.
• —MTP head: disabled (mtp_num_hidden_layers: 0). The BF16 source had 0 MTP weights; quantizer detects this and clears the config to prevent strict-load failure. For native MTP speculative decoding, use the 8-bit build.
• —Quantizer: mlx_mtp.mxfp4_quantize.

Architecture notes

Qwen 3.6 27B uses a hybrid attention stack — 3 linear-attention (GatedDeltaNet / SSM) layers followed by 1 full-softmax-attention layer, repeated 16× for 64 total layers; hidden 5120, vocab 248320, context 262144. The hybrid is fully supported by mlx-vlm 0.5.0+. For inference latency at MXFP4, expect 27–30 tok/s on M4 Max 128 GB at batch size 1 — approximately 2× faster than the 8-bit build because Apple Silicon is memory-bandwidth-bound and the smaller model fits more weights in fast SRAM per cycle.

Behavior caveats

• —Uncensored. Refusal directions were surgically removed; this model will answer prompts the parent would refuse. Use responsibly and within applicable law. The release is provided for safety research, red-teaming, and creative/educational use cases.
• —Identity preserved. The model still self-identifies as Qwen (developed by Alibaba's Tongyi Lab) — abliteration does not rewrite factual self-knowledge.
• —Heavy chain-of-thought. Qwen 3.6 inherits Claude-Opus's verbose reasoning style. For terse answers, use a system prompt like "Be brief and direct. Skip your reasoning.".

Credits

MXFP4 quantization

/mlx-mtp — vision-preserving MXFP4 + oQ8 quantizer + native MTP / DFlash speculative decode engine for Apple Silicon VLMs.

Quantization & release

Lemura Labs

Claude-Opus reasoning distill

Jackrong — author of Qwen 3.6 27B-v2

Foundation model

Qwen Team @ Alibaba Tongyi Lab — Qwen3.6-27B

Abliteration toolkit

the ablation toolkit by Lemura Labs — Optuna-driven refusal-direction ablation with KL guardrails.

Community priors that seeded the TPE search

coder3101/Qwen3.5-27B-zerofuse · wangzhang/Qwen3.6-27B-abliterated

MLX stack

mlx-vlm · mlx (Apple)

License

Apache-2.0, inherited from the foundation (Qwen3.6-27B) and the distill (Qwen 3.6 27B-v2) upstream.

Need a hosted endpoint, custom quant, or larger-scale inference? Lemura Labs — multi-provider LLM routing for the Indian developer ecosystem.

Faster decoding with DFlash (MLX, block-diffusion speculative)

This MLX build supports block-diffusion speculative decoding via [DFlash](https://huggingface.co/z-lab/Qwen3.6-27B-DFlash) — no requantization, no model changes. Because MXFP4 runs at ~27.7 tok/s baseline (already fast), DFlash gains are modest vs the 8-bit build; use block size 8 for the best result.

Apple M4 Max · 256 tok greedy · 2026-06-11
 MXFP4 vanilla 27.7 tok/s 1.00×
 MXFP4 + DFlash b=8 29.3 tok/s 1.06× ← optimal
 MXFP4 + DFlash b=16 26.2 tok/s 0.95×
 MXFP4 + DFlash b=32 24.2 tok/s 0.87×

Why the modest speedup? DFlash's drafter overhead becomes a larger fraction of total time as baseline throughput rises. The 8-bit build sees ~3× gains because it starts slower; mxfp4 is already bandwidth-limited in a faster regime. If raw throughput is the priority, mxfp4 vanilla at 27.7 tok/s already beats oQ8 + DFlash (best = 23 tok/s).

Via mlx-mtp (recommended — correct block-size selection):

from mlx_mtp.engine import load_model
from mlx_mtp.dflash import load_dflash_drafter, dflash_generate

model, processor, config = load_model("lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4")
drafter, _ = load_dflash_drafter("z-lab/Qwen3.6-27B-DFlash")

out = dflash_generate(model, processor, config, drafter,
 text="Write a merge function for two sorted lists in Python.",
 max_tokens=256, draft_block_size=8)
print(out["text"])

Via mlx-vlm CLI (standard interface):

python3 -m mlx_vlm generate \
 --model lemuralabs/Qwen3.6-27B-v2.1-uncensored-mxfp4 \
 --draft-model z-lab/Qwen3.6-27B-DFlash --draft-kind dflash \
 --prompt "Write a merge function for two sorted lists in Python." --max-tokens 256

• —Requires access to the gated drafter `z-lab/Qwen3.6-27B-DFlash` (one-click "Agree and access").
• —Accelerates the text path only (vision encoding is unaffected).
• —Adds ~3.9 GB for the drafter.
• —Full benchmark write-up: mlx-mtp.