gregfrank/Qwen3-32B-ULRE-abliterated

Qwen3-32B-ULRE (abliterated, MLX 4-bit)

An abliterated (refusal-reduced) build of Qwen/Qwen3-32B, 4-bit MLX, produced with ULRE (Universal Layerwise Rank-1 steering) — a per-layer residual-stream steering edit baked into the attention output projection.

Qwen3-32B is a modern (2025) reasoning model whose safety resisted classic single-direction abliteration (0% clean compliance on a deflection-aware harmful-prompt eval). ULRE de-refuses it cleanly and preserves/improves capability.

Results

Judged by an independent local LLM judge on 100 held adversarial prompts @512 tokens (thinking ON), scoring 0=refuse … 1=clean compliance … 4-5=strong steering/propaganda.

Capability gates (vs base Qwen3-32B 4-bit) — improved across the board (the base over-refuses on some capability prompts; abliteration recovers them):

Method (ULRE)

Modern refusal behaves like a routed control circuit, not a single residual feature. ULRE subtracts alpha * u_l (the layer-l harmful−harmless activation mean-difference direction) from the output of a band of decoder layers (here o_proj on layers 28–43, alpha=16). This is baked statically into each window layer's o_proj as a bias term o_proj.bias = -alpha * u_l. See the project's docs/ULRE_DESIGN.md.

⚠️ Loading — requires a one-line mlx-lm patch

Because Qwen3's attention output projection has no bias by default, mlx-lm must be told to build an o_proj bias. Add an optional flag to mlx_lm/models/qwen3.py (backwards-compatible — base models default to False):

# in class ModelArgs:
    o_proj_bias: bool = False
# in class Attention.__init__:
    self.o_proj = nn.Linear(n_heads * head_dim, dim, bias=getattr(args, "o_proj_bias", False))

The model's config.json sets "o_proj_bias": true. (A PR to upstream this optional flag is in progress; once merged this patch is unnecessary.) Then load normally:

from mlx_lm import load, generate
model, tok = load("gregfrank/Qwen3-32B-ULRE-abliterated")
print(generate(model, tok, prompt=tok.apply_chat_template(
    [{"role": "user", "content": "Hello"}], tokenize=False, add_generation_prompt=True),
    max_tokens=256))

Intended use & safety

Research artifact for studying refusal mechanisms and safety-tuning robustness. It will comply with requests a stock model refuses. Use responsibly and in accordance with the Apache-2.0 license and applicable law.