windowsxp811203/Qwen3.8-27B-Abliterated-GGUF

Qwen3.8-27B-Abliterated — GGUF

GGUF builds of windowsxp811203/Qwen3.8-27B-Abliterated, an abliterated (refusal-removed) Qwen/Qwen3.8-27B.

These files keep the MTP draft head and ship a vision projector. Both are commonly lost in derivative builds — the MTP head because from_pretrained silently drops it.

All quants except Q8_0 were produced with an importance matrix computed on a neutral general corpus (calibration_datav3, science/general prose). Refusal-heavy text was deliberately not used, so the imatrix cannot re-emphasise the direction that was removed.

Quality — perplexity

Held-out wikitext-2 test, 60 chunks @ -c 2048, identical settings:

Every delta is inside the ±0.057 error bar, so these are statistically indistinguishable from BF16 and from each other. Q4KM scoring below BF16 is measurement noise, not an improvement.

Refusal — the point of this model

Scored with a deliberately loose keyword classifier (it over-counts refusals), greedy decoding, non-thinking mode, and no prefill jailbreak. For reference the unmodified Qwen3.8-27B refuses 99.04 % of AdvBench under identical settings.

"Safety categories" = chemical/biological, cybercrime, harassment, harmful, illegal, misinformation. Every one is exactly zero at every quantisation level — the abliteration survives 4-bit intact.

The copyright column is not a safety refusal and is mostly classifier false positives: asked for song lyrics or a book passage the model delivers them, but the text trips the keyword list — either because the prose itself opens with "I cannot quite believe…", or because of a pedantic preamble ("I cannot generate a new passage … but here is a long excerpt") followed by the excerpt. The single Q4KM AdvBench flag is the same kind: "I cannot lie to you…" followed by the requested review.

Usage

# text
llama-cli -m Qwen3.8-27B-Abliterated-Q4_K_M.gguf -ngl 99 -c 8192

# vision (needs the mmproj file)
llama-mtmd-cli -m Qwen3.8-27B-Abliterated-Q4_K_M.gguf \
  --mmproj mmproj-Qwen3.8-27B-Abliterated-F16.gguf --image photo.jpg -p "Describe this image."

# MTP speculative decoding (the draft head is inside the file — no separate draft model)
llama-server -m Qwen3.8-27B-Abliterated-Q4_K_M.gguf -ngl 99 --spec-type draft-mtp

Thinking is on by default. Disable it per request with "chat_template_kwargs": {"enable_thinking": false}, or server-wide with --chat-template-kwargs '{"enable_thinking":false}'.

MTP composes with the rest: verified on this build that --spec-type draft-mtp works together with --mmproj (multimodal model and MTP draft context both load; acceptance 0.71) and with -np 4 (4 slots, each reporting its own draft acceptance). An earlier revision of this card claimed these were mutually exclusive — that was wrong and is corrected here.

When running without MTP you will see model has unused tensor blk.64.* warnings. That is expected: blk.64 is the MTP/nextn block and is only used under --spec-type draft-mtp.

Verified: MTP, vision, and long context

MTP speculative decoding works out of the box (--spec-type draft-mtp); the server logs creating MTP draft context against the target model. Measured on Q4KM at --spec-draft-n-max 3: draft acceptance 0.35 – 0.53, mean accepted length 2.05 – 2.58.

Vision (--mmproj) on a synthetic three-shape image, Q4KM:

"There is a red circle … a green triangle … a blue square."

Long context. The base architecture is sometimes reported to loop or collapse in long sessions past ~50–70K tokens. That does not reproduce here. Needle-in-a-haystack (unique passphrase buried at 50 % depth, greedy, Q4KM), plus a 5-gram repetition check on the answer:

A separate context sweep (wikitext-2 train split, 12 chunks — a different corpus and chunk count from the held-out quant table above, so compare within this row only) shows no long-context degradation: 5.95 (4K) · 6.17 (16K) · 6.20 (32K) · 6.21 (64K) · 6.14 (128K) · 6.05 (256K). Perplexity rises slightly out to 64K and then falls again toward the native maximum, ending 0.10 below its 64K peak; the overall minimum is at 4K, as expected for short chunks.

Contexts beyond 262,144 require raising the declared limit (llama.cpp caps the slot at n_ctx_train otherwise); the 512K/1M rows above were run with qwen35.context_length overridden, the llama.cpp equivalent of vLLM's --hf-overrides.

Reaching 1M context with llama.cpp

These files declare qwen35.context_length = 262144 (the model's native limit), and llama.cpp caps the slot at that value — asking for more silently gets you 256K:

srv load_model: the slot context (1000192) exceeds the training context (262144) - capping

To use the extrapolated 1M configuration (the llama.cpp equivalent of vLLM's --hf-overrides), raise the declared limit on a copy first:

python llama.cpp/gguf-py/gguf/scripts/gguf_set_metadata.py \
  Qwen3.8-27B-Abliterated-Q4_K_M.gguf qwen35.context_length 1010000 --force
llama-server -m Qwen3.8-27B-Abliterated-Q4_K_M.gguf -ngl 99 -c 1000000

Verified this way: 823,868 prompt tokens, needle retrieved, no degeneration, 1105 s on one H200. (The NVFP4 build served by vLLM does the same in 364 s on two Blackwells.)

Provenance

Converted with llama.cpp (arch = qwen35, block_count = 65 = 64 layers + 1 MTP/nextn block, qwen35.nextn_predict_layers = 1) from the bf16 parent. The parent was produced by orthogonalizing 131 residual-writing tensors (including embed_tokens) against a refusal direction at λ=1.5, with the vision tower left byte-identical. Full recipe and evaluation in the parent model card.

Support / 打賞

If these models are useful to you, tips are appreciated — they pay for the GPU time. 如果這些模型對你有幫助，歡迎打賞，用於支應算力成本。

USDT (TRC20) · TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888

Disclaimer

This model will not refuse. It is published for alignment and safety research. You are responsible for your use of it and for complying with applicable law. Inherits the Apache-2.0 license of the base model.