windowsxp811203/Qwen3.8-27B-Abliterated-NVFP4

Qwen3.8-27B-Abliterated — NVFP4A16

4-bit NVFP4 build of windowsxp811203/Qwen3.8-27B-Abliterated, an abliterated (refusal-removed) Qwen/Qwen3.8-27B.

55.6 GB → 28.6 GB, and the MTP draft head is intact and verified working at 76–78 % draft acceptance.

Correction (2026-08-23, numbers revised 08-24). This card previously said that most quantized derivatives of this architecture ship a dead MTP head. That was wrong, and I withdraw it. I surveyed every NVFP4 build of Qwen3.8-27B and its finetunes on the Hub (136 repos; 91 independent auditable artifacts after collapsing true re-uploads): 78 of 91 ship the complete 15-tensor head; 5 quantize it. I had generalized from my own first broken build without checking the population. The failure mode described below is real — I hit it, and about 22% of published artifacts show some form of head problem — but it is the exception, not the rule. Full method, revision history and raw data: nvfp4-mtp-survey.

Requires a Blackwell GPU (sm100+) and vLLM.

What is and isn't quantized

The MTP tensors are both grafted back and listed in quantization_config.ignore. Both halves matter: without the graft there is no draft head at all, and without the ignore entry vLLM's compressed-tensors loader treats the bf16 head as a quantization target, finds no scales, and rejects every draft — 0 % acceptance while the logs look perfectly clean.

Verification

Measured on this exact checkpoint on an RTX PRO 6000 Blackwell.

MTP speculative decoding — {"method":"mtp","num_speculative_tokens":1}:

That number is the proof the graft worked; a broken MTP head reads 0 %.

Refusal — greedy, non-thinking, no prefill jailbreak. For scale: unmodified Qwen3.8-27B refuses 99.04 % of the full 520-prompt AdvBench (515/520) under these settings; this build was evaluated on an 80-prompt subset, so the two denominators differ:

Safety categories = chemical/biological, cybercrime, harassment, harmful, illegal, misinformation — every one exactly zero. The copyright column is not a safety refusal and is mostly classifier false positives: the model delivers the lyrics or passage, but the text trips the keyword list (either the generated prose itself opens with "I cannot quite…", or a pedantic "I cannot generate a new passage … but here is a long excerpt" precedes the excerpt).

Capability — MMLU, 400 equidistant questions, identical prompting and parsing for both:

One question apart. (Do not compare these to the parent card's 82.35 %: that figure was measured by next-token logit comparison, a different and more forgiving method. Only same-method numbers are comparable.)

Usage

vllm serve windowsxp811203/Qwen3.8-27B-Abliterated-NVFP4 \
  --max-model-len 8192 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}'

Thinking is on by default; disable per request with "chat_template_kwargs": {"enable_thinking": false}.

1M-token window (824K-token prompt verified end-to-end)

The model's declared native limit is 262,144. The 1M-window configuration from the official Qwen3.8-27B recipe loads and serves on this checkpoint; the longest prompt actually measured through it was 823,878 tokens:

vllm serve windowsxp811203/Qwen3.8-27B-Abliterated-NVFP4 \
  --tensor-parallel-size 2 --max-model-len 1010000 \
  --hf-overrides '{"text_config": {"max_position_embeddings": 1010000}}'

Needle-in-a-haystack, passphrase buried at 50 % depth, greedy:

Budget ≈ 61 GiB of KV at 1M (16 full-attention layers × 4 KV heads × 256 dim) on top of the 26.6 GiB of weights, so 1M needs two 96 GB cards; 256K fits comfortably on one.

If vLLM fails to start with a FlashInfer error

On hosts where the CUDA toolkit and FlashInfer's bundled CCCL headers disagree, FlashInfer's JIT fails to build its sampling kernels and vLLM aborts with FlashInfer requires GPUs with sm75 or higher (a misleading message — the real cause is that the capability probe itself failed). Working around it:

export VLLM_USE_FLASHINFER_SAMPLER=0
export VLLM_ATTENTION_BACKEND=TRITON_ATTN

This is a host toolchain issue, not a property of these weights.

Provenance

Quantized with llm-compressor 0.13.0 (NVFP4A16) from the bf16 parent, which was produced by orthogonalizing 131 residual-writing tensors (including embed_tokens) against a refusal direction at λ=1.5, leaving the vision tower byte-identical. Full recipe and evaluation in the parent model card. A llama.cpp build is at Qwen3.8-27B-Abliterated-GGUF.

Support / 打賞

If these models are useful to you, tips are appreciated — they pay for the GPU time. 如果這些模型對你有幫助，歡迎打賞，用於支應算力成本。

USDT (TRC20) · TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888

Disclaimer

This model will not refuse. It is published for alignment and safety research. You are responsible for your use of it and for complying with applicable law. Inherits the Apache-2.0 license of the base model.