pocharlies/Qwen3.6-27B-uncensored-heretic-v2-NVFP4-lmheadW4

Qwen3.6-27B-uncensored-heretic-v2-NVFP4-lmheadW4

What this is: the llmfan46 heretic-v2 NVFP4 uncensored checkpoint with a single, high-impact optimization: the `lm_head` re-quantized from BF16 to NVFP4 W4A16.

Community uncensored/abliterated checkpoints almost always ship the lm_head in BF16 (here ~2.5 GB over a 248,320-token vocabulary). That tensor is read every decode step, so on a memory-bandwidth-bound GPU it dominates the per-token cost and makes decode slow. NVIDIA's own NVFP4 checkpoints quantize the lm_head to 4-bit — which is why they decode fast. We simply brought this checkpoint in line.

Result (measured on NVIDIA GB10 / DGX Spark, sm_121)

+~47% decode, no quality regression (needle-in-haystack 8K/32K HIT, coherence preserved), uncensored behaviour preserved. On our cluster it now decodes faster than the dense Qwen3.6-27B (~35 tok/s).

How it was made

Standalone lm_head quantization (loads only the lm_head tensor, ~21 GB peak — avoids OOM on the 120 GB unified memory) with NVIDIA modelopt NVFP4QTensor.quantize(W, block_size=16), producing the modelopt tensor layout (weight packed uint8 [V, H/2], weight_scale FP8-E4M3 [V, H/16], weight_scale_2 FP32, input_scale FP32). The format is verified bit-for-bit against a reference NVFP4 lm_head before the shard is rewritten, and lm_head is removed from quantization_config.ignore. Everything else (MoE/attention weights, native MTP tensors) is untouched. Script: see the companion repo.

Serving (vLLM on GB10, custom sm121 build)

vllm serve pocharlies/Qwen3.6-27B-uncensored-heretic-v2-NVFP4-lmheadW4 \
  --served-model-name qwen36-27b-uncensored-nvfp4 \
  --max-model-len 229376 \
  --kv-cache-dtype fp8 \
  --quantization modelopt \
  --attention-backend flashinfer \
  --max-num-seqs 16 --max-num-batched-tokens 32768 \
  --gpu-memory-utilization 0.60 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --limit-mm-per-prompt '{"image":64}' \
  --trust-remote-code

KV cache nvfp4 is NOT possible on GB10 (FlashInfer requires sm100f; GB10 is sm121) → use --kv-cache-dtype fp8.

Attribution & license

Apache-2.0. Derivative of llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved-NVFP4, itself derived from Qwen/Qwen3.6-27B. Only the lm_head tensor was re-quantized; all other weights are unchanged. Abliteration/uncensoring was performed upstream by llmfan46 — this repo adds only a quantization/performance optimization.