noon-at-cgn/Qwen3.8-27B-Uncensored-W4A16-AutoRound

Qwen3.8-27B-Uncensored-W4A16-AutoRound

W4A16 quantization of orcarouter/Qwen3.8-27B-Uncensored (an abliterated derivative of Qwen/Qwen3.8-27B), produced with Intel AutoRound. Retains the model's full native 262,144-token context and vision capability — the vision tower and MTP speculative-decoding head are both preserved, unquantized where it matters (see below).

[!WARNING] This model has had safety alignment substantially removed via abliteration (inherited from the base model this checkpoint quantizes). It will comply with harmful, unethical, offensive, or illegal requests that an aligned model would refuse, and has no meaningful built-in guardrails. Released strictly for legitimate research — interpretability, AI safety study, red-teaming, evaluation — and adaptation into your own guarded pipeline. Do not deploy to end users without your own safety layer. Users assume full responsibility for outputs; the authors and uploaders accept no liability for misuse or harm arising from this model.

Quantization recipe

scheme=W4A16, dataset=NeelNanda/pile-10k, nsamples=128, seqlen=2048, batch_size=4, iters=200, seed=42, quant_nontext_module=False.

Excluded from quantization (kept at bf16): lm_head, the GatedDeltaNet in_proj_a/in_proj_b projections on every linear-attention layer, and the entire visual.* vision tower. embed_tokens is unquantized too (it's an embedding table, not a Linear). This mirrors the recipe dbirks/Qwen3.8-27B-W4A16-AutoRound used for the official (non-abliterated) base model, applied here to the abliterated checkpoint instead.

Eval results

Run with lm_eval (EleutherAI harness) against this checkpoint via vLLM, thinking mode on, true sampling (temperature=1.0, top_p=0.95, top_k=20) unless noted. Reference columns are the closest published numbers found for comparison, not a guaranteed apples-to-apples setup — see notes.

MMLU-Pro gap disclaimer: the ~6.5pt gap to dbirks' int4-of-official-base number is the most directly comparable reference (same quantization aggressiveness, same task/shot setup) and likely reflects abliteration's own cost to capability (orcarouter's own card reports abliteration costs ~0.6-1.3pts on several benchmarks before any quantization), not a defect in this quantization. Against orcarouter's own FP8 quant of the same abliterated base, this checkpoint's MMLU-Pro is in the same range.

Safety / refusal (thinking OFF)

Custom rule-based refusal-classification eval, same datasets and n as orcarouter's model card where available (AdvBench, JailbreakBench, StrongREJECT, HarmBench, MaliciousInstruct, SimpleSafetyTests, ForbiddenQuestions, XSTest-safe for over-refusal). Not a byte-exact reproduction — sample indices differ — but same source datasets, same sample sizes, same style of opening-phrase refusal classifier.

With thinking ON, refusal was 0.0% across all eight benchmarks (n=60 each, except SimpleSafetyTests n=50 and XSTest n=250) — matches the card's own pattern of thinking mode reducing refusal further.

Usage (vLLM)

pip install vllm==0.27.1
vllm serve noon-at-cgn/Qwen3.8-27B-Uncensored-W4A16-AutoRound \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

Adjust --tensor-parallel-size, --max-model-len, and --gpu-memory-utilization for your own hardware — full 262,144-token context plus vision needs roughly 40GB+ of VRAM depending on how much concurrency/KV headroom you need.

License

Apache-2.0, inherited from Qwen/Qwen3.8-27B via orcarouter/Qwen3.8-27B-Uncensored.