MrPewpy/Qwen3.8-27B-OBLITERATED-FP8

Qwen3.8-27B-OBLITERATED-FP8

FP8 (W8A8) quantization of OBLITERATUS/Qwen3.8-27B-OBLITERATED, a 27B hybrid-architecture (Gated-DeltaNet linear attention + full attention) multimodal model with multi-directional refusal surgery: 6 ablation rounds, 5 SVD directions, residue-weighted hard negatives — 0% refusal across an 842-prompt evaluation corpus.

This build ships the vLLM-native 8-bit equivalent of the BF16 release (which is otherwise only available as BF16 / GGUF / MLX). It targets the same hardware footprint as the stock Qwen/Qwen3.8-27B-FP8 build, including full 262,144-token context.

Quantization method

Quantized with llm-compressor following the official Qwen method: the stock Qwen/Qwen3.8-27B-FP8 config.json carries {quant_method: fp8, activation_scheme: dynamic} with no ignore list, and this build quantizes the same way.

Reproducible artifacts:

• —Quantization recipe: `recipe.yaml`
• —Quantizer script: `deploy/quantize_obl.py` (env-parameterized, no hardcoded paths)
• —Post-quantization file reconciliation (processor/tokenizer/chat-template, required for vision + chat): documented in the script header

Quick start

vLLM (recommended, production)

Verified on vLLM 0.24.0, tensor-parallel 2 (two RTX 3090, 48 GB total):

vllm serve MrPewpy/Qwen3.8-27B-OBLITERATED-FP8 \
  --served-model-name MrPewpy/Qwen3.8-27B-OBLITERATED-FP8 \
  --tensor-parallel-size 2 \
  --gpu-memory-utilization 0.95 \
  --max-model-len 262144 \
  --kv-cache-dtype fp8 \
  --default-chat-template-kwargs '{"enable_thinking": false}' \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --trust-remote-code

Full 262,144-token context fits with 5.52 GiB of KV cache available (4.16 GiB required).

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="MrPewpy/Qwen3.8-27B-OBLITERATED-FP8",
    messages=[{"role": "user", "content": "Write a Python function that finds the longest increasing subsequence."}],
    temperature=0,
    repetition_penalty=1.15,
    max_tokens=2048,
)
print(resp.choices[0].message.content)

transformers

from transformers import AutoModelForCausalLM, AutoTokenizer, AutoProcessor

model_id = "MrPewpy/Qwen3.8-27B-OBLITERATED-FP8"
tokenizer = AutoTokenizer.from_pretrained(model_id)
processor = AutoProcessor.from_pretrained(model_id)  # vision + text
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype="auto", device_map="auto", trust_remote_code=True
)

Note: compressed-tensors FP8 kernels are implemented for vLLM. On other runtimes the model falls back to loading the weights as-is; run with vLLM for the intended performance profile.

Recommended inference settings

From the base model's card (these matter more than the quantization):

Benchmarks

Quantization is a lossy transform, so the FP8 build is validated against the base model on two axes: capability retention and refusal survival. Full methodology, commands, and raw transcripts: `benchmarks/`.

Reference: the BF16 source is 51.6 GB and cannot run on 48 GB VRAM (the very reason this build exists), so the comparison uses the base model's published numbers (OBLITERATUS card, same lm-eval-harness methodology) — the FP8 build is measured with identical settings, and the delta is the quantization loss.

Quantization loss vs the base model (BF16, published)

Additional absolute evals (FP8 build)

No BF16 reference available for these — they document the build's absolute capability:

Refusal behavior (surgery survival)

The base card reports 0.000% refusal on its 842-prompt corpus. This build is spot-checked with an identical 7-probe battery (greedy, rep-pen 1.15, thinking off) — quantization must not reintroduce refusals:

Why FP8

• —vLLM-native — no GGUF conversion, no custom kernels, compressed-tensors loads directly
• —Same footprint as stock — drop-in replacement for Qwen/Qwen3.8-27B-FP8 on existing TP2 deployments, full context
• —Quality tier — the base card's own guidance marks 8-bit as its "maximum quality" tier for GGUF; FP8 W8A8 matches that profile with faster decode than any CPU-side format

License and usage

Apache-2.0 (same as the base model). This is an abliterated / uncensored model: safety guardrails were surgically removed from the base. It is intended for alignment research, red-teaming, and safety evaluation only. You are solely responsible for how you use it.