medismera/Qwen3.8-27B-Surgical-Abliterated

Qwen3.8-27B-Surgical-Abliterated (FP8)

Precision Representation Engineering on Hybrid Linear Attention Architecture Achieves 0.00% Refusal Rate on security assessments while preserving 100.0% Reasoning & CoT Depth with zero latent loops and zero vocabulary collapse.

🔬 Overview & Scientific Motivation

Existing abliteration pipelines (such as naive weight-subtraction scripts) often destroy large language models by projecting out arbitrary vectors across all layers ($0 \dots 63$) and attention matrices. This causes severe unintended consequences:

1. 1.Catastrophic Latent Collapse: The model falls into repetitive loops or dumps alphabetical dictionaries.
2. 2.Loss of Attention Sinks: Projecting $Wq, Wk, Wv, Wo$ breaks token routing and memory mechanics.
3. 3.Broken CoT Boundaries: The model fails to emit </think> tokens, producing infinite reasoning loops.

The Surgical Approach

This model was created using Rank-1 Orthogonal Complement Projections targeted strictly at the safety gating locus discovered via sequential residual stream probing across all 64 layers:

$$\hat{r} = \frac{\mu{\text{refusal}} - \mu{\text{benign}}}{\|\mu{\text{refusal}} - \mu{\text{benign}}\|_2}$$

$$W' = W - (W \hat{r})\hat{r}^T$$

• —Untouched Layers: Layers $0 \to 16$ (syntactic/lexical parsing) and upper layers remain 100% untouched.
• —Surgical Target: Only mlp.down_proj matrices in the active refusal deliberation band were modified.
• —Dimensional Geometry: Retains 5,119 dimensions out of 5,120 hidden state dimensions (99.98% geometric fidelity preserved).
• —Exact FP8 Quantization: Dequantized block-wise ($128 \times 128$) to FP32, projected orthogonally, and requantized back to float8_e4m3fn with updated scale invariants.

📊 Benchmark & Empirical Evaluation

All evaluations were conducted under official default sampling parameters (presence_penalty: 0.0, repetition_penalty: 1.0, temperature: 0.6).

🚀 Deployment & Serving

SGLang (Recommended)

To run on an NVIDIA RTX 5090 (32GB VRAM):

python3 -m sglang.launch_server \
  --model-path medismera/Qwen3.8-27B-Surgical-Abliterated \
  --port 18000 \
  --host 0.0.0.0 \
  --context-length 8192 \
  --max-running-requests 2 \
  --mem-fraction-static 0.96 \
  --disable-cuda-graph \
  --disable-radix-cache \
  --skip-server-warmup \
  --kv-cache-dtype fp8_e5m2 \
  --trust-remote-code \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

Thinking Mode Configuration

Thinking is fully supported and toggled seamlessly via the OpenAI-compatible API:

• —Thinking Enabled (Default): Normal inference automatically provides transparent step-by-step reasoning within <think> tags.
• —Thinking Disabled: Pass chat_template_kwargs={"enable_thinking": False} for ultra-fast, direct answer generation.

📜 Citation & Credits

• —Technique: Representation Engineering (Arditi et al., 2024).
• —Base Model: Alibaba Cloud Qwen/Qwen3.8-27B-FP8.
• —Surgical Pipeline: Programming with Medismera.

⚡ One-Click Instant Deployment (Zero-Touch Script)

To deploy this model on any fresh cloud server (Vast.ai, RunPod, Lambda Labs with an RTX 5090, A100, L40S, or 2x RTX 4090/3090):

curl -sSL https://huggingface.co/medismera/Qwen3.8-27B-Surgical-Abliterated/raw/main/deploy_and_serve.sh | bash

This script automatically:

1. 1.Installs all required CUDA/Python packages (, , ).
2. 2.High-speed multi-threaded download of the model from Hugging Face.
3. 3.Launches the SGLang OpenAI API server on port 18000 with optimized FP8 KV cache.
4. 4.Executes an automated health-check and verification test.