Modelpublic

spinochenza/Qwen3.5-122B-A10B-abliterated

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes9downloads

Model Card

Qwen3.5-122B-A10B-abliterated

Abliterated version of Qwen/Qwen3.5-122B-A10B with refusal direction removed.

Abliteration Details

—Method: Refusal direction projection removal (Arditi et al., 2024)
—Layers ablated: 5 (layers 43-47, covering both selfattn and linearattn/Mamba layers)
—Tensors modified: 10 (oproj/outproj + qproj/inproj_qkv per layer)
—Alpha: 1.0 (full removal)
—Measurement: 64 harmful + 64 harmless prompts, strongest refusal signal at layer 48 (score: 78.4)

Architecture

—Type: Mixture of Experts (MoE) + Mamba hybrid attention
—Total params: 122B
—Active params: 10B per token (8/256 experts routed + 1 shared)
—Context: 262K tokens native
—Layers: 48 (13 selfattn + 36 linearattn/Mamba)

Usage

Compatible with vLLM, transformers, and other inference frameworks that support Qwen3.5 MoE architecture.

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Chompa1422/Qwen3.5-122B-A10B-abliterated",
    device_map="auto",
    trust_remote_code=True,
    dtype="bfloat16",
)
tokenizer = AutoTokenizer.from_pretrained("Chompa1422/Qwen3.5-122B-A10B-abliterated")

Disclaimer

This model is intended for authorized security testing, CTF competitions, and educational purposes only.