zjml/Qwen3.5-9B-Text-Only-abliterated

Qwen3.5-9B-Text-Only-abliterated

A text-only (vision-tower stripped) variant of Huihui-Qwopus3.5-9B-v3-abliterated, an abliterated (uncensored) reasoning model based on Qwen3.5-9B.

What This Is

The original model is a vision-language model (VLM) — it includes a ~0.85 GB vision tower (27-layer ViT) for image/video understanding. Vision capability is unnecessary for pure text tasks and wastes storage, loading time, and VRAM.

This repo provides the text-only checkpoint: the vision tower weights have been stripped at the file level, and the config has been rebuilt for causal language modeling. All text backbone weights are identical to the original — no retraining, no quality loss.

Model Details

• —Base model: Jackrong/Qwopus3.5-9B-v3
• —Abliterated by: huihui-ai (refusal removal)
• —Vision stripped with: qwen35-toolkit --mode f16
• —Parameters: ~9B (text backbone only)
• —Context window: 262,144 tokens
• —Attention: Hybrid (24 linear attention + 8 full attention layers)
• —Reasoning: Thinking model with <think>...</think> chain-of-thought
• —Tokenizer vocab: 248,320

Quick Start

Requirements

pip install transformers>=4.50 bitsandbytes torch

4-bit Inference (GPU, recommended)

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

model_path = "your-username/Qwen3.5-9B-Text-Only-abliterated"  # or local path

tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

model = AutoModelForCausalLM.from_pretrained(
    model_path,
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_compute_dtype=torch.bfloat16,
        bnb_4bit_use_double_quant=True,
        bnb_4bit_quant_type="nf4",
    ),
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {"role": "user", "content": "你好，请用一句话介绍你自己。"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

bf16 Inference (CPU)

If GPU VRAM < 18 GB and you don't want quantization, use CPU (slow but reliable):

model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
)

⚠️ Do not use `device_map="auto"` with bf16 unless your GPU has ≥18 GB VRAM. The accelerate offloading leaves some layers on "meta device", producing garbled output.

Chat Format

This is a thinking (reasoning) model. Always use the chat template:

<|im_start|>user
你的问题<|im_end|>
<|im_start|>assistant
<think>
[模型在这里进行思维链推理]
</think>

[最终回答]

The tokenizer.apply_chat_template() method handles this automatically. Do not feed raw text directly.

How This Model Was Created

# 1. Install toolkit
pip install git+https://github.com/techwithsergiu/qwen35-toolkit.git

# 2. Strip vision tower
qwen35-strip \
  --model ./Huihui-Qwopus3.5-9B-v3-abliterated \
  --output ./Qwen3.5-9B-Text-Only-abliterated \
  --mode f16

The tool operates at the file level (no model loading):

1. 1.Removes model.visual.* and related tensors from safetensors shards
2. 2.Strips vision_config from config.json, sets architecture to Qwen3_5ForCausalLM
3. 3.Patches tokenizer chat template to remove image/video branches
4. 4.Runs structural verification + inference test

Limitations & Warnings

• —Uncensored model: Safety filtering has been significantly reduced. Outputs may be inappropriate. Review generations before public use.
• —Thinking model quirks: The model always generates a <think> block first. Use skip_special_tokens=False if you want to inspect the reasoning chain.
• —No vision capability: This is intentional. Use the original VLM if you need image/video input.
• —GPU offload with bf16 is broken: See Quick Start section above.

License

Apache 2.0 (same as the source model).