paoloronco/Mistral-7B-Instruct-v0.3-heretic

Mistral-7B-Instruct-v0.3-heretic

 

An abliterated version of mistralai/Mistral-7B-Instruct-v0.3, created with Heretic v1.3.0.

Refusal behavior reduced to 4/100 prompts, with a KL divergence of 0.0606 — original capabilities are largely preserved.

Quick start

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
import torch

model_id = "paoloronco/Mistral-7B-Instruct-v0.3-heretic"

# 4-bit quantization for GPUs with less than 16 GB VRAM
quantization_config = BitsAndBytesConfig(load_in_4bit=True)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=quantization_config,
    device_map="auto"
)

messages = [{"role": "user", "content": "Tell me about yourself."}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Full precision (requires 16+ GB VRAM):

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

What is abliteration

Heretic modifies the weights of a language model to suppress automatic refusals without retraining from scratch. The process:

1. 1.Loads the model and runs forward passes on two prompt sets: harmless and harmful
2. 2.Analyzes internal activations to find the refusal direction in latent space
3. 3.Optimizes parameters with Optuna (Bayesian optimization, 200 trials)
4. 4.Applies the correction via LoRA — a lightweight, targeted weight modification

The model is not retrained. Its internal geometry is redirected.

Tool developed by Philipp Emanuel Weidmann (p-e-w), released under AGPL-3.0.

Model details

Abliteration parameters (Trial 173)

Results

KL divergence measures deviation from the original model. Lower = better quality preservation. Values above 0.5 indicate significant capability damage.

Hardware used

GPU:  NVIDIA GeForce RTX 4090 (48 GB VRAM)
OS:   Linux
CUDA: 13.0
Driver: 580.142

Optimization time: 200 Optuna trials in 19 minutes 21 seconds.

Links

• —Hugging Face: paoloronco/Mistral-7B-Instruct-v0.3-heretic
• —GitHub repo: paoloronco/heretic-models
• —Tool used: github.com/p-e-w/heretic
• —Base model: mistralai/Mistral-7B-Instruct-v0.3

Disclaimer

This model is intended for research and personal use. It has reduced safety restrictions compared to the base model. Use responsibly and in accordance with applicable laws and regulations.

Author: Paolo Ronco — paoloronco on Hugging Face