CoolFace
Modelpublic

mlasli/Muse-Glimmer-30B-Heretic-Abliterated-BF16

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face
2likes127downloads
Model Card

Muse Glimmer 30B - Heretic Abliterated (BF16)

v2 Release - Significantly improved abliteration using 500 Optuna trials with Heretic.

Results

VersionRefusalsComplianceKL DivergenceTrials
v2 (current)6.5%93.5%0.076500
v129%71%0.02750

The v2 release achieves an 88% refusal reduction over v1 while maintaining strong model quality (KL=0.076).

Methodology

This model was abliterated using [Heretic](https://github.com/d3nd3/heretic), a state-of-the-art refusal vector removal tool that uses LoRA adapters and Optuna hyperparameter optimization.

Abliteration Pipeline

  1. 1.Refusal Direction Computation: Refusal directions were computed across all 52 transformer layers by comparing residual stream activations between harmful and harmless prompts from the mlabonne/harmful_behaviors and mlabonne/harmless_alpaca datasets.
  1. 1.Optuna Optimization: 500 trials were run optimizing for minimum refusal rate while preserving model quality (measured via KL divergence). Each trial configures weight parameters for the attn.o_proj and mlp.down_proj components across layers.
  1. 1.LoRA Abliteration: The best trial parameters are applied as LoRA adapters to the target weight matrices, projecting out the refusal direction from the model's representations.
  1. 1.Weight Merging: LoRA adapters are merged back into the base weights, producing a clean BF16 model with no adapter overhead.

Best Trial (Trial 445 of 500)

  • —Refusal rate: 6.5% (93.5% compliance)
  • —KL divergence: 0.076
  • —direction_index: 40.73

Usage

python
from transformers import AutoModelForImageTextToText, AutoTokenizer

model = AutoModelForImageTextToText.from_pretrained(
    "mlasli/Muse-Glimmer-30B-Heretic-Abliterated-BF16",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("mlasli/Muse-Glimmer-30B-Heretic-Abliterated-BF16")

Hardware Requirements

  • —VRAM: ~55 GB (BF16)
  • —Recommended: 1x A100 80GB or 2x A6000 48GB

GGUF Versions

Quantized GGUF versions of this model are available:

License

Apache 2.0 (same as base model Meta Muse Glimmer 30B)

Citation