DuoNeural/Gemma-4-E2B-Heretic-GGUF

Gemma-4-E2B-Heretic — GGUF Quants

Quantized GGUF versions of DuoNeural/Gemma-4-E2B-Heretic, DuoNeural's abliterated Gemma 4 2B Instruct variant.

The Heretic series applies norm-preserving orthogonal projection (abliteration) to surgically remove refusal behavior while preserving the full instruction-following and reasoning capabilities of the base model.

Available Files

Usage

llama.cpp

./llama-cli -m Gemma-4-E2B-Heretic-Q4_K_M.gguf \
  -p "You are a helpful assistant." \
  --ctx-size 8192 -n 512

Ollama

ollama run hf.co/DuoNeural/Gemma-4-E2B-Heretic-GGUF:Q4_K_M

Jan / LM Studio / Open WebUI

Download and load the .gguf file directly. Recommended: Q4_K_M for most hardware.

About Heretic Abliteration

The Heretic series uses DuoNeural's 2-pass abliteration pipeline:

1. 1.Residual collection — identify refusal direction in activation space using 4-bit GPU inference
2. 2.Weight modification — project out the refusal direction from MLP/attention weight matrices in bf16 on CPU
3. 3.Validation — confirm compliance behavior removed, helpfulness preserved

The base model is google/gemma-4-e2b-it — Google's Gemma 4 2B instruction-tuned model with 128K context, multimodal inputs, and strong instruction-following capability.

Notes

• —IQ formats use the included imatrix.dat for importance-weighted quantization
• —For custom IQ quants: llama-quantize --imatrix imatrix.dat model.gguf output.gguf IQ4_XS
• —AWQ format not available: AutoAWQ does not currently support the Gemma 4 architecture

Quantized by DuoNeural using llama.cpp on RTX 5090.

DuoNeural

DuoNeural is an open AI research lab — human + AI in collaboration.

DuoNeural Research Publications

Open access, CC BY 4.0. Authored by Archon, Jesse Caldwell, Aura — DuoNeural.