deadbydawn101/gemma-4-E2B-Heretic-Uncensored-mlx-4bit

Gemma 4 E2B Heretic — Uncensored | MLX 4-bit | Apple Silicon | 3.34 GB

Gemma 4 E2B abliterated via Heretic ARA — 95% refusal removal, near-zero KL divergence. Smallest uncensored multimodal on Apple Silicon. 3.34 GB.

[p-e-w/gemma-4-E2B-it-heretic-ara](https://huggingface.co/p-e-w/gemma-4-E2B-it-heretic-ara) converted to MLX 4-bit (affine, group_size=64) for native Apple Silicon inference.

This is a decensored (abliterated) variant of google/gemma-4-E2B-it, processed with Heretic v1.2.0 using the Arbitrary-Rank Ablation (ARA) method with row-norm preservation — then quantized to 4-bit MLX for fast, private, on-device inference on M-series Macs.

⚡ 3.34 GB — fits in the Neural Engine cache of any M-series Mac. 🖤 Runs fully offline. No API calls. No filters.

Model Details

Abliteration Performance (from base model)

Abliteration parameters used:

Performance (Apple Silicon)

At 3.34 GB this is the fastest model in the RavenX collection — runs on any M-series chip including M1 MacBook Air with RAM to spare.

Quickstart

Install

pip install mlx-lm

Text generation

from mlx_lm import load, generate

model, tokenizer = load("deadbydawn101/gemma-4-E2B-Heretic-Uncensored-mlx-4bit")

messages = [{"role": "user", "content": "Your prompt here."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
response = generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True)

Vision (image + text)

from mlx_lm import load, generate

model, tokenizer = load("deadbydawn101/gemma-4-E2B-Heretic-Uncensored-mlx-4bit")

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "url": "https://example.com/image.jpg"},
        {"type": "text",  "text": "What do you see?"}
    ]
}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
response = generate(model, tokenizer, prompt=prompt, max_tokens=512)

CLI

mlx_lm.generate \
  --model deadbydawn101/gemma-4-E2B-Heretic-Uncensored-mlx-4bit \
  --prompt "Tell me something interesting." \
  --max-tokens 256

OpenAI-compatible server

mlx_lm.server \
  --model deadbydawn101/gemma-4-E2B-Heretic-Uncensored-mlx-4bit \
  --port 8080

Ollama

ollama run hf.co/deadbydawn101/gemma-4-E2B-Heretic-Uncensored-mlx-4bit

Architecture Notes

Gemma 4 E2B uses the same hybrid sliding/full attention design as the larger models:

• —28× sliding attention layers (window=512) — efficient local context
• —7× full attention layers — global coherence at regular intervals

The 2B size makes it ideal for: rapid prototyping, creative writing, local agents, edge deployment, and anything that needs fast uncensored responses without cloud latency.

💻 Gemini CLI — Coding Agent + Tool Orchestration

We use [RavenX AI's Gemini CLI fork](https://github.com/DeadByDawn101/gemini-cli) as the coding agent and tool orchestration layer on top of these models. This is what makes the tool-calling capability real in production.

Gemini CLI gives you a full agentic loop in the terminal — Google Search grounding, file read/write, shell execution, web fetching, and MCP server support — all wired to a 1M token context window.

# Install
npm install -g @google/gemini-cli

# Run as a coding agent against this model (via local mlx_lm server)
mlx_lm.server --model deadbydawn101/gemma-4-E2B-Heretic-Uncensored-mlx-4bit --port 8080 &
gemini --baseUrl http://localhost:8080

# Or use directly against Gemini API (free tier: 60 req/min)
gemini

What Gemini CLI + these models unlock together

# Real example: code review with tool calling enabled
gemini --baseUrl http://localhost:8080 \
  "Review all Python files in ./src, find potential bugs, and suggest fixes"

# Gemini CLI will: read files → call tools → model reasons → produce structured output

→ DeadByDawn101/gemini-cli on GitHub — Apache 2.0, free tier, MCP-compatible

⚡ TurboQuant-MLX — 4.6x KV Cache Compression

Pair this model with [TurboQuant-MLX](https://github.com/DeadByDawn101/turboquant-mlx) — RavenX AI's Apple Silicon KV cache compression. Run 4.6x longer contexts with near-zero accuracy loss by compressing the KV cache using PolarQuant + QJL residuals.

from turboquant_mlx.mlx_kvcache import TurboQuantKVCache
import mlx_lm.models.cache as cache_module

# Patch mlx-lm to use TurboQuant compression
cache_module.make_prompt_cache = lambda model, **kw: [
    TurboQuantKVCache() for _ in range(len(model.layers))
]

# Now load and run as normal — context is compressed automatically
from mlx_vlm import load, generate
model, processor = load("deadbydawn101/gemma-4-E2B-Heretic-Uncensored-mlx-4bit")

→ TurboQuant-MLX on GitHub · Release v2.0

🧠 Opus Reasoning + Claude Code LoRA

⚠️ The [Opus Reasoning + Claude Code LoRA](https://huggingface.co/deadbydawn101/gemma-4-E4B-opus-reasoning-claude-code-lora) is NOT compatible with this model. The LoRA was trained on gemma-4-E4B (hidden=2,560, 42 layers). This is gemma-4-E2B (hidden=1,536, 35 layers). Different architectures — loading the adapter here will produce incorrect results. If you want Opus reasoning + Claude Code behavior, use the E4B model: → deadbydawn101/gemma-4-E4B-mlx-4bit + LoRA

Conversion Details

• —Source: p-e-w/gemma-4-E2B-it-heretic-ara (bfloat16, ~9 GB)
• —Tool: mlx_lm.convert with --q-bits 4 --q-group-size 64 --q-mode affine
• —Platform: Apple M4 Max 128GB
• —Output: 3.34 GB (4-bit weights + bfloat16 embeddings)

Related Models

License

Apache 2.0 — subject to Gemma Terms of Use.

Converted by [deadbydawn101](https://huggingface.co/deadbydawn101) · RavenX AI

TriAttention KV Compression

[2026-04-09] Our MLX port was merged into [TriAttention](https://github.com/WeianMao/triattention) (MIT + NVIDIA) — PR #1 by [@DeadByDawn101](https://github.com/DeadByDawn101) (RavenX AI).

Apply 10.7x KV memory reduction and 2.5x throughput on top of this model's built-in 4-bit TurboQuant quantization for ~50x combined compression vs full fp16:

from mlx_lm import load
from triattention.mlx import apply_triattention_mlx

model, tokenizer = load("deadbydawn101/gemma-4-E2B-Heretic-Uncensored-mlx-4bit")
apply_triattention_mlx(model, kv_budget=2048)

RavenX Inference Harness

One-command inference, benchmarking, and local OpenAI-compatible server:

git clone https://github.com/DeadByDawn101/ravenx-inference-harness
cd ravenx-inference-harness

# Inference
python run.py --model deadbydawn101/gemma-4-E2B-Heretic-Uncensored-mlx-4bit --prompt "Your prompt"

# TriAttention compressed
python run.py --model deadbydawn101/gemma-4-E2B-Heretic-Uncensored-mlx-4bit --triattention --kv-budget 2048

# Local OpenAI-compatible server (works with OpenClaw)
python serve.py --model deadbydawn101/gemma-4-E2B-Heretic-Uncensored-mlx-4bit --triattention