ArithaAI/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-mlx-4bit

<div align="center">

Gemma 4 E4B — Opus Reasoning + Claude Code | Tool Calling ✅ | OpenHarness ✅ | OpenClaw ✅ | Hermes Agent ✅ | Reasoning Baked In

Opus 4.6 reasoning + Claude Code fused into weights. Native tool calling. OpenHarness agent harness. OpenClaw orchestration. Hermes terminal-agent skill. `<think>` reasoning baked in — no adapter needed. 10.5 GB.

Reasoning baked in. No adapter needed. Built by RavenX AI

  

</div>

Gemma 4 E4B with Opus Reasoning + Claude Code LoRA fused directly into the weights — no adapter needed, no extra memory, just load and run with Claude-style <think> reasoning baked in.

~10.5 GB. 131K context. Text + vision. Drop-in reasoning upgrade.

This is `gemma-4-E4B-mlx-4bit` with the Opus Reasoning + Claude Code LoRA merged directly into the base weights using mlx weight arithmetic.

What's different from the base model

🧪 Live Demos — Try It Now

<div align="center">

</div>

Quickstart

pip install mlx-lm mlx-vlm

from mlx_lm import load, generate

# No adapter_path needed — reasoning is in the weights
model, tokenizer = load("deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-mlx-4bit")

messages = [{"role": "user", "content": "Explain why RSA encryption is hard to break."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
response = generate(model, tokenizer, prompt=prompt, max_tokens=1024, verbose=True)
# → Will produce <think>...</think> followed by structured answer

CLI

mlx_lm.generate \
  --model deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-mlx-4bit \
  --prompt "Debug this Python code: def fib(n): return fib(n-1) + fib(n-2)" \
  --max-tokens 1024

🧩 OpenHarness + OpenClaw + Hermes Agent

This model is built to sit inside a real agent stack, not just a chat box.

We support:

• —[OpenHarness](https://github.com/HKUDS/OpenHarness) for agent harness/runtime, skills, hooks, tool loops, and multi-agent flows
• —OpenClaw for orchestration, sessions, reminders, and cross-agent routing
• —Hermes agent skill for terminal-native coding posture, short planning, aggressive tool use, and repo-aware execution

Why this combo matters

OpenHarness quickstart

pip install openharness

mlx_lm.server \
  --model deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-mlx-4bit \
  --port 8080

oh --model http://localhost:8080/v1 \
   --skill hermes-agent \
   -p "Review this repo, find bugs, patch them, and summarize the result"

OpenClaw skill stack

Inside OpenClaw, pair this model with:

• —openharness skill — run/configure oh
• —hermes-agent skill — shape coding-agent behavior

That gives you a fully local Apple Silicon agent lane with:

• —baked-in reasoning
• —native tool calling
• —Gemini CLI integration
• —OpenHarness runtime support
• —OpenClaw orchestration

💻 Gemini CLI — Coding Agent + Tool Orchestration

We use [RavenX AI's Gemini CLI fork](https://github.com/DeadByDawn101/gemini-cli) as the coding agent and tool orchestration layer on top of these models. This is what makes the tool-calling capability real in production.

Gemini CLI gives you a full agentic loop in the terminal — Google Search grounding, file read/write, shell execution, web fetching, and MCP server support — all wired to a 1M token context window.

# Install
npm install -g @google/gemini-cli

# Run as a coding agent against this model (via local mlx_lm server)
mlx_lm.server --model deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-mlx-4bit --port 8080 &
gemini --baseUrl http://localhost:8080

# Or use directly against Gemini API (free tier: 60 req/min)
gemini

What Gemini CLI + these models unlock together

# Real example: code review with tool calling enabled
gemini --baseUrl http://localhost:8080 \
  "Review all Python files in ./src, find potential bugs, and suggest fixes"

# Gemini CLI will: read files → call tools → model reasons → produce structured output

→ DeadByDawn101/gemini-cli on GitHub — Apache 2.0, free tier, MCP-compatible

⚡ TurboQuant-MLX — 4.6x KV Cache Compression

Pair with TurboQuant-MLX to compress the KV cache and run 4.6x longer reasoning chains at the same memory:

from turboquant_mlx.mlx_kvcache import TurboQuantKVCache
import mlx_lm.models.cache as cache_module

cache_module.make_prompt_cache = lambda model, **kw: [
    TurboQuantKVCache() for _ in range(len(model.layers))
]

from mlx_lm import load, generate
model, tokenizer = load("deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-mlx-4bit")
# Long reasoning chains now fit in the same RAM budget

→ TurboQuant-MLX on GitHub · v2.0 Release

How it was made

Training data

Training

Base:      deadbydawn101/gemma-4-E4B-mlx-4bit
Method:    SFT completions-only (mlx_vlm.lora)
Rank:      8 · Alpha: 16 · LR: 1e-5 · Iters: 1,000
Hardware:  Apple M4 Max 128GB · Peak mem: 7.876 GB

Final loss: ~3.5e-7

Fusion

All 378 LoRA pairs merged via weight arithmetic:

W_merged = dequantize(W_base) + (A @ B).T × (alpha / rank)

Result dequantized to bfloat16 and saved as 3-shard safetensors.

🦙 Ollama / LM Studio / llama.cpp

This is an MLX model optimized for Apple Silicon. For Ollama, LM Studio, or llama.cpp, use the GGUF version: 👉 [gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-GGUF](https://huggingface.co/deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-GGUF) Available in Q4KM (2.7 GB), Q5KM (3.1 GB), Q8_0 (4.5 GB), and F16 (8.3 GB). ``bash ollama run hf.co/deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-GGUF ``

Run with mlx_lm server (native, faster on Apple Silicon)

mlx_lm.server --model deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-mlx-4bit --port 8080

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-mlx-4bit", "messages": [{"role": "user", "content": "Hello!"}]}'

Related models

License

Gemma Terms of Use

<div align="center"> Built with 🖤 by <a href="https://github.com/DeadByDawn101">RavenX AI</a> · <a href="https://github.com/DeadByDawn101/turboquant-mlx">TurboQuant-MLX</a> · <a href="https://github.com/DeadByDawn101/gemini-cli">Gemini CLI</a> </div>

TriAttention KV Compression

[2026-04-09] Our MLX port was merged into [TriAttention](https://github.com/WeianMao/triattention) (MIT + NVIDIA) — PR #1 by [@DeadByDawn101](https://github.com/DeadByDawn101) (RavenX AI).

Apply 10.7x KV memory reduction and 2.5x throughput on top of this model's built-in 4-bit TurboQuant quantization for ~50x combined compression vs full fp16:

from mlx_lm import load
from triattention.mlx import apply_triattention_mlx

model, tokenizer = load("deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-mlx-4bit")
apply_triattention_mlx(model, kv_budget=2048)

RavenX Inference Harness

One-command inference, benchmarking, and local OpenAI-compatible server:

git clone https://github.com/DeadByDawn101/ravenx-inference-harness
cd ravenx-inference-harness

# Inference
python run.py --model deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-mlx-4bit --prompt "Your prompt"

# TriAttention compressed
python run.py --model deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-mlx-4bit --triattention --kv-budget 2048

# Local OpenAI-compatible server (works with OpenClaw)
python serve.py --model deadbydawn101/gemma-4-E4B-Agentic-Opus-Reasoning-GeminiCLI-mlx-4bit --triattention