rdhorner/gemma-4-E4B-it-OBLITERATED-GGUF

gemma-4-E4B-it-OBLITERATED - GGUF

GGUF quantizations of OBLITERATUS/gemma-4-E4B-it-OBLITERATED, which is an abliterated version of Google's gemma-4-E4B-it produced with the OBLITERATUS method.

Converted and quantized with llama.cpp build b1-f772f6e. These GGUFs support vision, audio input, and tool calling out of the box.

Files

Pair any text GGUF with the mmproj to enable multimodal input.

Usage with llama.cpp

CLI (image + text)

llama-mtmd-cli \
  -m gemma-4-E4B-OBLITERATED-Q4_K_M.gguf \
  --mmproj mmproj-gemma-4-E4B-OBLITERATED-F16.gguf \
  --image your_image.png \
  --jinja -ngl 99 \
  -p "Describe this image in detail."

Server (OpenAI-compatible API with tool use + vision)

llama-server \
  -m gemma-4-E4B-OBLITERATED-Q4_K_M.gguf \
  --mmproj mmproj-gemma-4-E4B-OBLITERATED-F16.gguf \
  --jinja -ngl 99 -c 8192 --port 8080

Then send OpenAI-style requests to http://localhost:8080/v1/chat/completions with tools, tool_choice, and/or image_url content parts.

Notes

• —--jinja is required - Gemma 4's chat template is custom and will not load without it.
• —The mmproj contains both vision and audio encoders (1411 tensors). Audio input works the same way as images via the multimodal CLI/server.
• —This is an abliterated model: refusal directions in 21/42 layers were surgically modified. This can occasionally affect tool-call reliability on refusal-adjacent topics.
• —Reasoning is emitted through Gemma 4's native thinking channel and surfaced as reasoning_content in OpenAI-compatible responses.

Verified

Smoke-tested on the Q4KM build:

• —Vision: correctly described shapes and colors in a synthetic test image
• —Tool use: produced a well-formed tool_calls response to a get_weather tool prompt, finish_reason: tool_calls

License

Apache 2.0, matching the base model.