CoolFace
Modelpublic

Abiray/gemma-4-E4B-it-The-DECKARD-HERETIC-UNCENSORED-Thinking-GGUF

sourceHugging Facegemmaupdated 6mo agoView on Hugging Face
11likes1.8kdownloads
Model Card

Gemma 4 E4B Instruct - The DECKARD HERETIC UNCENSORED Thinking (GGUF)

This repository contains GGUF format quantizations for DavidAU's Gemma-4-E4B-it-The-DECKARD-HERETIC-UNCENSORED-Thinking.

These files allow for highly efficient local inference on CPUs and Apple Silicon, as well as VRAM-constrained GPUs, utilizing llama.cpp and compatible frontends.

๐Ÿง  About the Base Model

The base model is a specialized, heavily fine-tuned version of Google's Gemma 4 E4B (Effective 4B parameters).

  • โ€”HERETIC / DECKARD Fine-Tune: This model has undergone the "HERETIC" abliteration and fine-tuning process by DavidAU. It is completely uncensored, unrestricted, and heavily calibrated for complex creative writing, roleplay, and deep fiction generation.
  • โ€”Reasoning & Thinking: It includes augmented reasoning/thinking capabilities, allowing the model to internally chain logic before outputting text, making its responses deeper and more contextually accurate.
  • โ€”Context Window: The Gemma 4 architecture supports extended context windows (up to 256k in base), making it highly capable for long-form conversations.

๐Ÿ“ฆ Available Quantization Formats

This repository provides multiple levels of quantization to help you balance VRAM/RAM usage, generation speed, and model fidelity.

File NameBit ResolutionRecommended Use
Q3_K_M3-bitUltra-low RAM usage. Noticeable perplexity degradation but runs on very constrained hardware.
Q4_K_M4-bitRecommended. The sweet spot for local LLMs. Great balance of speed, low memory footprint, and quality.
Q5_K_M5-bitHigher precision. Use this if you have the memory to spare and want slightly better reasoning.
Q6_K6-bitVery high fidelity. Close to unquantized performance with a larger memory footprint.
Q8_08-bitExtremely close to FP16 baseline. Best for users who want maximum precision and have ample RAM/VRAM.