CoolFace
Modelpublic

CQSystems/Google-Gemma-4-E4B-It-GGUF

sourceHugging Facegemmaupdated 5mo agoView on Hugging Face
2likes126downloads
Model Card

CQ-Gemma-E4B-IT (GGUF)

This repository contains quantized GGUF formats of the Google Gemma 4 E4B Instruction-Tuned model, published by CQ Systems.

These files are optimized for local inference on consumer hardware, particularly Apple Silicon (M-series Macs) and standard GPUs, using tools like llama.cpp, LM Studio, and Ollama.

Available Files

We provide three different versions of the model to suit different memory and performance requirements.

FilenameSizeQuantizationRecommendation / Notes
CQ\_Gemma4\_E4B\_f16\_Q4\_K.gguf5.0 GBQ4\K\M🏆 Recommended for most users. Best balance of speed, size, and quality. Easily fits in 8GB+ unified memory (e.g., M1/M2/M3/M4 Macs).
cq\_gemma4\_e4b\_q8.gguf7.5 GBQ8\_0High quality 8-bit integer quantization. Nearly indistinguishable from the F16 baseline. Requires \~10GB+ RAM/VRAM.
CQ\_Gemma4\_E4B\_f16.gguf14.0 GBF16Baseline 16-bit float. Highest fidelity, highest memory footprint. Useful for re-quantization or research purposes.

How to Use

Using llama.cpp

You can run this model directly in your terminal using the compiled llama.cpp CLI tool.

\# Basic chat launch with the recommended Q4\K model ./llama-cli \-m CQ\Gemma4\E4B\f16\Q4\K.gguf \\ \-c 8192 \\ \-n 1024 \\ \-p "You are a helpful assistant.\\n\\nUser: Write a short joke about saving RAM.\\nAssistant:"

Using LM Studio / Ollama

  1. 1.Download the .gguf file of your choice (we recommend CQ\Gemma4\E4B\f16\Q4\_K.gguf).
  2. 2.LM Studio: Drag and drop the downloaded file into your LM Studio model folder, or use the local import feature.
  3. 3.Ollama: Create a Modelfile with the line FROM ./CQ\Gemma4\E4B\f16\Q4\_K.gguf, then run ollama create cq-gemma-e4b \-f Modelfile.

About the Base Model (Gemma 4 E4B)

Gemma 4 is a family of open, multimodal models built by Google DeepMind. The E4B variant is explicitly optimized for on-device processing, making it incredibly powerful for local deployment on laptops and mobile devices.

Key Capabilities

  • —Thinking Mode: Features a built-in reasoning mode that lets the model think step-by-step before answering.
  • —Massive Context: Supports a context window of up to 128K tokens.
  • —Parameter Efficiency: "E4B" stands for 4.5B effective parameters (8B total with Per-Layer Embeddings). This allows it to perform well above its weight class while keeping active memory lookups fast.
  • —System Prompts: Gemma 4 uses standard system, assistant, and user roles natively.

(Note: While the base Gemma 4 E4B supports audio and image inputs natively, these GGUF files primarily support text generation out-of-the-box via llama.cpp. Multimodal capabilities in GGUF require an accompanying mmproj vision/audio encoder file).

Thinking Mode Configuration

To enable the built-in reasoning capabilities, include the \<|think|\> token at the start of your system prompt. The model will then output its internal reasoning inside \<|channel\>thought\\n ... \<channel|\> tags before providing the final response.

License & Acknowledgements

  • —Base Model: Google DeepMind
  • —License: Gemma 4 License
  • —Quantization: CQ Systems using llama.cpp

For full benchmark details, evaluations, and ethical considerations, please refer to the original Google Gemma 4 E4B documentation.