Modelpublic

MeghanaMakkapati13/gemma_quantization

sourceHugging Facegemmaupdated 3mo agoView on Hugging Face

0likes30downloads

Model Card

Gemma-4 E4B IQ4\_XS — Quantized GGUF

Compressed submission for the Resilient AI Challenge (Image-to-Text category), joint initiative of the Government of France, Government of India, UNESCO, and ITU.

Model details

Field	Value
Base model	`google/gemma-4-E4B-IT`
Fine-tuning	Parameter-efficient fine-tuning
Compression	IQ4\_XS (~4.25 bpw, importance-matrix quantization)
Format	GGUF (llama.cpp)
Vision projector	`mmproj-BF16.gguf` (BF16, unchanged)

Model size

Model	Size
F16 baseline	15.05 GB
IQ4\_XS (this model)	5.06 GB
mmproj (vision projector)	0.99 GB
Compression ratio	~3.7× smaller than F16

Running the model

Use llama-server with the provided llama_server_config.json:

bash

llama-server \
  -m gemma4-E4B-IQ4_XS.gguf \
  --mmproj mmproj-BF16.gguf \
  --host 0.0.0.0 --port 8080 \
  --n-gpu-layers 99 \
  --ctx-size 8192

Generation parameters: temperature=1.0, top_p=0.95, top_k=64