FreedomAISVR/Gemma-4-E2B-it-NVFP4-GGUF

Gemma 4 E2B IT — NVFP4 GGUF

Base Model

• —Model: google/gemma-4-E2B-it
• —Architecture: Gemma4ForConditionalGeneration (Vision + Text)
• —Parameters: ~3B (E2B = Efficient 2B-class)
• —Context Length: 131,072 tokens (128K)
• —License: Apache 2.0
• —Update Date: July 20, 2026

Quantization Details

• —Format: NVFP4 (NVIDIA Blackwell FP4)
• —BPW: 5.76 bits per weight
• —File Size: 3.4 GB
• —Non-expert tensors: F16

Vision Support

• —mmproj: mmproj-gemma-4-E2B-it-f16.gguf (985 MB, F16)
• —Supports image and video input

Performance (RTX 5060 Ti 16GB)

• —Generation Speed: ~153 t/s at 128K context
• —Context: 128K with Q8_0 KV cache fits in 16GB VRAM

Usage (llama.cpp)

llama-cli -m gemma-4-e2b-it-nvfp4.gguf -ngl 99 --flash-attn on -c 131072 --cache-type-k q8_0 --cache-type-v q8_0

Why No MTP?

The Gemma 4 E2B model does not include MTP (Multi-Token Prediction) heads.

Hardware Target

• —NVIDIA RTX 50 series (Blackwell) for NVFP4 acceleration
• —16GB+ VRAM recommended