Modelpublic

Neural-ICE/Gemma-4-E2B-it-NVFP4

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face

3likes223downloads

Model Card

Gemma-4-E2B-it-NVFP4

NVFP4 quantized version of google/gemma-4-E2B-it for vLLM.

Quantization Profile

—Text backbone: NVFP4
—lm_head: higher precision
—Vision tower and vision embeddings: higher precision
—Audio tower and audio embeddings: higher precision
—KV cache: FP8

Usage

bash

vllm serve Neural-ICE/Gemma-4-E2B-it-NVFP4 \
  --quantization modelopt \
  --gpu-memory-utilization 0.90

Official Gemma 4 vLLM recipe:

https://docs.vllm.ai/projects/recipes/en/latest/Google/Gemma4.html

Text Generation

bash

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Neural-ICE/Gemma-4-E2B-it-NVFP4",
    "messages": [
      {"role": "user", "content": "Explain quantum entanglement in simple terms."}
    ],
    "max_tokens": 512
  }'