sunil-pathak/gemma-3n-E2B-it-Q4_K_M

gemma-3n-E2B-it — GGUF (Q4KM)

📊 Performance Metrics

• —Hardware: Intel(R) Xeon(R) CPU @ 2.20GHz (4 vCPUs)
• —Size: 3.08 GB
• —Speed (Generation): 6.83 tokens/sec
• —Speed (Prompt): 17.89 tokens/sec
• —KV Cache Usage: 0.0143 GB
• —Quantization: Q4KM

🔷 Model Overview

This repository contains a GGUF quantized version of:

• —Base Model: gemma-3n-E2B-it
• —Format: GGUF (optimized for llama.cpp inference)
• —Precision: Q4KM
• —Efficiency Score: 2.2163 (TPS/GB)

GGUF format provides:

• —Fast loading via memory mapping
• —Single-file model distribution
• —Cross-platform compatibility
• —Efficient inference with llama.cpp

📦 Files

⚙️ Technical Details

⚡ Why GGUF?

GGUF is designed for efficient inference:

• —Optimized for llama.cpp
• —Supports CPU and GPU inference
• —Single-file deployment
• —Memory-mapped loading for speed
• —Ideal for edge / local environments

⚠️ License & Usage

This is a converted derivative model.

• —You must comply with the original model license of gemma-3n-E2B-it
• —This is not an official release
• —No additional rights are granted
• —Original ownership remains with the base model creator

🚀 Quick Start (llama.cpp)

./llama-cli -m gemma-3n-E2B-it-Q4_K_M.gguf -p "Explain AI simply"