Modelpublic

smdesai/gemma-4-E2B-coreml

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face

2likes10downloads

Model Card

Gemma 4 E2B — CoreML (ANE+GPU Optimized)

Converted from google/gemma-4-E2B-it for on-device inference on Apple devices via CoreML.

Models

File	Size	Description
`model.mlpackage`	2.4 GB	Text decoder with stateful KV cache (int4 quantized)
`vision.mlpackage`	322 MB	Vision encoder (SigLIP-based, 16 transformer layers)
`model_config.json`	—	Model configuration
`hf_model/tokenizer.json`	31 MB	Tokenizer

Features

—Multimodal: Image + text input → text output
—ANE-optimized: Conv2d linear layers, ANE RMSNorm, in-model argmax
—Stateful KV cache: MLState API (iOS 18+)
—Int4 quantized: Block-wise palettization (group_size=32)
—HF-exact match: "solid red square centered on white background" ✅

Usage

python

import coremltools as ct
import numpy as np

# Load models
vision = ct.models.MLModel('vision.mlpackage')
decoder = ct.models.MLModel('model.mlpackage')
state = decoder.make_state()

# Process image → vision features → text generation

See CoreML-LLM for the full conversion pipeline and iOS sample app.

Conversion

bash

git clone https://github.com/john-rocky/CoreML-LLM
cd CoreML-LLM/conversion
pip install -r requirements.txt
python convert.py --model gemma4-e2b --context-length 512 --output ./output/gemma4-e2b