CoolFace
Modelpublic

anemll/anemll-gemma-3-270m-it-MONO-ctx512-lut6

sourceHugging Facemitupdated 8mo agoView on Hugging Face
1likes29downloads
Model Card

ANEMLL Gemma 3 270M — Apple Neural Engine (Monolithic, LUT6)

Pre-converted Gemma 3 270M IT model optimized for Apple Neural Engine inference using ANEMLL.

This is a monolithic (single-file) CoreML model with in-model argmax, ideal for quick testing and development on any Apple Silicon device.

Model Details

PropertyValue
Base Modelgoogle/gemma-3-270m-it
ArchitectureGemma 3 (gemma3_text)
Parameters270M
Context Length512
Batch Size64
QuantizationLUT6 (6-bit, per-channel group size 4)
ArgmaxIn-model (outputs token IDs)
FormatMonolithic (single CoreML file)
DedupANEMLL-Dedup enabled
ANEMLL Version0.3.5
Model Size~335 MB (compiled)

Files

FileSizeDescription
gemma3_monolithic_full_lut6.mlmodelc/335 MBCompiled CoreML model (infer + prefill)
meta.yaml2 KBModel configuration
tokenizer.json32 MBTokenizer data
tokenizer.model4.5 MBSentencePiece model
tokenizer_config.json1.1 MBTokenizer configuration
chat_template.jinja1.5 KBChat template
config.json66 BiOS tokenizer config

Quick Start

Download

bash
# Clone with git-lfs
git lfs install
git clone https://huggingface.co/anemll/anemll-gemma-3-270m-it-ctx512-lut6

# Or use huggingface-cli
huggingface-cli download anemll/anemll-gemma-3-270m-it-ctx512-lut6 \
  --local-dir ~/Models/ANE/gemma3-270m

Run with ANEMLL

bash
# Install ANEMLL
git clone https://github.com/Anemll/Anemll.git
cd Anemll
./create_uv_env.sh
source env-anemll/bin/activate
./install_dependencies.sh

# Chat with the model
python tests/chat.py \
  --meta ~/Models/ANE/gemma3-270m/meta.yaml \
  --prompt "Who are you?"

# Full conversation mode
python tests/chat_full.py \
  --meta ~/Models/ANE/gemma3-270m/meta.yaml

Run with ANEMLL Chat (macOS app)

  1. 1.Open ANEMLL Chat
  2. 2.Go to Models > Link Local Model
  3. 3.Select the downloaded model directory
  4. 4.Start chatting

Conversion

This model was converted using:

bash
python tests/test_gemma3_model.py \
  --model google/gemma-3-270m-it \
  --lut 6,4 \
  --lut-embeddings 6,4 \
  --lut-lmhead 6,4 \
  --context 512 \
  --batch 64

Or equivalently:

bash
./anemll/utils/convert_monolith.sh \
  --model google/gemma-3-270m-it \
  --output ./output \
  --lut 6,4 \
  --lut-embeddings 6,4 \
  --lut-lmhead 6,4 \
  --context 512 \
  --batch 64 \
  --argmax \
  --prefix gemma3

Notes

  • —270M is a tiny model — it's designed for testing the conversion pipeline, not for production quality output. For better quality, use larger models (1B+).
  • —Runs entirely on Apple Neural Engine — no GPU or cloud required.
  • —Supports Apple Silicon: M1, M2, M3, M4 and later.
  • —Inference speed: ~200+ tokens/sec on M-series chips.

License

This model conversion is released under the MIT license. The base model (Gemma 3) is subject to Google's Gemma Terms of Use.

Links