cstr/embeddinggemma-300m-GGUF

embeddinggemma-300m GGUF

GGUF format of google/embeddinggemma-300m for use with CrispEmbed and Ollama-compatible runtimes.

Google EmbeddingGemma 300M. Lightweight multilingual embedding model based on Gemma 3, optimized for search, retrieval, and semantic similarity across 100+ languages.

Model details

• —Architecture: Gemma 3 transformer (300M params), bidirectional attention
• —Pooling: Mean pooling + Dense projection (768→3072→768) + L2 normalize
• —Embedding dimension: 768
• —Languages: 100+ languages
• —Context length: 2,048 tokens
• —License: Gemma

Files

Q4K shows mild degradation (~1.7%) which is typical for 4-bit quantization on embedding models. Use Q80 or Q5_K if highest fidelity is needed.

embeddinggemma-300m-qat-q8_0-dense.gguf (community / gemma-embedding arch)

This is the official llama.cpp QAT export (`ggml-org/embeddinggemma-300m-qat-q8_0-GGUF`, general.architecture = gemma-embedding) with the SentenceTransformers Dense head (2_Dense/3_Dense from `google/embeddinggemma-300m`) baked in as dense.0/1.weight. The upstream gemma-embedding GGUF omits the Dense modules (llama.cpp applies them from an external --sentence-transformers-dense-modules file); without them the output is the raw backbone mean-pool, orthogonal to real EmbeddingGemma. With Dense baked, CrispEmbed reproduces the HF SentenceTransformer output at cos 0.984 (the QAT checkpoint's weights differ slightly from vanilla google/embeddinggemma-300m, so this is measured vs vanilla — not a defect). Reproduce with models/add-st-dense-to-gguf.py.

Quick Start

# With CrispEmbed
crispembed -m embeddinggemma-300m.gguf "Hello world"

See CrispEmbed for full documentation.

Notes

These GGUFs use the Ollama-compatible format with CrispEmbed extension keys:

• —gemma3.is_bidirectional = 1 — bidirectional (no causal mask)
• —gemma3.pooling_type = 1 — mean pooling
• —gemma3.rope.freq_base_local = 10000.0 — sliding-window RoPE theta
• —Dense projection weights stored in F32 for correctness across all quant levels

Provenance and EU AI Act Art. 53 note

• —Upstream model: google/embeddinggemma-300m — published by google.
• —Upstream licence: gemma. This repository redistributes under the same terms; it grants no rights the upstream licence does not.
• —What was done here: format conversion and/or quantisation only (GGUF/GGML). No training, no fine-tuning, no merging, no distillation, no change to architecture, vocabulary or capability. Only the numeric representation of the upstream weights differs.
• —Training data: documented — where it is documented at all — by the upstream provider; see the upstream model card. No training data was used, added or selected by this repository.
• —Provider status: under Regulation (EU) 2024/1689 the upstream authors remain the provider of this model. Converting the serialisation format does not make this repository the provider of a new general-purpose AI model, and no such claim is made. Questions about training content, copyright policy or model capability belong upstream.