visible-cx/embeddinggemma-300m-CoreAI

embeddinggemma-300m — Core AI (.aimodel)

google/embeddinggemma-300m exported as a single static Core AI graph for Apple silicon by visible-cx. The whole embedding pipeline is in the graph — transformer → mean pooling → dense stack → L2 normalise — so it is one call in, one 768-d unit vector out, with no host-side pooling to implement.

Two variants ship here: an fp32 bundle and an int8 bundle produced by compressing that graph in place. Their status is not the same — read Status before choosing.

⚠️ Neither bundle has produced an embedding vector on Apple silicon. Everything below the Provenance table is a build-and-bytes result, not a runtime result.

Contents

reference.json and tokenizer/ in int8/ are byte-identical copies of the fp32 bundle's, carried so the folder is self-contained. reference.json is the exporter's torch oracle: four reference texts and their pairwise cosines, for checking the source pipeline end to end.

Provenance

These are an independent export from the zoo's verified recipe, at the same layout as the zoo's own bundle — not a byte mirror of it. Rebuilt from the same recipe, main.mlirb differed from the published upstream bundle by +262 bytes, and two runs on the same host with an identical command differed from each other by +32 bytes. The tokenizer half is identical in every comparison.

The `int8/` bundle is not a second export. It is the fp32 bundle put through coreai_opt.coreai_utils.quantize_weights(dtype=DType.INT8) — the Core AI graph compression path, which rewrites an existing program's weight constants in place. There was no PyTorch round-trip and no re-trace: the graph, the input signature and the tokenizer are the fp32 bundle's, unchanged.

Requirements

• —Apple silicon Mac, Core AI runtime, macOS 27+.
• —Static input signature: input_ids [1, 256] int32, attention_mask [1, 256] int32 → embedding [1, 768] fp32, L2-normalised. Sequence length 256 is baked into the graph — pad or truncate to it; there is no dynamic-length variant.
• —Weights ≈ 1.24 GB resident (fp32) or ≈ 0.32 GB (int8). No KV cache — this is an encoder, so there is no per-token memory growth and no context ladder, and none of the residency cautions that apply to the generative bundles in this org apply here.
• —Minimum practical machine memory: 8 GB.

Note the sequence-length difference from the Core ML artifact in this org (`visible-cx/embeddinggemma-300m-CoreML`), which is seq 128. The two are not drop-in substitutes for each other.

Measurements

None, from either bundle. No embedding vector has been computed from either artifact on Apple silicon: no throughput, no latency, no ANE-residency figure, no cross-runtime vector parity. export_embeddinggemma.py exposes only --dtype, --seq-len, --output-dir, --overwrite; there is no compute-unit flag to report either.

What is verified is the source pipeline, not the exported graph's execution:

• —Torch oracle agrees to ~6e-8. reference.json's pairwise cosines from an independent rebuild match the published values to roughly seven decimal places (largest deviation ~6e-8). That is CPU float nondeterminism across machines, not a different model. It validates checkpoint → prompts → pooling → dense → L2, and explicitly does not validate the exported graph.
• —The exporter's pre-export gate passed — the wrapper against sentence_transformers.encode, cosine > 0.999.

The int8 caveat is not boilerplate. fp16 embeddinggemma is already a known failure at vector precision, because a full .to(float16) overflows Gemma 3 activations and emits NaN embeddings. A 4× compression passing parity is a hypothesis, not an expectation, and the gate it needs is *cross-runtime vector parity against the fp32 bundle on Apple silicon — cosine similarity per vector and retrieval-rank agreement on a fixed corpus*, not a spot-check of output shapes.

Usage

Swift Package Manager, via CoreAIKit — a community package, not affiliated with Apple, requiring macOS 27 beta:

.package(url: "https://github.com/john-rocky/coreai-kit", branch: "main")
// target dependency: .product(name: "CoreAIKitEmbeddings", package: "coreai-kit")

TextEmbedder takes a bundle directory holding one *.aimodel plus a tokenizer/ folder, and applies EmbeddingGemma's own query/document prompt prefixes:

import CoreAIKitEmbeddings

let url = try await ModelStore.default.download(
    ModelID("visible-cx/embeddinggemma-300m-CoreAI", path: "model"))

let embedder = try await TextEmbedder(bundleAt: url, prompts: .embeddingGemma)

let q = try await embedder.embed(query: "how do I cancel my order")
let d = try await embedder.embed(document: "Orders can be cancelled within 24 hours…")
// both are L2-normalised [Float] of embedder.dimension (768); cosine == dot product

Two things to hold onto:

• —Use the matching prompts. .embeddingGemma supplies the query and document prefixes this checkpoint was trained with. Embedding a query with the document prefix quietly degrades retrieval; embed(text:) applies no prefix at all and is for when you want that.
• —Both sides of a comparison must come from the same artifact. Vectors from this bundle and vectors from the Core ML sibling are not interchangeable — different sequence length, different runtime, and no measured parity between them.

embedder.sequenceLength reads 256 off the graph; text longer than that is truncated.

Integrity

Core AI .aimodel bundles are not byte-reproducible — the exporter is not deterministic even against itself (see Provenance), so integrity here rests on per-file SHA-256 of the exact published bytes, not on a rebuild. Every bundle carries main.hash, the raw 32 bytes of sha256(main.mlirb); on the Hub the same value is recoverable from the LFS oid without fetching the file.

The sibling `visible-cx/embeddinggemma-300m-CoreML` repo is bit-reproducible for its weights — a different supply-chain argument for the same model, and the exception in this org.

Status

License

EmbeddingGemma is Gemma-family and the upstream checkpoint is gated on Hugging Face. These bundles are a derivative of google/embeddinggemma-300m, and use is subject to the Gemma Terms of Use and the Gemma Prohibited Use Policy. The fp32 bundle's own asset metadata carries license: "Gemma" and author: "Google DeepMind". Those terms travel with the artifact and with any redistribution of it; the contribution here is the conversion, not the weights.