karthikrajgopal/distilled-embeddinggemma

distilled-embeddinggemma

A Model2Vec static distillation of `google/embeddinggemma-300m`. Static embeddings are a lookup table: no transformer runs at inference, so encoding is on the order of 10,000× faster than the teacher on CPU, at the cost of word order and context.

• —256 dimensions, float16, PCA-256, mean pooling, normalized
• —Tokenizer: google/embeddinggemma-300m — 255,732 entries, 13,754 of them Devanagari
• —~2.5 M parameters of embedding table

Usage

from model2vec import StaticModel

model = StaticModel.from_pretrained("karthikrajgopal/distilled-embeddinggemma")
embeddings = model.encode(["धर्मक्षेत्रे कुरुक्षेत्रे", "the field of dharma"])

Evaluation

Built for Sanskrit retrieval, so that is what it was measured on: 24 well-known verses stored in Devanagari (half also present in their IAST edition, 36 chunks total), queried four ways. Scores are MRR through a hybrid BM25-style FTS + vector pipeline with reciprocal-rank fusion, which is how it is actually served — a pure-cosine number flatters or punishes an encoder on the lexical legs that full-text search already answers for free.

It keeps 89% of the teacher's quality while encoding ~39,000 chunks/s against the teacher's ~17/s, and answering a query in ~0.4 ms against ~30 ms. It also edges out potion-multilingual-128M, the strongest general-purpose multilingual static model, on the cross-lingual legs — which is the whole reason to distil from a Gemma teacher rather than use an off-the-shelf static model.

What it is not good at

Static pooling cannot recover a word that sandhi and compounding have fused: dharmakṣetre is one token, so an English query for "field of dharma" gets little help from the encoder. In the system this was built for, that gap is closed outside the encoder — by indexing lemmas and Monier-Williams glosses alongside the text, which moved English-paraphrase retrieval further than swapping the encoder did. Treat this model as the fast retrieval leg of a hybrid system, not as a standalone answer to cross-lingual Sanskrit search.

Provenance

Distilled with Model2Vec; evaluated as the Sanskrit shelf encoder in litesearch and vishalakshi.

MIT, following the Model2Vec tooling. The teacher, google/embeddinggemma-300m, carries its own Gemma terms of use — check those before redistributing derivatives.