headwAI/embeddinggemma-300m

1---2license: gemma3pipeline_tag: sentence-similarity4library_name: sentence-transformers5tags:6- sentence-transformers7- sentence-similarity8- feature-extraction9- text-embeddings-inference10base_model: google/embeddinggemma-300m11language:12- en13- de14---15 16# EmbeddingGemma model card17 18**Base Model Page** [EmbeddingGemma Huggingface](https://huggingface.co/google/embeddinggemma-300m)19 20**Blog Model Page**: [EmbeddingGemma Blog](https://ai.google.dev/gemma/docs/embeddinggemma)21 22**Resources and Technical Documentation**:23 24*   [Responsible Generative AI Toolkit](https://ai.google.dev/responsible)25*   [EmbeddingGemma on Kaggle](https://www.kaggle.com/models/google/embeddinggemma/)26*   [EmbeddingGemma on Vertex Model Garden](https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/embeddinggemma)27 28**Terms of Use**: [Terms](https://ai.google.dev/gemma/terms)29 30**Authors**: Google DeepMind31 32## Model Information33 34### Description35 36EmbeddingGemma is a 300M parameter, state-of-the-art for its size, open embedding model from Google, built from Gemma 3 (with T5Gemma initialization) and the same research and technology used to create Gemini models. EmbeddingGemma produces vector representations of text, making it well-suited for search and retrieval tasks, including classification, clustering, and semantic similarity search. This model was trained with data in 100+ spoken languages.37 38The small size and on-device focus makes it possible to deploy in environments with limited resources such as mobile phones, laptops, or desktops, democratizing access to state of the art AI models and helping foster innovation for everyone.39 40For more technical details, refer to our paper: [EmbeddingGemma: Powerful and Lightweight Text Representations](https://arxiv.org/abs/2509.20354).41 42### Inputs and outputs43 44-   **Input:**45    -   Text string, such as a question, a prompt, or a document to be embedded46    -   Maximum input context length of 2048 tokens47 48-   **Output:**49    -   Numerical vector representations of input text data50    -   Output embedding dimension size of 768, with smaller options available (512, 256, or 128) via Matryoshka Representation Learning (MRL). MRL allows users to truncate the output embedding of size 768 to their desired size and then re-normalize for efficient and accurate representation.51 52### Citation53 54```none55@article{embedding_gemma_2025gemma_embedding_2025,56    title={EmbeddingGemma: Powerful and Lightweight Text Representations},57    author={Schechter Vera, Henrique* and Dua, Sahil* and Zhang, Biao and Salz, Daniel and Mullins, Ryan and Raghuram Panyam, Sindhu and Smoot, Sara and Naim, Iftekhar and Zou, Joe and Chen, Feiyang and Cer, Daniel and Lisak, Alice and Choi, Min and Gonzalez, Lucas and Sanseviero, Omar and Cameron, Glenn and Ballantyne, Ian and Black, Kat and Chen, Kaifeng and Wang, Weiyi and Li, Zhe and Martins, Gus and Lee, Jinhyuk and Sherwood, Mark and Ji, Juyeong and Wu, Renjie and Zheng, Jingxiao and Singh, Jyotinder and Sharma, Abheesht and Sreepat, Divya and Jain, Aashi and Elarabawy, Adham and Co, AJ and Doumanoglou, Andreas and Samari, Babak and Hora, Ben and Potetz, Brian and Kim, Dahun and Alfonseca, Enrique and Moiseev, Fedor and Han, Feng and Palma Gomez, Frank and Hernández Ábrego, Gustavo and Zhang, Hesen and Hui, Hui and Han, Jay and Gill, Karan and Chen, Ke and Chen, Koert and Shanbhogue, Madhuri and Boratko, Michael and Suganthan, Paul and Duddu, Sai Meher Karthik and Mariserla, Sandeep and Ariafar, Setareh and Zhang, Shanfeng and Zhang, Shijie and Baumgartner, Simon and Goenka, Sonam and Qiu, Steve and Dabral, Tanmaya and Walker, Trevor and Rao, Vikram and Khawaja, Waleed and Zhou, Wenlei and Ren, Xiaoqi and Xia, Ye and Chen, Yichang and Chen, Yi-Ting and Dong, Zhe and Ding, Zhongli and Visin, Francesco and Liu, Gaël and Zhang, Jiageng and Kenealy, Kathleen and Casbon, Michelle and Kumar, Ravin and Mesnard, Thomas and Gleicher, Zach and Brick, Cormac and Lacombe, Olivier and Roberts, Adam and Sung, Yunhsuan and Hoffmann, Raphael and Warkentin, Tris and Joulin, Armand and Duerig, Tom and Seyedhosseini, Mojtaba},58    publisher={Google DeepMind},59    year={2025},60    journal={arXiv preprint arXiv:2509.20354}61}62```63 64### Usage65 66These model weights are designed to be used with [Sentence Transformers](https://www.SBERT.net), using the [Gemma 3](https://huggingface.co/docs/transformers/main/en/model_doc/gemma3) implementation from [Hugging Face Transformers](https://huggingface.co/docs/transformers/en/index) as the backbone.67 68First install the Sentence Transformers library:69 70```bash71pip install -U sentence-transformers72```73 74Then you can load this model and run inference.75 76```python77from sentence_transformers import SentenceTransformer78 79# Download from the 🤗 Hub80model = SentenceTransformer("google/embeddinggemma-300m")81 82# Run inference with queries and documents83query = "Which planet is known as the Red Planet?"84documents = [85    "Venus is often called Earth's twin because of its similar size and proximity.",86    "Mars, known for its reddish appearance, is often referred to as the Red Planet.",87    "Jupiter, the largest planet in our solar system, has a prominent red spot.",88    "Saturn, famous for its rings, is sometimes mistaken for the Red Planet."89]90query_embeddings = model.encode_query(query)91document_embeddings = model.encode_document(documents)92print(query_embeddings.shape, document_embeddings.shape)93# (768,) (4, 768)94 95# Compute similarities to determine a ranking96similarities = model.similarity(query_embeddings, document_embeddings)97print(similarities)98# tensor([[0.3011, 0.6359, 0.4930, 0.4889]])99```100 101**NOTE**: EmbeddingGemma activations do not support `float16`. Please use `float32` or `bfloat16` as appropriate for your hardware.102 103## Model Data104 105### Training Dataset106 107This model was trained on a dataset of text data that includes a wide variety of sources totaling approximately 320 billion tokens. Here are the key components:108 109-   **Web Documents**: A diverse collection of web text ensures the model is exposed to a broad range of linguistic styles, topics, and vocabulary. The training dataset includes content in over 100 languages.110-   **Code and Technical Documents**: Exposing the model to code and technical documentation helps it learn the structure and patterns of programming languages and specialized scientific content, which improves its understanding of code and technical questions.111-   **Synthetic and Task-Specific Data**: Synthetically training data helps to teach the model specific skills. This includes curated data for tasks like information retrieval, classification, and sentiment analysis, which helps to fine-tune its performance for common embedding applications.112 113The combination of these diverse data sources is crucial for training a powerful multilingual embedding model that can handle a wide variety of different tasks and data formats.114 115### Data Preprocessing116 117Here are the key data cleaning and filtering methods applied to the training data:118 119-   CSAM Filtering: Rigorous CSAM (Child Sexual Abuse Material) filtering was applied at multiple stages in the data preparation process to ensure the exclusion of harmful and illegal content.120-   Sensitive Data Filtering: As part of making Gemma pre-trained models safe and reliable, automated techniques were used to filter out certain personal information and other sensitive data from training sets.121-   Additional methods: Filtering based on content quality and safety in line with [our policies](https://ai.google/static/documents/ai-responsibility-update-published-february-2025.pdf).122 123## Model Development124 125### Hardware126 127EmbeddingGemma was trained using the latest generation of [Tensor Processing Unit (TPU)](https://cloud.google.com/tpu/docs/intro-to-tpu) hardware (TPUv5e), for more details refer to the [Gemma 3 model card](https://ai.google.dev/gemma/docs/core/model_card_3).128 129### Software130 131Training was done using [JAX](https://github.com/jax-ml/jax) and [ML Pathways](https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/). For more details refer to the [Gemma 3 model card](https://ai.google.dev/gemma/docs/core/model_card_3).132 133## Evaluation134 135### Benchmark Results136 137The model was evaluated against a large collection of different datasets and metrics to cover different aspects of text understanding.138 139#### Full Precision Checkpoint140 141<table>142  <thead>143    <tr>144      <th colspan="3"><strong>MTEB (Multilingual, v2)</strong></th>145    </tr>146  </thead>147  <tbody>148    <tr>149      <td><strong>Dimensionality</strong></td>150      <td><strong>Mean (Task)</strong></td>151      <td><strong>Mean (TaskType)</strong></td>152    </tr>153    <tr>154      <td>768d</td>155      <td>61.15</td>156      <td>54.31</td>157    </tr>158    <tr>159      <td>512d</td>160      <td>60.71</td>161      <td>53.89</td>162    </tr>163    <tr>164      <td>256d</td>165      <td>59.68</td>166      <td>53.01</td>167    </tr>168    <tr>169      <td>128d</td>170      <td>58.23</td>171      <td>51.77</td>172    </tr>173  </tbody>174</table>175 176<table>177  <thead>178    <tr>179      <th colspan="3"><strong>MTEB (English, v2)</strong></th>180    </tr>181  </thead>182  <tbody>183    <tr>184      <td><strong>Dimensionality</strong></td>185      <td><strong>Mean (Task)</strong></td>186      <td><strong>Mean (TaskType)</strong></td>187    </tr>188    <tr>189      <td>768d</td>190      <td>68.36</td>191      <td>64.15</td>192    </tr>193    <tr>194      <td>512d</td>195      <td>67.80</td>196      <td>63.59</td>197    </tr>198    <tr>199      <td>256d</td>200      <td>66.89</td>201      <td>62.94</td>202    </tr>203    <tr>204      <td>128d</td>205      <td>65.09</td>206      <td>61.56</td>207    </tr>208  </tbody>209</table>210 211<table>212  <thead>213    <tr>214      <th colspan="3"><strong>MTEB (Code, v1)</strong></th>215    </tr>216  </thead>217  <tbody>218    <tr>219      <td><strong>Dimensionality</strong></td>220      <td><strong>Mean (Task)</strong></td>221      <td><strong>Mean (TaskType)</strong></td>222    </tr>223    <tr>224      <td>768d</td>225      <td>68.76</td>226      <td>68.76</td>227    </tr>228    <tr>229      <td>512d</td>230      <td>68.48</td>231      <td>68.48</td>232    </tr>233    <tr>234      <td>256d</td>235      <td>66.74</td>236      <td>66.74</td>237    </tr>238    <tr>239      <td>128d</td>240      <td>62.96</td>241      <td>62.96</td>242    </tr>243  </tbody>244</table>245 246#### QAT Checkpoints247 248<table>249  <thead>250    <tr>251      <th colspan="3"><strong>MTEB (Multilingual, v2)</strong></th>252    </tr>253  </thead>254  <tbody>255    <tr>256      <td><strong>Quant config (dimensionality)</strong></td>257      <td><strong>Mean (Task)</strong></td>258      <td><strong>Mean (TaskType)</strong></td>259    </tr>260    <tr>261      <td>Q4_0 (768d)</td>262      <td>60.62</td>263      <td>53.61</td>264    </tr>265    <tr>266      <td>Q8_0 (768d)</td>267      <td>60.93</td>268      <td>53.95</td>269    </tr>270    <tr>271      <td>Mixed Precision* (768d)</td>272      <td>60.69</td>273      <td>53.82</td>274    </tr>275  </tbody>276</table>277 278<table>279  <thead>280    <tr>281      <th colspan="3"><strong>MTEB (English, v2)</strong></th>282    </tr>283  </thead>284  <tbody>285    <tr>286      <td><strong>Quant config (dimensionality)</strong></td>287      <td><strong>Mean (Task)</strong></td>288      <td><strong>Mean (TaskType)</strong></td>289    </tr>290    <tr>291      <td>Q4_0 (768d)</td>292      <td>67.91</td>293      <td>63.64</td>294    </tr>295    <tr>296      <td>Q8_0 (768d)</td>297      <td>68.13</td>298      <td>63.85</td>299    </tr>300    <tr>301      <td>Mixed Precision* (768d)</td>302      <td>67.95</td>303      <td>63.83</td>304    </tr>305  </tbody>306</table>307 308<table>309  <thead>310    <tr>311      <th colspan="3"><strong>MTEB (Code, v1)</strong></th>312    </tr>313  </thead>314  <tbody>315    <tr>316      <td><strong>Quant config (dimensionality)</strong></td>317      <td><strong>Mean (Task)</strong></td>318      <td><strong>Mean (TaskType)</strong></td>319    </tr>320    <tr>321      <td>Q4_0 (768d)</td>322      <td>67.99</td>323      <td>67.99</td>324    </tr>325    <tr>326      <td>Q8_0 (768d)</td>327      <td>68.70</td>328      <td>68.70</td>329    </tr>330    <tr>331      <td>Mixed Precision* (768d)</td>332      <td>68.03</td>333      <td>68.03</td>334    </tr>335  </tbody>336</table>337 338Note: QAT models are evaluated after quantization339 340\* Mixed Precision refers to per-channel quantization with int4 for embeddings, feedforward, and projection layers, and int8 for attention (e4_a8_f4_p4).341 342### Prompt Instructions343 344EmbeddingGemma can generate optimized embeddings for various use cases—such as document retrieval, question answering, and fact verification—or for specific input types—either a query or a document—using prompts that are prepended to the input strings.345Query prompts follow the form `task: {task description} | query: ` where the task description varies by the use case, with the default task description being `search result`. Document-style prompts follow the form `title: {title | "none"} | text: ` where the title is either `none` (the default) or the actual title of the document. Note that providing a title, if available, will improve model performance for document prompts but may require manual formatting.346 347Use the following prompts based on your use case and input data type. These may already be available in the EmbeddingGemma configuration in your modeling framework of choice.348 349<table>350  <thead>351    <tr>352      <th><br>353<strong>Use Case (task type enum)</strong></th>354      <th><br>355<strong>Descriptions</strong></th>356      <th><br>357<strong>Recommended Prompt</strong></th>358    </tr>359  </thead>360  <tbody>361    <tr>362      <td><br>363Retrieval (Query)</td>364      <td rowspan="4"><br>365Used to generate embeddings that are optimized for document search or information retrieval</td>366      <td><br>367task: search result | query: {content}</td>368    </tr>369    <tr>370      <td><br>371Retrieval (Document)</td>372      <td><br>373title: {title | "none"} | text: {content}</td>374    </tr>375    <tr>376      <td><br>377Question Answering</td>378      <td><br>379task: question answering | query: {content}</td>380    </tr>381    <tr>382      <td><br>383Fact Verification</td>384      <td><br>385task: fact checking | query: {content}</td>386    </tr>387    <tr>388      <td><br>389Classification</td>390      <td><br>391Used to generate embeddings that are optimized to classify texts according to preset labels</td>392      <td><br>393task: classification | query: {content}</td>394    </tr>395    <tr>396      <td><br>397Clustering</td>398      <td><br>399Used to generate embeddings that are optimized to cluster texts based on their similarities</td>400      <td><br>401task: clustering | query: {content}</td>402    </tr>403    <tr>404      <td><br>405Semantic Similarity</td>406      <td><br>407Used to generate embeddings that are optimized to assess text similarity. This is not intended for retrieval use cases.</td>408      <td><br>409task: sentence similarity | query: {content}</td>410    </tr>411    <tr>412      <td><br>413Code Retrieval</td>414      <td><br>415Used to retrieve a code block based on a natural language query, such as <em>sort an array</em> or <em>reverse a linked list</em>. Embeddings of the code blocks are computed using retrieval_document.</td>416      <td><br>417task: code retrieval | query: {content}</td>418    </tr>419  </tbody>420</table>421 422## Usage and Limitations423 424These models have certain limitations that users should be aware of.425 426### Intended Usage427 428Open embedding models have a wide range of applications across various industries and domains. The following list of potential uses is not comprehensive. The purpose of this list is to provide contextual information about the possible use-cases that the model creators considered as part of model training and development.429 430-   **Semantic Similarity**: Embeddings optimized to assess text similarity, such as recommendation systems and duplicate detection431-   **Classification**: Embeddings optimized to classify texts according to preset labels, such as sentiment analysis and spam detection432-  **Clustering**: Embeddings optimized to cluster texts based on their similarities, such as document organization, market research, and anomaly detection433-   **Retrieval**434    -   **Document**: Embeddings optimized for document search, such as indexing articles, books, or web pages for search435    -   **Query**: Embeddings optimized for general search queries, such as custom search436    -   **Code Query**: Embeddings optimized for retrieval of code blocks based on natural language queries, such as code suggestions and search437 438-   **Question Answering**: Embeddings for questions in a question-answering system, optimized for finding documents that answer the question, such as chatbox.439-   **Fact Verification**: Embeddings for statements that need to be verified, optimized for retrieving documents that contain evidence supporting or refuting the statement, such as automated fact-checking systems.440 441### Limitations442 443-   Training Data444    -   The quality and diversity of the training data significantly influence the model's capabilities. Biases or gaps in the training data can lead to limitations in the model's responses.445    -   The scope of the training dataset determines the subject areas the model can handle effectively.446 447-   Language Ambiguity and Nuance448    -   Natural language is inherently complex. Models might struggle to grasp subtle nuances, sarcasm, or figurative language.449 450### Ethical Considerations and Risks451 452Risks identified and mitigations:453 454-   **Perpetuation of biases**: It's encouraged to perform continuous monitoring (using evaluation metrics, human review) and the exploration of de-biasing techniques during model training, fine-tuning, and other use cases.455-   **Misuse for malicious purposes**: Technical limitations and developer and end-user education can help mitigate against malicious applications of embeddings. Educational resources and reporting mechanisms for users to flag misuse are provided. Prohibited uses of Gemma models are outlined in the [Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy).456-   **Privacy violations**: Models were trained on data filtered for removal of certain personal information and other sensitive data. Developers are encouraged to adhere to privacy regulations with privacy-preserving techniques.457 458### Benefits459 460At the time of release, this family of models provides high-performance open embedding model implementations designed from the ground up for responsible AI development compared to similarly sized models. Using the benchmark evaluation metrics described in this document, these models have shown superior performance to other, comparably-sized open model alternatives.