mlboydaisuke/Granite-Embedding-97M-Multilingual-R2-CoreAI

1---2language:3- ar4- az5- bg6- bn7- ca8- cs9- da10- de11- el12- en13- es14- et15- fa16- fi17- fr18- he19- hi20- hr21- hu22- id23- is24- it25- ja26- ka27- kk28- km29- ko30- lt31- lv32- mr33- ms34- nl35- 'no'36- pl37- pt38- ro39- ru40- sk41- sl42- sq43- sr44- sv45- sw46- te47- th48- tl49- tr50- uk51- ur52- uz53- vi54- zh55library_name: sentence-transformers56license: apache-2.057pipeline_tag: feature-extraction58tags:59- granite60- embeddings61- transformers62- multilingual63- mteb64- feature-extraction65- sentence-similarity66- onnx67- openvino68---69 70# Granite-Embedding-97M-Multilingual-R271 72**Model Summary:** Granite-Embedding-97M-Multilingual-R2 is a 97M parameter dense embedding model from the Granite Embeddings collection for high-quality multilingual text embeddings at minimal compute cost. It produces 384-dimensional vectors with a context length of up to 32,768 tokens. The model supports **200+ languages** (based on the multilingual pretraining corpus of the underlying encoder), with **enhanced support for 52 languages and programming code** that receive explicit retrieval-pair and cross-lingual training. All training data uses permissive, enterprise-friendly licenses, plus IBM-collected and IBM-generated datasets.73 74> Granite Embedding 97M Multilingual R2 scores **60.3** on [Multilingual MTEB Retrieval (18 tasks)](https://huggingface.co/spaces/mteb/leaderboard) — the highest retrieval score of any open multilingual embedding model under 100M parameters, outperforming the next-best model in its size class (multilingual-e5-small at 50.9) by **+9.4 points** — while being roughly **3× smaller** than the full-size granite-embedding-311m-multilingual-r2. The multilingual R2 model shows strong performance across multilingual information retrieval benchmarks, code retrieval, long-document search, conversational multi-turn, and reasoning retrieval tasks.75 76### What's New in R277 78- **Architecture upgrade:** ModernBERT replaces XLM-RoBERTa, bringing alternating attention, SiLU activations, and rotary position embeddings.79- **Extended context:** 32,768 tokens (up from 512 in R1), enabling long-document and multi-passage retrieval.80- **Compact multilingual vocabulary:** A purpose-trained 180K-token tokenizer preserves broad multilingual coverage while reducing model size.81- **Model pruning:** Layer pruning (22 → 12 layers) from the full-size multilingual model, followed by continued distillation training to recover quality.82- **Broader code coverage:** Code retrieval training set that includes Python, Go, Java, JavaScript, PHP, Ruby, SQL, C, C++.83- **Training advances:** Knowledge distillation from multiple teachers and contrastive fine-tuning yield a **+14.6 point** average gain over the previous-generation granite-embedding-107m-multilingual.84- **Deployment flexibility:** Released with ONNX and OpenVINO models; compatible with vLLM and llama.cpp (GGUF).85 86The model uses a bi-encoder architecture to generate high-quality embeddings from text inputs such as queries, passages, code, and documents, enabling seamless comparison through cosine similarity. Built using contrastive fine-tuning, knowledge distillation, model pruning, and vocabulary selection, granite-embedding-97m-multilingual-r2 is optimized to ensure strong alignment between query and passage embeddings across many languages while maintaining a compact model size.87 88The Granite Embedding Multilingual R2 release consists of two multilingual embedding models, both based on the ModernBERT architecture:89 90- _granite-embedding-311m-multilingual-r2_ (**311M** parameters): with an output embedding size of _768_, replacing _granite-embedding-278m-multilingual_. See [granite-embedding-311m-multilingual-r2](https://huggingface.co/ibm-granite/granite-embedding-311m-multilingual-r2).91- **_granite-embedding-97m-multilingual-r2_** (**97M** parameters): A reduced-size multilingual model built via layer pruning and vocabulary selection from the larger model, with fewer layers and a smaller output embedding size (_384_).92 93## Model Details94 95- **Developed by:** Granite Embedding Team, IBM96- **Repository:** [ibm-granite/granite-embedding-models](https://github.com/ibm-granite/granite-embedding-models)97- **Project Page:** [IBM Granite](https://www.ibm.com/granite)98- **Paper:** [Granite Embedding Multilingual R2 Models](https://huggingface.co/papers/2605.13521)99- **Language(s) (NLP):** 200+ languages supported, with enhanced support for 52 languages and programming code (see [full language list](#supported-languages))100- **Release Date**: April 29, 2026101- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)102 103### Supported Languages104 105The underlying encoder was pretrained on text from **200+ languages**, and we report general-purpose embeddings for any of them. In addition, we provide **enhanced support for 52 languages and programming code** that receive explicit retrieval-pair and cross-lingual training data, producing higher-quality embeddings on retrieval tasks.106 107<details>108<summary>Click to expand the list of 52 enhanced-support languages</summary>109 110Albanian (sq), Arabic (ar), Azerbaijani (az), Bengali (bn), Bulgarian (bg), Catalan (ca), Chinese (zh), Croatian (hr), Czech (cs), Danish (da), Dutch (nl), English (en), Estonian (et), Finnish (fi), French (fr), Georgian (ka), German (de), Greek (el), Hebrew (he), Hindi (hi), Hungarian (hu), Icelandic (is), Indonesian (id), Italian (it), Japanese (ja), Kazakh (kk), Khmer (km), Korean (ko), Latvian (lv), Lithuanian (lt), Malay (ms), Marathi (mr), Norwegian (no), Persian (fa), Polish (pl), Portuguese (pt), Romanian (ro), Russian (ru), Serbian (sr), Slovak (sk), Slovenian (sl), Spanish (es), Swahili (sw), Swedish (sv), Tagalog (tl), Telugu (te), Thai (th), Turkish (tr), Ukrainian (uk), Urdu (ur), Uzbek (uz), Vietnamese (vi).111 112Additionally, the models are trained on **programming code** (Python, Go, Java, JavaScript, PHP, Ruby, SQL, C, C++) and support cross-lingual code retrieval.113 114</details>115 116### When to Use This Model117 118- **Use granite-embedding-97m-multilingual-r2** for latency-sensitive production workloads, edge deployment, or when you need maximum encoding throughput with competitive multilingual quality. At 97M parameters, this model is 3× smaller than the full-size granite-embedding-311m-multilingual-r2 while preserving strong cross-lingual retrieval performance.119- **Use [granite-embedding-311m-multilingual-r2](https://huggingface.co/ibm-granite/granite-embedding-311m-multilingual-r2)** when accuracy is the top priority, and you can afford the throughput of a 311M-parameter model.120- **Use [granite-embedding-english-r2](https://huggingface.co/ibm-granite/granite-embedding-english-r2) or [granite-embedding-small-english-r2](https://huggingface.co/ibm-granite/granite-embedding-small-english-r2)** when your data is predominantly English, as these English-specific models offer optimized performance for monolingual English use cases.121 122## Usage123 124**Intended Use:** The model is designed to produce fixed-length vector representations for a given text, which can be used for text similarity, retrieval, and search applications across multiple languages.125 126For efficient inference, these models support Flash Attention 2. Installing it is optional but can lead to faster encoding:127 128```shell129pip install flash_attn130```131 132**Usage with Sentence Transformers:**133 134The model is compatible with the SentenceTransformer library and is very easy to use:135 136First, install the sentence transformers library137 138```shell139pip install sentence_transformers140```141 142The model can then be used to encode pairs of text and find the similarity between their representations143 144```python145from sentence_transformers import SentenceTransformer, util146 147model_path = "ibm-granite/granite-embedding-97m-multilingual-r2"148# Load the Sentence Transformer model149model = SentenceTransformer(model_path)150 151input_queries = [152    'What is the tallest mountain in Japan?',          # English query153    'Wer hat das Lied Achy Breaky Heart geschrieben?', # German query154    'ドイツの首都はどこですか？',                            # Japanese query155    ]156 157input_passages = [158    "富士山は、静岡県と山梨県にまたがる活火山で、標高3776.12 mで日本最高峰の独立峰である。",  # Japanese passage159    "Achy Breaky Heart is a country song written by Don Von Tress. Originally titled Don't Tell My Heart and performed by The Marcy Brothers in 1991.",  # English passage160    "Berlin ist die Hauptstadt und ein Land der Bundesrepublik Deutschland. Die Stadt ist with rund 3,7 Millionen Einwohnern die bevölkerungsreichste Kommune Deutschlands.",  # German passage161    ]162 163# Cross-lingual retrieval: each query should score highest with its matching passage in a different language164query_embeddings = model.encode(input_queries)165passage_embeddings = model.encode(input_passages)166 167# calculate cosine similarity — expect high scores on the diagonal (EN→JA, DE→EN, JA→DE)168print(util.cos_sim(query_embeddings, passage_embeddings))169# output: tensor([[0.8869, 0.6658, 0.7213],170#         [0.6792, 0.9577, 0.6420],171#        [0.7534, 0.6771, 0.9112]])172```173 174**Usage with Hugging Face Transformers:**175 176This is a simple example of how to use the granite-embedding-97m-multilingual-r2 model with the Transformers library and PyTorch. For a complete retrieval workflow including passage encoding and cosine similarity, see the Sentence Transformers example above.177 178First, install the required libraries179 180```shell181pip install transformers torch182```183 184The model can then be used to encode text185 186```python187import torch188from transformers import AutoModel, AutoTokenizer189 190model_path = "ibm-granite/granite-embedding-97m-multilingual-r2"191 192# Load the model and tokenizer193model = AutoModel.from_pretrained(model_path)194tokenizer = AutoTokenizer.from_pretrained(model_path)195model.eval()196 197input_queries = [198    'What is the tallest mountain in Japan?',          # English query199    'Wer hat das Lied Achy Breaky Heart geschrieben?', # German query200    'ドイツの首都はどこですか？',                            # Japanese query201    ]202 203# tokenize inputs204tokenized_queries = tokenizer(input_queries, padding=True, truncation=True, return_tensors='pt')205 206# encode queries207with torch.no_grad():208    model_output = model(**tokenized_queries)209    # Perform pooling. granite-embedding-97m-multilingual-r2 uses CLS Pooling210    query_embeddings = model_output[0][:, 0]211 212# normalize the embeddings213query_embeddings = torch.nn.functional.normalize(query_embeddings, dim=1)214```215 216### Optimized Inference and Deployment217 218**ONNX and OpenVINO:**219 220Pre-converted ONNX and OpenVINO models are released alongside the PyTorch weights for production deployment. These can be loaded directly via the `backend` parameter in Sentence Transformers:221 222```python223from sentence_transformers import SentenceTransformer224 225# ONNX backend226model = SentenceTransformer("ibm-granite/granite-embedding-97m-multilingual-r2", backend="onnx")227embeddings = model.encode(["example text"])228 229# OpenVINO backend230model = SentenceTransformer("ibm-granite/granite-embedding-97m-multilingual-r2", backend="openvino")231embeddings = model.encode(["example text"])232 233# OpenVINO INT8 quantized backend (smaller & faster on CPU)234model = SentenceTransformer(235    "ibm-granite/granite-embedding-97m-multilingual-r2",236    backend="openvino",237    model_kwargs={"file_name": "openvino/openvino_model_qint8_quantized.xml"},238)239embeddings = model.encode(["example text"])240```241 242The ONNX model is compatible with any ONNX Runtime backend (CPU, CUDA, TensorRT, DirectML). The OpenVINO model is optimized for Intel hardware including CPUs and integrated GPUs.243 244**vLLM:**245 246The model can be served as an embedding endpoint using [vLLM](https://docs.vllm.ai/):247 248```shell249vllm serve ibm-granite/granite-embedding-97m-multilingual-r2 --task embed250```251 252**llama.cpp (GGUF):**253 254The model can be converted to GGUF format for use with [llama.cpp](https://github.com/ggerganov/llama.cpp):255 256```shell257# Convert to GGUF258python convert_hf_to_gguf.py ibm-granite/granite-embedding-97m-multilingual-r2 \259    --outfile granite-embedding-97m-multilingual-r2.gguf260 261# Generate embeddings262llama-embedding -m granite-embedding-97m-multilingual-r2.gguf -p "example text"263```264 265Note: Ollama does not currently support ModernBERT-based models.266 267## Evaluation Results268 269Granite-Embedding-97M-Multilingual-R2 delivers strong retrieval quality at minimal compute cost. At 97M parameters and 384-dimensional embeddings, it offers a compelling accuracy-efficiency tradeoff: it matches the retrieval quality of gte-multilingual-base (a 305M model) at nearly 3× the encoding speed, and gains **+14.6 points** on average over its predecessor granite-embedding-107m-multilingual.270 271### Multilingual Retrieval Performance272 273Performance on Multilingual MTEB Retrieval, MTEB English Retrieval, MTEB Code Retrieval, long-document search (LongEmbed), and Reasoning as Retrieval (RaR-b) benchmarks. Scores are averages across tasks; higher is better. Throughput (documents per second) was measured on a single NVIDIA H100 GPU using a sliding window with 512-token chunks.274 275At nearly 2,900 documents per second, granite-embedding-97m-multilingual-r2 delivers comparable throughput to its R1 predecessor while gaining close to 10 points on multilingual retrieval. It retains the majority of the full-size 311M model's retrieval quality at roughly 3× smaller size and 1.5× higher throughput.276 277| Model                                     | Parameters (M) | Embedding Size | MTEB ML Retrieval (18) | MTEB Retrieval (eng, v2) (10) | MTEB (Code, v1) (12) | LongEmbed (6) | RaR-b (17) | **AVG**  | Throughput (docs/s) |278| ----------------------------------------- | -------------- | -------------- | ---------------------- | ----------------------------- | -------------------- | ------------- | ---------- | -------- | ------------------: |279| granite-embedding-107m-multilingual       | 107            | 384            | 48.1                   | 47.9                          | 40.7                 | 34.3          | 17.1       | 37.6     |               3,113 |280| granite-embedding-278m-multilingual       | 278            | 768            | 52.2                   | 51.5                          | 48.5                 | 37.7          | 18.9       | 41.8     |               2,164 |281| granite-embedding-311m-multilingual-r2    | 311            | 768            | 65.2                   | 52.6                          | 63.8                 | 71.7          | 28.0       | 56.3     |               1,828 |282| **granite-embedding-97m-multilingual-r2** | **97**         | **384**        | **60.3**               | **50.1**                      | **60.4**             | **65.5**      | **24.9**   | **52.2** |           **2,534** |283 284## Model Architecture and Key Features285 286The Granite Embedding Multilingual R2 release consists of two multilingual embedding models, both based on the ModernBERT architecture. The 97M model is derived from the 311M model via layer pruning (22 → 12 layers) and vocabulary selection (262K → 180K tokens), using a compact tokenizer purpose-trained for multilingual coverage at reduced size. The 97M model uses SiLU activation (from the pruned architecture) rather than GeGLU used in the full-size model.287 288| Feature                   | **granite-embedding-97m-multilingual-r2** | granite-embedding-311m-multilingual-r2 |289| :------------------------ | :---------------------------------------: | :------------------------------------: |290| Embedding size            |                  **384**                  |                  768                   |291| Number of layers          |                  **12**                   |                   22                   |292| Number of attention heads |                  **12**                   |                   12                   |293| Intermediate size         |                 **1536**                  |                  1152                  |294| Activation Function       |                 **SiLU**                  |                 GeGLU                  |295| Vocabulary Size           |                **180,000**                |                262,152                 |296| Max. Sequence Length      |                **32,768**                 |                 32,768                 |297| Matryoshka Dimensions     |                     —                     |        768, 512, 384, 256, 128         |298| # Parameters              |                 **~97M**                  |                 ~311M                  |299 300## Training and Optimization301 302The Granite Embedding Multilingual R2 models incorporate key enhancements from the ModernBERT architecture, including:303 304- Alternating attention lengths to accelerate processing305- Rotary position embeddings for extended sequence length306- A multilingual tokenizer trained on code and text data across 200+ languages307- Flash Attention 2.0 for improved efficiency308- Streamlined parameters, eliminating unnecessary bias terms309 310The 97M model was built via **layer pruning and vocabulary selection**, starting from the larger granite-embedding-311m-multilingual-r2 and reducing from 22 to 12 transformer layers. A compact, purpose-trained multilingual tokenizer (180K vocabulary) preserves broad multilingual coverage while significantly reducing parameter count. The pruned model was then trained using knowledge distillation with multiple teacher models and contrastive fine-tuning to recover retrieval quality, retaining the majority of the full-size model's performance at 3× smaller size.311 312## Data Collection313 314All training data is sourced under permissive, commercial-friendly licenses, making Granite Embedding R2 suitable for unrestricted enterprise deployment.315 316Training data comes from four key sources:317 3181. Unsupervised title-body paired data scraped from the web3192. Publicly available paired data with permissive, enterprise-friendly licenses3203. IBM-internal paired data targeting specific technical domains3214. IBM-generated multilingual synthetic data including long-document pairs322 323For governance, all our data undergoes a data clearance process subject to technical, business, and governance review. This comprehensive process captures critical information about the data, including but not limited to their content description, ownership, intended use, data classification, licensing information, usage restrictions, how the data will be acquired, as well as an assessment of sensitive information (e.g., personal information).324 325## Infrastructure326 327We trained the Granite Embedding Multilingual R2 models using IBM's computing cluster, BlueVela Cluster, which is outfitted with NVIDIA H100 80GB GPUs. This cluster provides a scalable and efficient infrastructure for training our models over multiple GPUs.328 329## Ethical Considerations and Limitations330 331Granite Embedding 97M Multilingual R2 leverages both permissively licensed open-source and select proprietary data for enhanced performance. The training data for the base language model was filtered to remove text containing hate, abuse, and profanity, though the effectiveness of such filtering may vary across language families.332 333As a pruned model, granite-embedding-97m-multilingual-r2 trades some accuracy for significantly faster inference and lower resource requirements. Performance varies across languages: higher-resource languages and those in the 52-language enhanced-support set generally achieve better results, while low-resource languages rely on cross-lingual transfer and may exhibit lower retrieval quality — an effect that is more pronounced in this smaller model than in the full-size 311M variant. The reduced vocabulary (180K vs. 262K tokens) may also affect tokenization efficiency for some languages. Longer texts will be truncated to the 32,768-token context limit.334 335## Resources336 337- Learn about the latest updates with Granite: https://www.ibm.com/granite338- Get started with tutorials, best practices, and prompt engineering advice: https://www.ibm.com/granite/docs/339- Learn about the latest Granite learning resources: https://ibm.biz/granite-learning-resources340 341## Citation342 343```344@misc{awasthy2026graniteembeddingmultilingualr2,345      title={Granite Embedding Multilingual R2 Models}, 346      author={Parul Awasthy and Aashka Trivedi and Yushu Yang and Ken Barker and Yulong Li and Bhavani Iyer and Martin Franz and Juergen Bross and Meet Doshi and Vignesh P and Vishwajeet Kumar and Todd Ward and Abraham Daniels and Madison Lee and Luis Lastras and Jaydeep Sen and Radu Florian},347      year={2026},348      eprint={2605.13521},349      archivePrefix={arXiv},350      primaryClass={cs.IR},351      url={https://arxiv.org/abs/2605.13521}, 352}353```