Modelpublic

rolf-mozilla/granite-embedding-97m-multilingual-r2-q8

sourceHugging Faceapache-2.0updated 13d agoView on Hugging Face

0likes29downloads

Model Card

granite-embedding-97m-multilingual-r2 — q8 ONNX, renamed for transformers.js

Experimental artifact for Mozilla Smart Window embedding evaluation.

This is IBM's own int8 ONNX export of `ibm-granite/granite-embedding-97m-multilingual-r2`, unmodified, republished with the filename the Mozilla inference engine (transformers.js) resolves by default.

upstream path	path here
`onnx/model_quint8_avx2.onnx`	`onnx/model_quantized.onnx`

The weights are byte-identical to upstream — only the filename differs. Despite the avx2 in the original name it is a portable uint8 ONNX graph; the evaluation below was run on Apple Silicon.

Pooling: CLS, not mean

granite-embedding-*-r2 uses CLS pooling (see the upstream 1_Pooling/config.json, mirrored here). Mean-pooling this model produces noticeably wrong vectors — on a sample of EN/FR texts, mean pooling agrees with the reference fp32 embeddings at only ~0.85 cosine, versus ~0.97 for CLS.

import { pipeline } from '@huggingface/transformers';
const extractor = await pipeline('feature-extraction', 'rolf-mozilla/granite-embedding-97m-multilingual-r2-q8');
const out = await extractor(['bonjour le monde'], { pooling: 'cls', normalize: true });

No query or passage prefix is required.

Accuracy

Per-query retrieval on memory_usage_eval (500 queries, each with 11-21 candidate memories, 2-4 gold; macro-averaged; CLS pooling + L2 normalization), against the models Firefox ships or has evaluated:

Model	Precision	EN nDCG@5	EN MRR@10	FR nDCG@5	FR MRR@10
granite-97m-r2 (this repo)	q8	0.7779	0.8952	0.7236	0.8721
granite-97m-r2	fp32	0.8180	0.9277	0.7613	0.8977
all-MiniLM-L6-v2	q8	0.8077	0.9160	0.5591	0.7444
multilingual-e5-small	q8	0.7188	0.8665	0.5599	0.7431
potion-multilingual-128M	full d384	0.6495	0.8230	0.6275	0.8008

French is where it wins: +0.164 nDCG@5 (+29% relative) over multilingual-e5-small q8, and +0.096 over the previous best French model. It is the only model evaluated that is strong in both languages rather than trading one against the other; on English it trails all-MiniLM-L6-v2 q8 by 0.030.

Note the quantization cost is unusually high for this model: fp32 to q8 loses 0.040 EN / 0.038 FR nDCG@5, where multilingual-e5-small loses 0.005. A better int8 recipe than the shipped one is likely worth ~0.04 nDCG@5.

Sequence length was capped at 512 for parity with the other models; the architecture supports 32k.

Provenance

Republished by scripts/publish_hf_models.py in Mozilla's multilingual_embeddings_eval bundle. Weights and license (Apache-2.0) are IBM's. Temporary evaluation artifact — expect it to be deleted.