tss-deposium/bge-reranker-v2-m3-onnx-int8

bge-reranker-v2-m3 (ONNX INT8)

ONNX INT8 quantization of BAAI/bge-reranker-v2-m3 for browser & edge inference.

Files

Both files are byte-identical — same INT8 weights. Pick whichever matches your loader convention.

Quantization

• —Method: Optimum dynamic INT8 quantization
• —Source: BAAI/bge-reranker-v2-m3 (multilingual cross-encoder, XLM-RoBERTa-large, 568M params)
• —Size: ~570 MB (vs ~2.3 GB for the FP32 source)

Usage (Transformers.js)

import { AutoTokenizer, AutoModel } from '@huggingface/transformers';

const model = await AutoModel.from_pretrained(
  'tss-deposium/bge-reranker-v2-m3-onnx-int8',
  { dtype: 'int8' }  // or 'q8' for legacy file name
);

Notes

• —Cross-encoder: tokenize (query, doc) together, score from logits.
• —Multilingual (100+ langs); handles cross-language reranking better than bge-reranker-base.
• —For the smaller variant (279 MB, faster but weaker cross-lang), see `Xenova/bge-reranker-base`.