szdr/ruri-v3-reranker-310m-onnx_int8_arm64

ruri-v3-reranker-310m-onnxint8arm64 (unofficial ONNX conversion)

This is an unofficial, community-produced ONNX conversion of `cl-nagoya/ruri-v3-reranker-310m`.

This repository is not published, reviewed, or endorsed by the original authors. All credit for the model architecture, training data, and weights belongs to the original authors (Hayato Tsukagoshi, Ryohei Sasano, and the cl-nagoya lab). Please refer to the original model card for the full description, intended use, and evaluation results (e.g. JQaRA).

What was changed

• —The PyTorch weights were exported to ONNX via torch.onnx.export (legacy TorchScript-based exporter, opset 14, attn_implementation="eager" to avoid FlashAttention-specific control flow).
• —No weights were modified. Verified against the original PyTorch model on sample inputs (max logit difference ~1e-5).
• —Tokenizer and config files are copied unchanged from the original repository.

Usage

import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer
from huggingface_hub import hf_hub_download

repo_id = "szdr/ruri-v3-reranker-310m-onnx_int8_arm64"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
onnx_path = hf_hub_download(repo_id, "model.onnx")
session = ort.InferenceSession(onnx_path)

inputs = tokenizer(["query"], ["document"], return_tensors="np", padding=True, truncation=True)
logits = session.run(None, {
    "input_ids": inputs["input_ids"].astype(np.int64),
    "attention_mask": inputs["attention_mask"].astype(np.int64),
})[0]
score = 1 / (1 + np.exp(-logits))

License

Apache License 2.0, identical to the original model. See LICENSE.

Citation

Please cite the original work (not this conversion):

@misc{Ruri,
  title={Ruri: Japanese General Text Embeddings},
  author={Hayato Tsukagoshi and Ryohei Sasano},
  year={2024},
  eprint={2409.07737},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2409.07737},
}