llm-semantic-router/Vela-1.0-Encoder-307M-Reranker
<div align="center"> <img src="https://vllm-sr.ai/img/vllm-sr-logo.social.png" alt="vLLM Semantic Router" width="560" /> <p> <a href="https://vllm-sr.ai/"><strong>Docs</strong></a> | <a href="https://vllm-sr.ai/blog/"><strong>Blog</strong></a> | <a href="https://vllm-dev.slack.com/archives/C09CTGF8KCN"><strong>Slack</strong></a> | <a href="https://github.com/vllm-project/semantic-router"><strong>GitHub</strong></a> </p> </div>
Vela Reranker
Bring relevant context to the top. Vela Reranker scores passages for retrieval, RAG, and context selection in your router.
307M encoder · 32K context · Multilingual
Evaluation
Compared with the original mmBERT Reranker. Scores are ×100; higher is better.
Matched development subsets with identical candidate pools and complete inputs, using 22 layers, 768 dimensions, and FP32. MIRACL covers Arabic, Spanish, Japanese, and Chinese. The context set tests 24 queries across four lengths and three positions; its views are not independent examples.
Selected instruction-retrieval tasks from MTEB 2.20.12:
These results describe the listed tasks and development subsets, not a full MTEB score or ranking.
Quick start
Install torch, transformers, and safetensors. Use a ROCm-enabled PyTorch build for AMD GPUs.
import torch
from transformers import AutoModel, AutoTokenizer
model_id = "llm-semantic-router/Vela-1.0-Encoder-307M-Reranker"
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModel.from_pretrained(model_id, trust_remote_code=True).to(device).eval()
query = "How do I reset my password?"
passages = [
"Select Forgot password on the sign-in page to receive a reset link.",
"Your monthly invoice is available in the billing dashboard.",
]
inputs = tokenizer(
[query] * len(passages), passages,
padding=True, truncation=False, return_tensors="pt",
).to(device)
with torch.inference_mode():
scores = model(**inputs).logits.flatten().tolist()
print(sorted(zip(scores, passages), reverse=True))Higher scores mean greater relevance. The default uses all 22 layers and 768 dimensions. The 32,768-token limit includes the query, passage, and special tokens.