hugging-apps/cross-encoder-reranker-demo

Cross-Encoder Re-Ranking — Training Strategies Compared

A Gradio demo for the cross-encoders released with **Reproducing and Comparing Distillation Techniques for Cross-Encoders** (Morand et al., 2026). The paper fine-tunes 9 encoder backbones — from BERT, RoBERTa, ELECTRA and DeBERTa-v3 to ModernBERT-based Ettin models — with 6 training objectives (MarginMSE, InfoNCE, BCE, Hinge, DistillRankNET, ADR-MSE) and shows that objectives emphasizing relative comparisons (pairwise MarginMSE, listwise InfoNCE) consistently outperform pointwise baselines.

What the demo does

Enter a query and one candidate passage per line (as a first-stage retriever such as BM25, a bi-encoder or SPLADE would return them). The selected cross-encoder jointly encodes each (query, passage) pair and outputs a relevance logit; passages are returned sorted from most to least relevant.

The Advanced settings panel lets you switch between 9 of the 54 released checkpoints — one representative per (backbone, objective) cell of the paper's grid — so you can compare e.g. DeBERTa-v3 MarginMSE vs. InfoNCE vs. BCE, or the same MarginMSE objective across BERT-Base, RoBERTa, ELECTRA, Ettin-150M, MiniLM-L12 and DeBERTa-v3.

The default model is `xpmir/cross-encoder-DeBERTav3-MarginMSE`: the paper's strongest backbone (DeBERTa-v3-base, 184M params) trained with the pairwise MarginMSE objective on MS MARCO.

Models

All 54 checkpoints are in the 🤗 collection and share this exact usage:

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("xpmir/cross-encoder-DeBERTav3-MarginMSE")
model = AutoModelForSequenceClassification.from_pretrained("xpmir/cross-encoder-DeBERTav3-MarginMSE")

features = tokenizer(query, passage, padding=True, truncation=True, return_tensors="pt")
score = model(**features).logits  # relevance score: logits[:, 0]

Links

• —📄 Paper (arXiv:2603.03010)
• —💻 Code — xpmir/cross-encoders
• —🤗 Model collection — xpmir/reproducing-cross-encoders

Example data

The gr.Examples rows are real queries and passages from the MS MARCO Passage Ranking dev set — the dataset these cross-encoders were trained on — including each query's actual first-stage candidates and the passage judged relevant in the official dev qrels. MS MARCO is distributed by Microsoft under its data usage agreement.

Models are Apache-2.0 licensed.