xpmir/cross-encoder-ELECTRA-BCE
cross-encoder-ELECTRA-BCE
  
This model is a cross-encoder based on google/electra-base-discriminator. It was trained on Ms-Marco using loss bce as part of a reproducibility paper for training cross encoders: "[Reproducing and Comparing Distillation Techniques for Cross-Encoders](http://arxiv.org/abs/2603.03010)", see the paper for more details.
Contents
Model Description
This model is intended for re-ranking the top results returned by a retrieval system (like BM25, Bi-Encoders or SPLADE).
- Training Data: MS MARCO Passage
- Language: English
- Loss bce
Training can be easily reproduced using the assiciated repository. The exact training configuration used for this model is also detailed in config.yaml.
Usage
Quick Start:
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
tokenizer = AutoTokenizer.from_pretrained("xpmir/cross-encoder-ELECTRA-BCE")
model = AutoModelForSequenceClassification.from_pretrained("xpmir/cross-encoder-ELECTRA-BCE")
features = tokenizer("What is experimaestro ?", "Experimaestro is a powerful framework for ML experiments management...", padding=True, truncation=True, return_tensors="pt")
model.eval()
with torch.no_grad():
scores = model(**features).logits
print(scores)Evaluations
We provide evaluations of this cross-encoder re-ranking the top 1000 documents retrieved by naver/splade-v3-distilbert.