Amdestya/ce-bm25cat-minilm-l12

ce-bm25cat-minilm-l12

MiniLM-L12 cross-encoder reranker trained under the configuration stated in §4 of Askari et al., *"Injecting the BM25 Score as Text Improves BERT-Based Re-rankers"*, ECIR 2023: cross-entropy loss with early stopping on validation nDCG@10.

Paired with `Amdestya/ce-cat-minilm-l12` — identical data, schedule and seed; the arms differ only in the input construction.

Input format

text_a = f"{query} [SEP] {bm25_score}"
text_b = passage
# tokenises to: [CLS] query [SEP] score [SEP] passage [SEP]

Paper Eq. 3 ordering: the score sits BETWEEN query and passage. This differs from Amdestya/bm25cat-minilm-l12, which follows the authors' released code and puts the score BEFORE the query. The two are not interchangeable.

Normalise the score the same way

bm25_score = int((raw_bm25 / 50) * 100)   # global Min-Max, fixed constants, then x100 as an integer

Min-Max in the global setting (fixed min=0, max=50, not per-query), scaled by 100 and truncated to an integer — paper §3.3 row (f). Training scores came from Anserini/Lucene BM25 with k1=0.82, b=0.68. Other BM25 implementations produce a different score distribution.

Usage

from sentence_transformers import CrossEncoder
model = CrossEncoder("Amdestya/ce-bm25cat-minilm-l12", max_length=256)

Outputs are raw logits (identity activation), unbounded, higher = more relevant.

Training

Built as a reference artifact for a dissertation study on reproducing IR papers with LLMs.