Praise2112/siren-screening-biencoder

SIREN Screening Bi-encoder

<p align="center"> <a href="https://huggingface.co/datasets/Praise2112/siren-screening"> <img src="https://img.shields.io/badge/Dataset-siren--screening-yellow.svg" alt="Dataset"/> </a> <a href="https://huggingface.co/Praise2112/siren-screening-crossencoder"> <img src="https://img.shields.io/badge/Reranker-siren--screening--crossencoder-blue.svg" alt="Cross-encoder"/> </a> <img src="https://img.shields.io/badge/License-Apache_2.0-green.svg" alt="License"/> </p>

A bi-encoder model for systematic review screening, trained to retrieve documents matching eligibility criteria. Achieves +24 percentage points MRR@10 over MedCPT on criteria-based retrieval.

Model Details

Intended Use

Primary use case: First-stage retrieval for systematic review screening pipelines.

Given eligibility criteria (e.g., "RCTs of aspirin in adults with diabetes, published after 2015"), retrieve candidate documents from a corpus for human review or downstream reranking.

Recommended pipeline:

1. 1.Encode eligibility criteria with this bi-encoder
2. 2.Retrieve top-k candidates via similarity search
3. 3.Rerank with siren-screening-crossencoder for 3-class classification

Usage

Sentence-Transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Praise2112/siren-screening-biencoder")

# Encode eligibility criteria (query)
query = "Randomized controlled trials of aspirin for cardiovascular prevention in diabetic adults"
query_embedding = model.encode(query)

# Encode candidate documents
documents = [
    "A randomized trial of low-dose aspirin in 5,000 diabetic patients showed reduced MI risk...",
    "This cohort study examined statin use in elderly populations with hyperlipidemia...",
]
doc_embeddings = model.encode(documents)

# Compute similarity
similarities = model.similarity(query_embedding, doc_embeddings)
print(similarities)  # tensor([[0.85, 0.42]])

Transformers

import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("Praise2112/siren-screening-biencoder")
model = AutoModel.from_pretrained("Praise2112/siren-screening-biencoder")

def encode(texts):
    inputs = tokenizer(texts, padding=True, truncation=True, max_length=768, return_tensors="pt")
    with torch.no_grad():
        outputs = model(**inputs)
    # Mean pooling
    attention_mask = inputs["attention_mask"]
    embeddings = outputs.last_hidden_state
    mask_expanded = attention_mask.unsqueeze(-1).expand(embeddings.size()).float()
    sum_embeddings = torch.sum(embeddings * mask_expanded, dim=1)
    sum_mask = torch.clamp(mask_expanded.sum(dim=1), min=1e-9)
    return F.normalize(sum_embeddings / sum_mask, p=2, dim=1)

query_emb = encode(["RCTs of aspirin in diabetic patients"])
doc_emb = encode(["A randomized trial of aspirin in 5,000 diabetic patients..."])
similarity = torch.mm(query_emb, doc_emb.T)

Performance

Internal Benchmark (SIREN Test Set)

This is +24 percentage points MRR@10 over MedCPT, the leading biomedical retrieval model.

External Benchmarks

Training

This model was created by:

1. 1.Fine-tuning gte-modernbert-base on the siren-screening dataset
2. 2.SLERP merging with the base model (t=0.4) to preserve out-of-distribution generalization

Training details:

• —Loss: Multiple Negatives Ranking Loss (MNRL) with in-batch negatives
• —Batch size: 512 (via GradCache)
• —Hard negatives: BM25-mined + LLM-generated
• —Matryoshka dimensions: [768, 512, 256, 128, 64]
• —Epochs: 1

Limitations

• —Synthetic queries, real documents: The queries and relevance labels are LLM-generated, but the documents are real PubMed articles
• —English only: Trained on English PubMed articles
• —Not a classifier: This model retrieves candidates; use the cross-encoder for relevance classification

Citation

@misc{oketola2026siren,
  title={SIREN: Improving Systematic Review Screening with Synthetic Training Data for Neural Retrievers},
  author={Praise Oketola},
  year={2026},
  howpublished={\url{https://huggingface.co/Praise2112/siren-screening-biencoder}},
  note={Bi-encoder model}
}

License

Apache 2.0