anonymousauthor01/emnlp-2026-ifr-train-val-set
EMNLP 2026 Findings: On-Policy Distillation Meets Off-Policy GRPO — Training Compact Instruction-Following Rerankers Train and validation splits used to train and evaluate the instruction-following reranker (IFR) in an in-distribution setting. The benchmark is an aggregated compilation of eight public instruction-following retrieval datasets spanning web search, code, mathematics, news, and multi-hop retrieval. Composition Source dataset Train Validation… See the full description on the dataset page: https://huggingface.co/datasets/anonymousauthor01/emnlp-2026-ifr-train-val-set.
EMNLP 2026 Findings: On-Policy Distillation Meets Off-Policy GRPO — Training Compact Instruction-Following Rerankers
Train and validation splits used to train and evaluate the instruction-following reranker (IFR) in an in-distribution setting. The benchmark is an aggregated compilation of eight public instruction-following retrieval datasets spanning web search, code, mathematics, news, and multi-hop retrieval.
Composition
Format and construction
- Each example is a (query, instruction, candidate pool) triple with relevance labels.
- Candidate pools use benchmark-provided pools where available, and BM25-constructed pools otherwise.
- Pools were built once during preprocessing, frozen, and reused identically across all systems and ablations, so first-stage retrieval variation does not confound reranker comparisons.
- Typical pool size is 3–18 documents per query (10th–90th percentile); mean 7.5, median 3. Full observed range: 2–21 (train), 2–20 (validation).
- Model inputs are truncated to 512 tokens.