datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
msmarco-qrels
Dataset Card for BEIR Benchmark
Dataset Summary
BEIR is a heterogeneous benchmark that has been built from 18 diverse datasets representing 9 information retrieval tasks:
Fact-checking: FEVER, Climate-FEVER, SciFact
Question-Answering: NQ, HotpotQA, FiQA-2018
Bio-Medical IR: TREC-COVID, BioASQ, NFCorpus
News Retrieval: TREC-NEWS, Robust04
Argument Retrieval: Touche-2020, ArguAna
Duplicate Question Retrieval: Quora, CqaDupstack
Citation-Prediction: SCIDOCS
Tweet… See the full description on the dataset page: https://huggingface.co/datasets/BeIR/msmarco-qrels.msmarco-yesnoms_marco_colbertv2
MS MARCO v1 Passage, ColBERTv2
Token-level (late-interaction) ColBERTv2 embeddings of the MS MARCO v1 passage collection and the dev/small queries.
Source
Collection: MS MARCO v1 passage (ir_datasets msmarco-passage), 8,841,823 passages
Queries: dev/small, 6,980 queries and 7,437 qrels (msmarco-passage/dev/small)
Document order: passage id order (row i is pid i)
Encoding
Model: ColBERTv2 (colbert-ir/colbertv2.0, BERT-base-uncased tokenizer)… See the full description on the dataset page: https://huggingface.co/datasets/tuskanny/ms_marco_colbertv2.ms_marco_cocondenserkannolo-msmarco-splademsmarco-pl-qrelsPart of BEIR-PL: Zero Shot Information Retrieval Benchmark for the Polish Language.
Link to arxiv: https://arxiv.org/pdf/2305.19840.pdf
Contact: konrad.wojtasik@pwr.edu.pl
msmarco-sub
MS MARCO Subset
Purpose
This subset provides a standardized benchmark for evaluating sparse model performance on MS MARCO data, with negative examples pooled using BM25 retrieval.
Creation
This subset was created using the make_beir_subset.py script with the following command:
python ./make_beir_subset.py --dataset msmarco --es-host http://localhost:9200 --force-reindex --split dev
Parameters Used
Dataset: msmarco
Split: dev (development set)
ES… See the full description on the dataset page: https://huggingface.co/datasets/freethenation/msmarco-sub.ms_marco_inferencelessInRanker-msmarcomsmarco_modifiedkannolo-msmarco-cocondenserCIIR_MSMARCO