Datasetpublic

allenai/ms2_sparse_max

This is a copy of the MS^2 dataset, except the input source documents of its validation split have been replaced by a sparse retriever. The retrieval pipeline used: query: The background field of each example corpus: The union of all documents in the train, validation and test splits. A document is the concatenation of the title and abstract. retriever: BM25 via PyTerrier with default settings top-k strategy: "max", i.e. the number of documents retrieved, k, is set as the maximum number of… See the full description on the dataset page: https://huggingface.co/datasets/allenai/ms2_sparse_max.

sourceHugging Faceapache-2.0updated 4y agoView on Hugging Face

0likes61downloads

Dataset Card

This is a copy of the MS^2 dataset, except the input source documents of its validation split have been replaced by a _sparse_ retriever. The retrieval pipeline used:

—_query_: The background field of each example
—_corpus_: The union of all documents in the train, validation and test splits. A document is the concatenation of the title and abstract.
—_retriever_: BM25 via PyTerrier with default settings
—_top-k strategy_: "max", i.e. the number of documents retrieved, k, is set as the maximum number of documents seen across examples in this dataset, in this case k==25

Retrieval results on the train set:

Recall@100	Rprec	Precision@k	Recall@k
0.4333	0.2163	0.1746	0.2636

Retrieval results on the validation set:

Recall@100	Rprec	Precision@k	Recall@k
0.378	0.1827	0.1559	0.2188

Retrieval results on the test set:

Recall@100	Rprec	Precision@k	Recall@k
0.3928	0.1898	0.1672	0.2208