Datasetpublic

allenai/cochrane_dense_mean

This is a copy of the Cochrane dataset, except the input source documents of its train, validation and test splits have been replaced by a dense retriever. The retrieval pipeline used: query: The target field of each example corpus: The union of all documents in the train, validation and test splits. A document is the concatenation of the title and abstract. retriever: facebook/contriever-msmarco via PyTerrier with default settings top-k strategy: "max", i.e. the number of documents retrieved… See the full description on the dataset page: https://huggingface.co/datasets/allenai/cochrane_dense_mean.

sourceHugging Faceapache-2.0updated 4y agoView on Hugging Face

0likes47downloads

Dataset Card

This is a copy of the Cochrane dataset, except the input source documents of its train, validation and test splits have been replaced by a _dense_ retriever. The retrieval pipeline used:

—_query_: The target field of each example
—_corpus_: The union of all documents in the train, validation and test splits. A document is the concatenation of the title and abstract.
—_retriever_: `facebook/contriever-msmarco` via PyTerrier with default settings
—_top-k strategy_: "max", i.e. the number of documents retrieved, k, is set as the maximum number of documents seen across examples in this dataset, in this case k==9

Retrieval results on the train set:

Recall@100	Rprec	Precision@k	Recall@k
0.7790	0.4487	0.3438	0.4800

Retrieval results on the validation set:

Recall@100	Rprec	Precision@k	Recall@k
0.7856	0.4424	0.3534	0.4913

Retrieval results on the test set:

N/A. Test set is blind so we do not have any queries.