CoolFace
Datasetpublic

allenai/cochrane_dense_oracle

This is a copy of the Cochrane dataset, except the input source documents of the train, validation, and test splits have been replaced by a dense retriever. query: The target field of each example corpus: The union of all documents in the train, validation and test splits. A document is the concatenation of the title and abstract. retriever: facebook/contriever-msmarco via PyTerrier with default settings top-k strategy: "oracle", i.e. the number of documents retrieved, k, is set as the… See the full description on the dataset page: https://huggingface.co/datasets/allenai/cochrane_dense_oracle.

sourceHugging Faceapache-2.0updated 4y agoView on Hugging Face
0likes26downloads
Dataset Card

This is a copy of the Cochrane dataset, except the input source documents of the train, validation, and test splits have been replaced by a _dense_ retriever.

  • —_query_: The target field of each example
  • —_corpus_: The union of all documents in the train, validation and test splits. A document is the concatenation of the title and abstract.
  • —_retriever_: `facebook/contriever-msmarco` via PyTerrier with default settings
  • —_top-k strategy_: "oracle", i.e. the number of documents retrieved, k, is set as the original number of input documents for each example

Retrieval results on the train set:

Recall@100RprecPrecision@kRecall@k
0.77900.44870.44870.4487

Retrieval results on the validation set:

Recall@100RprecPrecision@kRecall@k
0.78560.44240.44240.4424

Retrieval results on the test set:

N/A. Test set is blind so we do not have any queries.