dbpedia_entity
contriever-gpl-dbpedia-entitybpr-gpl-dbpedia-entity-base-msmarco-distilbert-tas-bdbpedia-entity-tsdae-msmarco-distilbert-gpldbpedia-entity-distilbert-tas-b-gpl-self_minerdbpedia-entity-msmarco-distilbert-gpljpq-genq-dbpedia-entity-question_encoder-base-msmarco-distilbert-tas-bjpq-gpl-dbpedia-entity-question_encoder-base-msmarco-distilbert-tas-bjpq-gpl-dbpedia-entity-document_encoder-base-msmarco-distilbert-tas-b
dbpedia-entity
Dataset Card for BEIR Benchmark
dbpedia-entity is one of the datasets from the Entity Retrieval task within BEIR, measuring the retrieval of DbPedia articles for a given query entity.
Dataset Summary
BEIR is a heterogeneous benchmark built from 18 diverse datasets representing 9 information retrieval tasks.
Fact-checking: FEVER, Climate-FEVER, SciFact
Question-Answering: NQ, HotpotQA, FiQA-2018
Bio-Medical IR: TREC-COVID, BioASQ, NFCorpus
News Retrieval: TREC-NEWS… See the full description on the dataset page: https://huggingface.co/datasets/BeIR/dbpedia-entity.dbpedia-entity-decontaminated
dbpedia-entity (Decontaminated)
A decontaminated version of the dbpedia-entity dataset from the BEIR benchmark, with samples found in the mgte-en pre-training dataset removed.
Decontamination methodology
Contamination was detected using a two-pass approach against the full mgte-en dataset (484 GB, 1,235 parquet files):
Pass 1: Exact hash matching
All texts (queries and corpus documents) were normalized (lowercased, unicode NFKD, whitespace collapsed) and hashed… See the full description on the dataset page: https://huggingface.co/datasets/lightonai/dbpedia-entity-decontaminated.dbpedia-entity-qrels
Dataset Card for BEIR Benchmark
Dataset Summary
BEIR is a heterogeneous benchmark that has been built from 18 diverse datasets representing 9 information retrieval tasks:
Fact-checking: FEVER, Climate-FEVER, SciFact
Question-Answering: NQ, HotpotQA, FiQA-2018
Bio-Medical IR: TREC-COVID, BioASQ, NFCorpus
News Retrieval: TREC-NEWS, Robust04
Argument Retrieval: Touche-2020, ArguAna
Duplicate Question Retrieval: Quora, CqaDupstack
Citation-Prediction: SCIDOCS
Tweet… See the full description on the dataset page: https://huggingface.co/datasets/BeIR/dbpedia-entity-qrels.dbpedia-entity-generated-queries
Dataset Card for BEIR Benchmark
Dataset Summary
BEIR is a heterogeneous benchmark that has been built from 18 diverse datasets representing 9 information retrieval tasks:
Fact-checking: FEVER, Climate-FEVER, SciFact
Question-Answering: NQ, HotpotQA, FiQA-2018
Bio-Medical IR: TREC-COVID, BioASQ, NFCorpus
News Retrieval: TREC-NEWS, Robust04
Argument Retrieval: Touche-2020, ArguAna
Duplicate Question Retrieval: Quora, CqaDupstack
Citation-Prediction: SCIDOCS
Tweet… See the full description on the dataset page: https://huggingface.co/datasets/BeIR/dbpedia-entity-generated-queries.dbpedia-entity.retromae.flex
dbpedia-entity.retromae.flex
Description
RetroMAE index for DBPedia-Entity
Usage
# Load the artifact
import pyterrier_alpha as pta
artifact = pta.Artifact.from_hf('pyterrier/dbpedia-entity.retromae.flex')
artifact.np_retriever()
Benchmarks
dbpedia-entity/dev
name
nDCG@10
R@1000
np (flat)
0.4687
0.7372
dbpedia-entity/test
name
nDCG@10
R@1000
np (flat)
0.3729
0.678
Reproduction
import pyterrier as pt
from… See the full description on the dataset page: https://huggingface.co/datasets/pyterrier/dbpedia-entity.retromae.flex.BEIR-dbpedia-entity-interpret