Datasetpublic

jpq-repro/nq-train__tct_colbertfaiss2opqM96_nbits8ps159744neg200ibnlr__

nq-train__tct_colbert__faiss2opq__M96_nbits8__ps159744__neg200__ibn__lr__ Description This is the PyTerrier JPQIndex for the Wikipedia 2018 corpus used by Natural Questions (NQ). Usage # Load the artifact import pyterrier as pt import pyterrier_dr index = pt.Artifact.from_hf('jpq-repro/nq-train__tct_colbert__faiss2opq__M96_nbits8__ps159744__neg200__ibn__lr__') model = pyterrier_dr.TctColBert.hnp() model.model =… See the full description on the dataset page: https://huggingface.co/datasets/jpq-repro/nq-train__tct_colbert__faiss2opq__M96_nbits8__ps159744__neg200__ibn__lr__.

sourceHugging Faceupdated 2mo agoView on Hugging Face

1likes25downloads

Dataset Card

nq-train_tctcolbert_faiss2opqM96nbits8_ps159744neg200ibnlr_

Description

This is the PyTerrier JPQIndex for the Wikipedia 2018 corpus used by Natural Questions (NQ).

Usage

python

# Load the artifact
import pyterrier as pt
import pyterrier_dr
index = pt.Artifact.from_hf('jpq-repro/nq-train__tct_colbert__faiss2opq__M96_nbits8__ps159744__neg200__ibn__lr__')
model = pyterrier_dr.TctColBert.hnp()
model.model = model.model.from_pretrained(index.path)
retr_pipe = model >> index.retriever_pq()
retr_pipe.search("who got the first nobel prize in physics")

Metadata

{
  "type": "dense_index",
  "format": "jpq",
  "M": 96,
  "Ks": 256,
  "dsub": 8,
  "doc_count": 21015324,
  "opq": true
}