Datasetpublic

ittia/wiki_dpr

The project contains indexes, datasets, checkpoints for RAG training and research. Sources checkpoint: ColBERTv2 (https://downloads.cs.stanford.edu/nlp/data/colbert/colbertv2/colbertv2.0.tar.gz) dataset: wiki_dpr (https://github.com/facebookresearch/DPR/blob/main/dpr/data/download_data.py) indexes (https://github.com/ittia-research/check/tree/main/datasets/wiki_dpr) indexing config: ColBERTConfig(nbits=2, doc_maxlen=220) Didn't compress the index files to one archive because… See the full description on the dataset page: https://huggingface.co/datasets/ittia/wiki_dpr.

sourceHugging Facecc-by-nc-4.0updated 2y agoView on Hugging Face

0likes232downloads

Dataset Card

The project contains indexes, datasets, checkpoints for RAG training and research.

Sources

—checkpoint: ColBERTv2 (https://downloads.cs.stanford.edu/nlp/data/colbert/colbertv2/colbertv2.0.tar.gz)
—dataset: wikidpr (https://github.com/facebookresearch/DPR/blob/main/dpr/data/downloaddata.py)
—indexes (https://github.com/ittia-research/check/tree/main/datasets/wiki_dpr)
—indexing config: ColBERTConfig(nbits=2, doc_maxlen=220)
—Didn't compress the index files to one archive because compress save less than 5% space and takes a long time.