Datasetpublic

hf-internal-testing/wiki_dpr_dummy

This dummy dataset is used for testing purpose for rag model in transformers. It is proudced via the following steps: dataset = datasets.load_dataset("wiki_dpr", with_embeddings=True, with_index=True, index_name="exact", embeddings_name="nq", dummy=True, revision=None) dataset["train"].drop_index("embeddings") dataset.push_to_hub("hf-internal-testing/wiki_dpr_dummy", token="...") The index file `index.faiss` (after being renamed locally) is then uploaded manually.

sourceHugging Faceupdated 1y agoView on Hugging Face

0likes2.1kdownloads

Dataset Card

This dummy dataset is used for testing purpose for rag model in transformers. It is proudced via the following steps:

bash

dataset = datasets.load_dataset("wiki_dpr", with_embeddings=True, with_index=True, index_name="exact", embeddings_name="nq", dummy=True, revision=None)
dataset["train"].drop_index("embeddings")
dataset.push_to_hub("hf-internal-testing/wiki_dpr_dummy", token="...")

The index file `index.faiss` (after being renamed locally) is then uploaded manually.