datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
dpr_raw
"definite_pronoun_resolution" (dpr)
Dataset Summary
Composed by 30 students from one of the author's undergraduate classes. These
sentence pairs cover topics ranging from real events (e.g., Iran's plan to
attack the Saudi ambassador to the U.S.) to events/characters in movies (e.g.,
Batman) and purely imaginary situations, largely reflecting the pop culture as
perceived by the American kids born in the early 90s. Each annotated example
spans four lines: the first line… See the full description on the dataset page: https://huggingface.co/datasets/coref-data/dpr_raw.wiki_dpr_dummyThis dummy dataset is used for testing purpose for rag model in transformers. It is proudced via the following steps:
dataset = datasets.load_dataset("wiki_dpr", with_embeddings=True, with_index=True, index_name="exact", embeddings_name="nq", dummy=True, revision=None)
dataset["train"].drop_index("embeddings")
dataset.push_to_hub("hf-internal-testing/wiki_dpr_dummy", token="...")
The index file `index.faiss` (after being renamed locally) is then uploaded manually.
wiki_dpr_e5wiki_dpr encoded with intfloat/e5-base-v2
wikipedia-20240901-dprnq_corpus_dpr
Dataset Card for "nq_corpus_dpr"
More Information needed
wiki_dpr_token
Dataset Card for "wiki_dpr_token"
Distribution
[
{ // Token length
'~128': 2625007,
'128~256': 18370607,
'256~512': 19066,
'512~1024': 571,
'1024~2048': 47,
'2048~4096': 2,
'4096~8192': 0,
'8192~16384': 0,
'16384~32768': 0,
'32768~65536': 0,
'65536~128000': 0,
'128000~': 0,
},
{ // Text length
'~512': 86519,
'512~1024': 20927180,
'1024~2048': 1557,
'2048~4096': 43,
'4096~8192': 1,
'8192~16384':… See the full description on the dataset page: https://huggingface.co/datasets/seonglae/wiki_dpr_token.dpr_indiscrimThis dataset was generated by reformatting coref-data/dpr_raw into the indiscrim coreference format. See that repo for dataset details.
See ianporada/coref-data for additional conversion details and the conversion script.
Please create an issue in the repo above or in this dataset repo for any questions.
DPriv-Bench
DPrivBench: Benchmarking LLMs’ Reasoning for Differential Privacy
DPrivBench is a benchmark for evaluating whether language models can correctly reason about and verify claimed differential privacy (DP) guarantees from natural-language/LaTeX-format problem statements.
This release contains evaluation data from seven benchmark configs, along with one auxiliary function bank:
Category 1: 6 fundamental mechanism tracks, each with 98 questions.
Category 2: 125 more advanced… See the full description on the dataset page: https://huggingface.co/datasets/erchiw/DPriv-Bench.train-dpr-wikipedia
DPRWikipedia — Training, unified schema
A normalised copy of the dataset behind the mteb task DPRWikipedia, a retrieval training set built from Tevatron/wikipedia-nq-corpus. Same queries, documents
and relevance judgements as the benchmark evaluates — reshaped into one strict schema shared by every dataset
in this collection.
Source
Tevatron/wikipedia-nq-corpus @ 56c6e2438c13 (the revision pinned in mteb)
Domain · languages
Wikipedia QA (DPR) · eng
Queries /… See the full description on the dataset page: https://huggingface.co/datasets/Hyukkyu/train-dpr-wikipedia.telco-dpr-rag
Telco-DPR RAG
Dataset for Retrieval-Augmented Generation (RAG) based on Telco-DPR.
Structure
Subset
Splits
Description
corpus
train (default)
3GPP technical passages (text + tables) shared across all query splits
queries
train, dev, test
Synthetic telecom QA questions
qrels
train, dev, test
Relevance judgments (query ↔ passage)
answers
train, dev, test
Reference answers
Dataset statistics
Split
Queries
Corpus
train… See the full description on the dataset page: https://huggingface.co/datasets/DinoStackAI/telco-dpr-rag.asqa_dpr_wiki-text-6-3-tamber_top100convmix_dpr_wiki-text-6-3-tamber_top100Telco-DPR
Dataset Information
This paper proposes a Question-Answering (QA) system for the telecom domain using 3rd Generation Partnership Project (3GPP) technical documents.
Alongside, a hybrid dataset, Telco-DPR, which consists of a curated 3GPP corpus in a hybrid format, combining text and tables, is presented.
Additionally, the dataset includes a set of synthetic question/answer pairs designed to evaluate the retrieval performance of QA systems on this type of data.
The retrieval… See the full description on the dataset page: https://huggingface.co/datasets/thainasaraiva/Telco-DPR.Telco-DPR
Dataset Information
This paper proposes a Question-Answering (QA) system for the telecom domain using 3rd Generation Partnership Project (3GPP) technical documents.
Alongside, a hybrid dataset, Telco-DPR, which consists of a curated 3GPP corpus in a hybrid format, combining text and tables, is presented.
Additionally, the dataset includes a set of synthetic question/answer pairs designed to evaluate the retrieval performance of QA systems on this type of data.
The retrieval… See the full description on the dataset page: https://huggingface.co/datasets/ismailduru/Telco-DPR.nq_reranking_dprnqqampari_reranking_dprnqDPR_Pm3B_U10_beta0.10g0.30gamma0.30dpr-nq-readereval_DPRecordTestThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v2.1",
"robot_type": "so100",
"total_episodes": 1,
"total_frames": 224,
"total_tasks": 1,
"total_videos": 2,
"total_chunks": 1,
"chunks_size": 1000,
"fps": 30,
"splits": {
"train": "0:1"
},
"data_path": "data/chunk-{episode_chunk:03d}/episode_{episode_index:06d}.parquet",
"video_path":… See the full description on the dataset page: https://huggingface.co/datasets/shylee/eval_DPRecordTest.dpr-nq-reader-v2asqa_dpr_pyserini_top20_with_texttrec-news__gtr_base__dprclap_nq_dpr_pyserini_top100_with_textincontext_nq_v2_dpr_ctxsnq-hard-negative-dpr-sampled-seed-42-sample-4DPR_Q0.5B_U10_beta0.10g0.30gamma0.30convmix_dpr_pyserini_top1000_sentlevel_withtextgpdr-dpr-dataset
Dataset Card for "gpdr-dpr-dataset"
More Information needed
bioasq__gtr_base__dprwikitext-2-raw-v1-preprocessed-200-PP_RIP_PTP_-wikipedia-dpr-k-1-OP-False-train-perplexity