Datasetpublic

DinoStackAI/qasper-rag

QASPER RAG Dataset for Retrieval-Augmented Generation (RAG) based on QASPER. Structure Subset Splits Description corpus train (default) Paper chunks (abstract + full-text paragraphs) shared across all query splits queries train, dev, test Information-seeking questions over scientific papers qrels train, dev, test Relevance judgments (query ↔ paragraph chunk) answers train, dev, test Reference answers (longest valid free-form answer) top_ranked… See the full description on the dataset page: https://huggingface.co/datasets/DinoStackAI/qasper-rag.

sourceHugging Facecc-by-4.0updated 3mo agoView on Hugging Face

0likes125downloads

Dataset Card

QASPER RAG

Dataset for Retrieval-Augmented Generation (RAG) based on QASPER.

Structure

Subset	Splits	Description
`corpus`	train (default)	Paper chunks (abstract + full-text paragraphs) shared across all query splits
`queries`	train, dev, test	Information-seeking questions over scientific papers
`qrels`	train, dev, test	Relevance judgments (query ↔ paragraph chunk)
`answers`	train, dev, test	Reference answers (longest valid free-form answer)
`top_ranked`	train, dev, test	Paper-scoped candidate pool (all chunks of the query paper)
`retrieved_docs`	train, dev, test	Top-k retrieval results with relevance labels

Dataset statistics

Split	Queries	Corpus
train	2101	81550
dev	890	81550
test	1310	81550

The corpus is shared across all splits and contains paragraph-level chunks from the abstract and full text of each paper.

—Dev split: mapped from the original validation split
—Corpus source: unique papers from train, validation and test splits
—Chunking: one chunk for the abstract (section_name: abstract) and one chunk per paragraph in full_text

Source

Component	QASPER resource
Train	`train` split from allenai/qasper
Dev	`validation` split
Test	`test` split
Corpus	`abstract` + `full_text.paragraphs`
Queries	`qas.question`
Qrels	`qas.answers[].answer[].evidence` matched to corpus chunks
Top ranked	All paragraph chunks from the query paper (paper-scoped retrieval pool)
Answers	Longest valid answer per question (`free_form_answer`, joined `extractive_spans`, or `Yes`/`No`)

Filtering

Questions are kept only when at least one answer satisfies all of the following:

—unanswerable is false
—the answer has free_form_answer, non-empty extractive_spans, or yes_no
—after removing evidence items containing FLOAT SELECTED, at least one answer still has evidence that matches the corpus

Evidence items containing FLOAT SELECTED are removed individually. Questions are omitted only when no valid answer has remaining evidence.

Schema

corpus

json

{"id": "...", "title": "...", "section_name": "...", "text": "..."}

queries

json

{"id": "...", "text": "..."}

qrels

json

{"query_id": "...", "corpus_id": "...", "score": 1}

answers

json

{"query_id": "...", "answer": "..."}

top_ranked

json

{"query-id": "...", "corpus-ids": ["paper_00000", "paper_00001"]}

retrieved_docs

json

{"query_id": "...", "corpus_id": "...", "rank": 1, "retrieval_score": 0.92, "is_relevant": true}

Top-k documents retrieved from the indexed corpus (is_relevant is derived from qrels).

Usage

python

from datasets import load_dataset

corpus = load_dataset("DinoStackAI/qasper-rag", "corpus")["train"]
queries = load_dataset("DinoStackAI/qasper-rag", "queries")
qrels = load_dataset("DinoStackAI/qasper-rag", "qrels")
answers = load_dataset("DinoStackAI/qasper-rag", "answers")

train_queries = queries["train"]
dev_qrels = qrels["dev"]
test_answers = answers["test"]

Citation

QASPER is released under the CC BY 4.0 License.

bibtex

@inproceedings{Dasigi2021ADO,
  title={A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers},
  author={Pradeep Dasigi and Kyle Lo and Iz Beltagy and Arman Cohan and Noah A. Smith and Matt Gardner},
  year={2021}
}