Datasetpublic

Catkamakura/ts-qa

Time-Sensitive QA Dataset (StreamingQA + TimeQA) A time-sensitive question answering dataset combining StreamingQA and TimeQA for evaluating temporal reasoning in RAG systems. Dataset Description This dataset contains question-answer pairs where the correct answer depends on a specific timestamp. Each question is prefixed with a date (e.g., "Today is Tuesday, September 24, 2013.") and the model must use temporal context to provide the correct answer for that point… See the full description on the dataset page: https://huggingface.co/datasets/Catkamakura/ts-qa.

sourceHugging Facemitupdated 8mo agoView on Hugging Face

0likes54downloads

Dataset Card

Time-Sensitive QA Dataset (StreamingQA + TimeQA)

A time-sensitive question answering dataset combining StreamingQA and TimeQA for evaluating temporal reasoning in RAG systems.

Dataset Description

This dataset contains question-answer pairs where the correct answer depends on a specific timestamp. Each question is prefixed with a date (e.g., "Today is Tuesday, September 24, 2013.") and the model must use temporal context to provide the correct answer for that point in time.

Sources

—StreamingQA (Liska et al., 2022): Time-sensitive questions about news articles from the WMT corpus
—TimeQA (Chen et al., 2021): Temporal knowledge base questions derived from WikiData

Features

Field	Type	Description
`id`	string	Unique identifier
`question`	string	Full question with timestamp prefix (e.g., "Today is Monday, Jan 5, 2015. Who is...")
`context`	string	Evidence document containing the answer
`answers`	list[string]	List of acceptable answers
`source`	string	Original dataset (`streamingqa` or `timeqa`)
`question_timestamp`	string	When the question is asked (ISO format)
`evidence_timestamp`	string	When the evidence document was published
`dimensions`	string	Metadata about temporal dimensions (JSON)
`is_hard_negative`	bool	Whether this is a hard negative (temporal perturbation)
`original_question`	string	Question without timestamp prefix

Splits

Split	Examples	Description
train	25,041	Training set
validation	2,800	Validation set
test	9,089	Test/evaluation set

Hard Negatives

Examples with is_hard_negative=True are temporal perturbations where the question timestamp is shifted to make the context outdated or incorrect. These are useful for evaluating a model's ability to detect temporal mismatches between query time and evidence time.

Usage

python

from datasets import load_dataset

# Load the dataset
dataset = load_dataset("Catkamakura/ts-qa")

# Access splits
train = dataset["train"]
validation = dataset["validation"]
test = dataset["test"]

# Example
example = train[0]
print(example["question"])
# "Today is Tuesday, September 24, 2013. Who was the Transport Minister of Thailand?"
print(example["answers"])
# ["Chadchart Sittipunt"]
print(example["source"])
# "streamingqa"

# Filter by source
streamingqa_examples = train.filter(lambda x: x["source"] == "streamingqa")
timeqa_examples = train.filter(lambda x: x["source"] == "timeqa")

# Filter hard negatives
hard_negatives = train.filter(lambda x: x["is_hard_negative"])

Citation

If you use this dataset, please cite the original sources:

bibtex

@inproceedings{liska2022streamingqa,
  title={StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models},
  author={Liska, Adam and Kocisky, Tomas and Gribovskaya, Elena and Terber, Tayfun and Shutova, Ekaterina and Glaese, Amelia},
  booktitle={International Conference on Machine Learning},
  pages={13604--13622},
  year={2022},
  organization={PMLR}
}

@inproceedings{chen2021timeqa,
  title={A Dataset for Answering Time-Sensitive Questions},
  author={Chen, Wenhu and Wang, Xinyi and Wang, William Yang},
  booktitle={NeurIPS 2021 Datasets and Benchmarks Track},
  year={2021}
}

License

MIT License