dasyd/quants

QuAnTS: Question Answering on Time Series

 

QuAnTS is a challenging dataset designed to bridge the gap in question-answering research on time series data. The dataset features a wide variety of questions and answers concerning human movements, presented as tracked skeleton trajectories. QuAnTS also includes human reference performance to benchmark the practical usability of models trained on this dataset.

<img src="doc/intro-chat.png" alt="Example chat motivating time series question answering: Q: 'What is the person doing first?', A: 'They are waving.', Q: 'How many times are they jumping after that?', A: '...'" width="30%"/>

At present, there is no official leaderboard for this dataset.

Dataset Generation Overview

[image]

For details, please refer to the paper: Under Review

Task and Format

The primary task for the QuAnTS dataset is Time Series Question Answering. Given a time series of human skeleton trajectories and a question in natural language, the goal is to generate a correct answer. Answers are provided in one of the following formats: binary (Yes/No), multiple-choice (A/B/C), or open (free text). Additionally, to provide more training data for free-text answers, we provide entirely textual answers for all binary and multiple-choice questions. The ground truth action sequence or scene descriptions may not be used to answer the dataset — we provide them for debugging purposes only. The text in the dataset is in English.

We provide fixed splits into training, validation, and test portions, where only the latter may be used to compare performance across different approaches. You are free to mix the training and validation splits as needed.

Licensing, Citation, and Acknowledgments

The QuAnTS dataset is licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.

If you use the QuAnTS dataset in your research, please cite the paper):

@misc{divo2025quantsquestionansweringtime,
      title={QuAnTS: Question Answering on Time Series}, 
      author={Felix Divo and Maurice Kraus and Anh Q. Nguyen and Hao Xue and Imran Razzak and Flora D. Salim and Kristian Kersting and Devendra Singh Dhami},
      year={2025},
      eprint={2511.05124},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2511.05124}, 
}

The dataset was curated by a team of researchers from various institutions:

• —Felix Divo, Maurice Kraus, and Kristian Kersting (hessian.AI, DFKI, and the Centre for Cognitive Science) from Technische Universität Darmstadt.
• —Anh Q. Nguyen, Hao Xue, and Flora D. Salim from UNSW Sydney.
• —Imran Razzak from Mohamed bin Zayed University of Artificial Intelligence.
• —Devendra Singh Dhami from Eindhoven University of Technology.