ucinlp/drop

1---2annotations_creators:3- crowdsourced4language_creators:5- crowdsourced6language:7- en8license:9- cc-by-sa-4.010multilinguality:11- monolingual12size_categories:13- 10K<n<100K14source_datasets:15- original16task_categories:17- question-answering18- text2text-generation19task_ids:20- extractive-qa21- abstractive-qa22paperswithcode_id: drop23pretty_name: DROP24dataset_info:25  features:26  - name: section_id27    dtype: string28  - name: query_id29    dtype: string30  - name: passage31    dtype: string32  - name: question33    dtype: string34  - name: answers_spans35    sequence:36    - name: spans37      dtype: string38    - name: types39      dtype: string40  splits:41  - name: train42    num_bytes: 10557250643    num_examples: 7740044  - name: validation45    num_bytes: 1173775546    num_examples: 953547  download_size: 1153838748  dataset_size: 11731026149configs:50- config_name: default51  data_files:52  - split: train53    path: data/train-*54  - split: validation55    path: data/validation-*56---57 58# Dataset Card for "drop"59 60## Table of Contents61- [Dataset Description](#dataset-description)62  - [Dataset Summary](#dataset-summary)63  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)64  - [Languages](#languages)65- [Dataset Structure](#dataset-structure)66  - [Data Instances](#data-instances)67  - [Data Fields](#data-fields)68  - [Data Splits](#data-splits)69- [Dataset Creation](#dataset-creation)70  - [Curation Rationale](#curation-rationale)71  - [Source Data](#source-data)72  - [Annotations](#annotations)73  - [Personal and Sensitive Information](#personal-and-sensitive-information)74- [Considerations for Using the Data](#considerations-for-using-the-data)75  - [Social Impact of Dataset](#social-impact-of-dataset)76  - [Discussion of Biases](#discussion-of-biases)77  - [Other Known Limitations](#other-known-limitations)78- [Additional Information](#additional-information)79  - [Dataset Curators](#dataset-curators)80  - [Licensing Information](#licensing-information)81  - [Citation Information](#citation-information)82  - [Contributions](#contributions)83 84## Dataset Description85 86- **Homepage:** https://allenai.org/data/drop87- **Repository:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)88- **Paper:** https://aclanthology.org/N19-1246/89- **Paper:** https://arxiv.org/abs/1903.0016190- **Point of Contact:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)91- **Size of downloaded dataset files:** 8.30 MB92- **Size of the generated dataset:** 110.91 MB93- **Total amount of disk used:** 119.21 MB94 95### Dataset Summary96 97DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs.98. DROP is a crowdsourced, adversarially-created, 96k-question benchmark, in which a system must resolve references in a99question, perhaps to multiple input positions, and perform discrete operations over them (such as addition, counting, or100 sorting). These operations require a much more comprehensive understanding of the content of paragraphs than what was101 necessary for prior datasets.102 103### Supported Tasks and Leaderboards104 105[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)106 107### Languages108 109[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)110 111## Dataset Structure112 113### Data Instances114 115#### default116 117- **Size of downloaded dataset files:** 8.30 MB118- **Size of the generated dataset:** 110.91 MB119- **Total amount of disk used:** 119.21 MB120 121An example of 'validation' looks as follows.122```123This example was too long and was cropped:124 125{126    "answers_spans": {127        "spans": ["Chaz Schilens"]128    },129    "passage": "\" Hoping to rebound from their loss to the Patriots, the Raiders stayed at home for a Week 16 duel with the Houston Texans.  Oak...",130    "question": "Who scored the first touchdown of the game?"131}132```133 134### Data Fields135 136The data fields are the same among all splits.137 138#### default139- `passage`: a `string` feature.140- `question`: a `string` feature.141- `answers_spans`: a dictionary feature containing:142  - `spans`: a `string` feature.143 144### Data Splits145 146| name  |train|validation|147|-------|----:|---------:|148|default|77409|      9536|149 150## Dataset Creation151 152### Curation Rationale153 154[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)155 156### Source Data157 158#### Initial Data Collection and Normalization159 160[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)161 162#### Who are the source language producers?163 164[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)165 166### Annotations167 168#### Annotation process169 170[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)171 172#### Who are the annotators?173 174[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)175 176### Personal and Sensitive Information177 178[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)179 180## Considerations for Using the Data181 182### Social Impact of Dataset183 184[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)185 186### Discussion of Biases187 188[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)189 190### Other Known Limitations191 192[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)193 194## Additional Information195 196### Dataset Curators197 198[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)199 200### Licensing Information201 202[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)203 204### Citation Information205 206```207@inproceedings{Dua2019DROP,208  author={Dheeru Dua and Yizhong Wang and Pradeep Dasigi and Gabriel Stanovsky and Sameer Singh and Matt Gardner},209  title={  {DROP}: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs},210  booktitle={Proc. of NAACL},211  year={2019}212}213 214```215 216 217### Contributions218 219Thanks to [@patrickvonplaten](https://github.com/patrickvonplaten), [@thomwolf](https://github.com/thomwolf), [@mariamabarham](https://github.com/mariamabarham), [@lewtun](https://github.com/lewtun) for adding this dataset.