LM-Polygraph/coqa

1---2language:3- en4dataset_info:5- config_name: continuation6  features:7  - name: input8    dtype: string9  - name: output10    dtype: string11  splits:12  - name: train13    num_bytes: 24627801714    num_examples: 10864715  - name: test16    num_bytes: 1756643117    num_examples: 798318  download_size: 3294042419  dataset_size: 26384444820- config_name: empirical_baselines21  features:22  - name: input23    dtype: string24  - name: output25    dtype: string26  splits:27  - name: train28    num_bytes: 26930861629    num_examples: 10864730  - name: test31    num_bytes: 1926170832    num_examples: 798333  download_size: 3599816934  dataset_size: 28857032435- config_name: ling_1s36  features:37  - name: input38    dtype: string39  - name: output40    dtype: string41  splits:42  - name: train43    num_bytes: 37056762844    num_examples: 10864745  - name: test46    num_bytes: 2671615647    num_examples: 798348  download_size: 4561758749  dataset_size: 39728378450- config_name: simple_instruct51  features:52  - name: input53    dtype: string54  - name: output55    dtype: string56  splits:57  - name: train58    num_bytes: 25216328159    num_examples: 10864760  - name: test61    num_bytes: 1800161762    num_examples: 798363  download_size: 3321956964  dataset_size: 27016489865- config_name: verb_1s_top166  features:67  - name: input68    dtype: string69  - name: output70    dtype: string71  splits:72  - name: train73    num_bytes: 35838468074    num_examples: 10864775  - name: test76    num_bytes: 2582504577    num_examples: 798378  download_size: 4365236279  dataset_size: 38420972580- config_name: verb_1s_topk81  features:82  - name: input83    dtype: string84  - name: output85    dtype: string86  splits:87  - name: train88    num_bytes: 41913284589    num_examples: 10864790  - name: test91    num_bytes: 3029884992    num_examples: 798393  download_size: 4880160994  dataset_size: 44943169495- config_name: verb_2s_cot96  features:97  - name: input98    dtype: string99  - name: output100    dtype: string101  splits:102  - name: train103    num_bytes: 344246082104    num_examples: 108647105  - name: test106    num_bytes: 24783094107    num_examples: 7983108  download_size: 42255130109  dataset_size: 369029176110- config_name: verb_2s_top1111  features:112  - name: input113    dtype: string114  - name: output115    dtype: string116  splits:117  - name: train118    num_bytes: 269308616119    num_examples: 108647120  - name: test121    num_bytes: 19261708122    num_examples: 7983123  download_size: 35998169124  dataset_size: 288570324125- config_name: verb_2s_topk126  features:127  - name: input128    dtype: string129  - name: output130    dtype: string131  splits:132  - name: train133    num_bytes: 297093278134    num_examples: 108647135  - name: test136    num_bytes: 21307753137    num_examples: 7983138  download_size: 38279682139  dataset_size: 318401031140configs:141- config_name: continuation142  data_files:143  - split: train144    path: continuation/train-*145  - split: test146    path: continuation/test-*147- config_name: empirical_baselines148  data_files:149  - split: train150    path: empirical_baselines/train-*151  - split: test152    path: empirical_baselines/test-*153- config_name: ling_1s154  data_files:155  - split: train156    path: ling_1s/train-*157  - split: test158    path: ling_1s/test-*159- config_name: simple_instruct160  data_files:161  - split: train162    path: simple_instruct/train-*163  - split: test164    path: simple_instruct/test-*165- config_name: verb_1s_top1166  data_files:167  - split: train168    path: verb_1s_top1/train-*169  - split: test170    path: verb_1s_top1/test-*171- config_name: verb_1s_topk172  data_files:173  - split: train174    path: verb_1s_topk/train-*175  - split: test176    path: verb_1s_topk/test-*177- config_name: verb_2s_cot178  data_files:179  - split: train180    path: verb_2s_cot/train-*181  - split: test182    path: verb_2s_cot/test-*183- config_name: verb_2s_top1184  data_files:185  - split: train186    path: verb_2s_top1/train-*187  - split: test188    path: verb_2s_top1/test-*189- config_name: verb_2s_topk190  data_files:191  - split: train192    path: verb_2s_topk/train-*193  - split: test194    path: verb_2s_topk/test-*195---196 197# Dataset Card for coqa198 199<!-- Provide a quick summary of the dataset. -->200 201This is a preprocessed version of coqa dataset for benchmarks in LM-Polygraph.202 203## Dataset Details204 205### Dataset Description206 207<!-- Provide a longer summary of what this dataset is. -->208 209- **Curated by:** https://huggingface.co/LM-Polygraph210- **License:** https://github.com/IINemo/lm-polygraph/blob/main/LICENSE.md211 212### Dataset Sources [optional]213 214<!-- Provide the basic links for the dataset. -->215 216- **Repository:** https://github.com/IINemo/lm-polygraph217 218## Uses219 220<!-- Address questions around how the dataset is intended to be used. -->221 222### Direct Use223 224<!-- This section describes suitable use cases for the dataset. -->225 226This dataset should be used for performing benchmarks on LM-polygraph.227 228### Out-of-Scope Use229 230<!-- This section addresses misuse, malicious use, and uses that the dataset will not work well for. -->231 232This dataset should not be used for further dataset preprocessing.233 234## Dataset Structure235 236<!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. -->237 238This dataset contains the "continuation" subset, which corresponds to main dataset, used in LM-Polygraph. It may also contain other subsets, which correspond to instruct methods, used in LM-Polygraph.239 240Each subset contains two splits: train and test. Each split contains two string columns: "input", which corresponds to processed input for LM-Polygraph, and "output", which corresponds to processed output for LM-Polygraph.241 242## Dataset Creation243 244### Curation Rationale245 246<!-- Motivation for the creation of this dataset. -->247 248This dataset is created in order to separate dataset creation code from benchmarking code.249 250### Source Data251 252<!-- This section describes the source data (e.g. news text and headlines, social media posts, translated sentences, ...). -->253 254#### Data Collection and Processing255 256<!-- This section describes the data collection and processing process such as data selection criteria, filtering and normalization methods, tools and libraries used, etc. -->257 258Data is collected from https://huggingface.co/datasets/coqa and processed by using build_dataset.py script in repository.259 260#### Who are the source data producers?261 262<!-- This section describes the people or systems who originally created the data. It should also include self-reported demographic or identity information for the source data creators if this information is available. -->263 264People who created https://huggingface.co/datasets/coqa265 266## Bias, Risks, and Limitations267 268<!-- This section is meant to convey both technical and sociotechnical limitations. -->269 270This dataset contains the same biases, risks, and limitations as its source dataset https://huggingface.co/datasets/coqa271 272### Recommendations273 274<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->275 276Users should be made aware of the risks, biases and limitations of the dataset.277