google/code_x_glue_cc_code_refinement

1---2annotations_creators:3- expert-generated4language_creators:5- found6language:7- code8license:9- c-uda10multilinguality:11- other-programming-languages12size_categories:13- 10K<n<100K14source_datasets:15- original16task_categories:17- text2text-generation18task_ids: []19pretty_name: CodeXGlueCcCodeRefinement20tags:21- debugging22dataset_info:23- config_name: medium24  features:25  - name: id26    dtype: int3227  - name: buggy28    dtype: string29  - name: fixed30    dtype: string31  splits:32  - name: train33    num_bytes: 3261483434    num_examples: 5236435  - name: validation36    num_bytes: 408674137    num_examples: 654638  - name: test39    num_bytes: 406367340    num_examples: 654541  download_size: 3997972442  dataset_size: 4076524843- config_name: small44  features:45  - name: id46    dtype: int3247  - name: buggy48    dtype: string49  - name: fixed50    dtype: string51  splits:52  - name: train53    num_bytes: 1300671954    num_examples: 4668055  - name: validation56    num_bytes: 162925057    num_examples: 583558  - name: test59    num_bytes: 161970860    num_examples: 583561  download_size: 1555542162  dataset_size: 1625567763---64 65# Dataset Card for "code_x_glue_cc_code_refinement"66 67## Table of Contents68- [Dataset Description](#dataset-description)69  - [Dataset Summary](#dataset-summary)70  - [Supported Tasks and Leaderboards](#supported-tasks)71  - [Languages](#languages)72- [Dataset Structure](#dataset-structure)73  - [Data Instances](#data-instances)74  - [Data Fields](#data-fields)75  - [Data Splits](#data-splits-sample-size)76- [Dataset Creation](#dataset-creation)77  - [Curation Rationale](#curation-rationale)78  - [Source Data](#source-data)79  - [Annotations](#annotations)80  - [Personal and Sensitive Information](#personal-and-sensitive-information)81- [Considerations for Using the Data](#considerations-for-using-the-data)82  - [Social Impact of Dataset](#social-impact-of-dataset)83  - [Discussion of Biases](#discussion-of-biases)84  - [Other Known Limitations](#other-known-limitations)85- [Additional Information](#additional-information)86  - [Dataset Curators](#dataset-curators)87  - [Licensing Information](#licensing-information)88  - [Citation Information](#citation-information)89  - [Contributions](#contributions)90 91## Dataset Description92 93- **Homepage:** https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/code-refinement94- **Paper:** https://arxiv.org/abs/2102.0466495 96### Dataset Summary97 98CodeXGLUE code-refinement dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/code-refinement99 100We use the dataset released by this paper(https://arxiv.org/pdf/1812.08693.pdf). The source side is a Java function with bugs and the target side is the refined one. All the function and variable names are normalized. Their dataset contains two subsets ( i.e.small and medium) based on the function length.101 102### Supported Tasks and Leaderboards103 104- `text2text-generation-other-debugging`: The dataset can be used to train a model for automatically fixing buggy code.105 106### Languages107 108- Java **programming** language109 110## Dataset Structure111 112### Data Instances113 114#### medium115 116An example of 'train' looks as follows.117```118{119    "buggy": "public static TYPE_1 init ( java.lang.String name , java.util.Date date ) { TYPE_1 VAR_1 = new TYPE_1 ( ) ; VAR_1 . METHOD_1 ( name ) ; java.util.Calendar VAR_2 = java.util.Calendar.getInstance ( ) ; VAR_2 . METHOD_2 ( date ) ; VAR_1 . METHOD_3 ( VAR_2 ) ; return VAR_1 ; }\n", 120    "fixed": "public static TYPE_1 init ( java.lang.String name , java.util.Date date ) { TYPE_1 VAR_1 = new TYPE_1 ( ) ; VAR_1 . METHOD_1 ( name ) ; java.util.Calendar VAR_2 = null ; if ( date != null ) { VAR_2 = java.util.Calendar.getInstance ( ) ; VAR_2 . METHOD_2 ( date ) ; } VAR_1 . METHOD_3 ( VAR_2 ) ; return VAR_1 ; }\n", 121    "id": 0122}123```124 125#### small126 127An example of 'validation' looks as follows.128```129{130    "buggy": "public java.util.List < TYPE_1 > METHOD_1 ( ) { java.util.ArrayList < TYPE_1 > VAR_1 = new java.util.ArrayList < TYPE_1 > ( ) ; for ( TYPE_2 VAR_2 : VAR_3 ) { VAR_1 . METHOD_2 ( VAR_2 . METHOD_1 ( ) ) ; } return VAR_1 ; } \n", 131    "fixed": "public java.util.List < TYPE_1 > METHOD_1 ( ) { return VAR_1 ; } \n", 132    "id": 0133}134```135 136### Data Fields137 138In the following each data field in go is explained for each config. The data fields are the same among all splits.139 140#### medium, small141 142|field name| type |          description           |143|----------|------|--------------------------------|144|id        |int32 | Index of the sample            |145|buggy     |string| The buggy version of the code  |146|fixed     |string| The correct version of the code|147 148### Data Splits149 150| name |train|validation|test|151|------|----:|---------:|---:|152|medium|52364|      6546|6545|153|small |46680|      5835|5835|154 155## Dataset Creation156 157### Curation Rationale158 159[More Information Needed]160 161### Source Data162 163#### Initial Data Collection and Normalization164 165Downloaded from GitHub Archive every public GitHub event between March 2011 and October 2017 and used the Google BigQuery APIs.166[More Information Needed]167 168#### Who are the source language producers?169 170Software Engineering developers.171 172### Annotations173 174#### Annotation process175 176Automatically annotated by filtering commit messages containing the pattern: ("fix" or "solve") and ("bug" or "issue" or "problem" or "error"). A statistically significant amount of samples (95% confidence level with 5% confidence interval) were manually evaluated by two authors to check if the filtered bug/fix pairs were correct. After all disagreements were settled, authors conclude that 97.6% were true positives.177 178#### Who are the annotators?179 180Heuristics and the authors of the paper.181 182### Personal and Sensitive Information183 184[More Information Needed]185 186## Considerations for Using the Data187 188### Social Impact of Dataset189 190[More Information Needed]191 192### Discussion of Biases193 194[More Information Needed]195 196### Other Known Limitations197 198[More Information Needed]199 200## Additional Information201 202### Dataset Curators203 204https://github.com/microsoft, https://github.com/madlag205 206### Licensing Information207 208Computational Use of Data Agreement (C-UDA) License.209 210### Citation Information211 212```213@article{DBLP:journals/corr/abs-2102-04664,214  author    = {Shuai Lu and215               Daya Guo and216               Shuo Ren and217               Junjie Huang and218               Alexey Svyatkovskiy and219               Ambrosio Blanco and220               Colin B. Clement and221               Dawn Drain and222               Daxin Jiang and223               Duyu Tang and224               Ge Li and225               Lidong Zhou and226               Linjun Shou and227               Long Zhou and228               Michele Tufano and229               Ming Gong and230               Ming Zhou and231               Nan Duan and232               Neel Sundaresan and233               Shao Kun Deng and234               Shengyu Fu and235               Shujie Liu},236  title     = {CodeXGLUE: {A} Machine Learning Benchmark Dataset for Code Understanding237               and Generation},238  journal   = {CoRR},239  volume    = {abs/2102.04664},240  year      = {2021}241}242@article{tufano2019empirical,243  title={An empirical study on learning bug-fixing patches in the wild via neural machine translation},244  author={Tufano, Michele and Watson, Cody and Bavota, Gabriele and Penta, Massimiliano Di and White, Martin and Poshyvanyk, Denys},245  journal={ACM Transactions on Software Engineering and Methodology (TOSEM)},246  volume={28},247  number={4},248  pages={1--29},249  year={2019},250  publisher={ACM New York, NY, USA}251}252```253 254### Contributions255 256Thanks to @madlag (and partly also @ncoop57) for adding this dataset.