google/code_x_glue_cc_code_refinement

1---2annotations_creators:3- expert-generated4language_creators:5- found6language:7- code8license:9- c-uda10multilinguality:11- other-programming-languages12size_categories:13- 10K<n<100K14source_datasets:15- original16task_categories:17- text2text-generation18task_ids: []19pretty_name: CodeXGlueCcCodeRefinement20tags:21- debugging22dataset_info:23- config_name: medium24  features:25  - name: id26    dtype: int3227  - name: buggy28    dtype: string29  - name: fixed30    dtype: string31  splits:32  - name: train33    num_bytes: 3261478634    num_examples: 5236435  - name: validation36    num_bytes: 408673337    num_examples: 654638  - name: test39    num_bytes: 406366540    num_examples: 654541  download_size: 1492955942  dataset_size: 4076518443- config_name: small44  features:45  - name: id46    dtype: int3247  - name: buggy48    dtype: string49  - name: fixed50    dtype: string51  splits:52  - name: train53    num_bytes: 1300667954    num_examples: 4668055  - name: validation56    num_bytes: 162924257    num_examples: 583558  - name: test59    num_bytes: 161970060    num_examples: 583561  download_size: 589446262  dataset_size: 1625562163configs:64- config_name: medium65  data_files:66  - split: train67    path: medium/train-*68  - split: validation69    path: medium/validation-*70  - split: test71    path: medium/test-*72- config_name: small73  data_files:74  - split: train75    path: small/train-*76  - split: validation77    path: small/validation-*78  - split: test79    path: small/test-*80---81 82# Dataset Card for "code_x_glue_cc_code_refinement"83 84## Table of Contents85- [Dataset Description](#dataset-description)86  - [Dataset Summary](#dataset-summary)87  - [Supported Tasks and Leaderboards](#supported-tasks)88  - [Languages](#languages)89- [Dataset Structure](#dataset-structure)90  - [Data Instances](#data-instances)91  - [Data Fields](#data-fields)92  - [Data Splits](#data-splits-sample-size)93- [Dataset Creation](#dataset-creation)94  - [Curation Rationale](#curation-rationale)95  - [Source Data](#source-data)96  - [Annotations](#annotations)97  - [Personal and Sensitive Information](#personal-and-sensitive-information)98- [Considerations for Using the Data](#considerations-for-using-the-data)99  - [Social Impact of Dataset](#social-impact-of-dataset)100  - [Discussion of Biases](#discussion-of-biases)101  - [Other Known Limitations](#other-known-limitations)102- [Additional Information](#additional-information)103  - [Dataset Curators](#dataset-curators)104  - [Licensing Information](#licensing-information)105  - [Citation Information](#citation-information)106  - [Contributions](#contributions)107 108## Dataset Description109 110- **Homepage:** https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/code-refinement111- **Paper:** https://arxiv.org/abs/2102.04664112 113### Dataset Summary114 115CodeXGLUE code-refinement dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/code-refinement116 117We use the dataset released by this paper(https://arxiv.org/pdf/1812.08693.pdf). The source side is a Java function with bugs and the target side is the refined one. All the function and variable names are normalized. Their dataset contains two subsets ( i.e.small and medium) based on the function length.118 119### Supported Tasks and Leaderboards120 121- `text2text-generation-other-debugging`: The dataset can be used to train a model for automatically fixing buggy code.122 123### Languages124 125- Java **programming** language126 127## Dataset Structure128 129### Data Instances130 131#### medium132 133An example of 'train' looks as follows.134```135{136    "buggy": "public static TYPE_1 init ( java.lang.String name , java.util.Date date ) { TYPE_1 VAR_1 = new TYPE_1 ( ) ; VAR_1 . METHOD_1 ( name ) ; java.util.Calendar VAR_2 = java.util.Calendar.getInstance ( ) ; VAR_2 . METHOD_2 ( date ) ; VAR_1 . METHOD_3 ( VAR_2 ) ; return VAR_1 ; }\n", 137    "fixed": "public static TYPE_1 init ( java.lang.String name , java.util.Date date ) { TYPE_1 VAR_1 = new TYPE_1 ( ) ; VAR_1 . METHOD_1 ( name ) ; java.util.Calendar VAR_2 = null ; if ( date != null ) { VAR_2 = java.util.Calendar.getInstance ( ) ; VAR_2 . METHOD_2 ( date ) ; } VAR_1 . METHOD_3 ( VAR_2 ) ; return VAR_1 ; }\n", 138    "id": 0139}140```141 142#### small143 144An example of 'validation' looks as follows.145```146{147    "buggy": "public java.util.List < TYPE_1 > METHOD_1 ( ) { java.util.ArrayList < TYPE_1 > VAR_1 = new java.util.ArrayList < TYPE_1 > ( ) ; for ( TYPE_2 VAR_2 : VAR_3 ) { VAR_1 . METHOD_2 ( VAR_2 . METHOD_1 ( ) ) ; } return VAR_1 ; } \n", 148    "fixed": "public java.util.List < TYPE_1 > METHOD_1 ( ) { return VAR_1 ; } \n", 149    "id": 0150}151```152 153### Data Fields154 155In the following each data field in go is explained for each config. The data fields are the same among all splits.156 157#### medium, small158 159|field name| type |          description           |160|----------|------|--------------------------------|161|id        |int32 | Index of the sample            |162|buggy     |string| The buggy version of the code  |163|fixed     |string| The correct version of the code|164 165### Data Splits166 167| name |train|validation|test|168|------|----:|---------:|---:|169|medium|52364|      6546|6545|170|small |46680|      5835|5835|171 172## Dataset Creation173 174### Curation Rationale175 176[More Information Needed]177 178### Source Data179 180#### Initial Data Collection and Normalization181 182Downloaded from GitHub Archive every public GitHub event between March 2011 and October 2017 and used the Google BigQuery APIs.183[More Information Needed]184 185#### Who are the source language producers?186 187Software Engineering developers.188 189### Annotations190 191#### Annotation process192 193Automatically annotated by filtering commit messages containing the pattern: ("fix" or "solve") and ("bug" or "issue" or "problem" or "error"). A statistically significant amount of samples (95% confidence level with 5% confidence interval) were manually evaluated by two authors to check if the filtered bug/fix pairs were correct. After all disagreements were settled, authors conclude that 97.6% were true positives.194 195#### Who are the annotators?196 197Heuristics and the authors of the paper.198 199### Personal and Sensitive Information200 201[More Information Needed]202 203## Considerations for Using the Data204 205### Social Impact of Dataset206 207[More Information Needed]208 209### Discussion of Biases210 211[More Information Needed]212 213### Other Known Limitations214 215[More Information Needed]216 217## Additional Information218 219### Dataset Curators220 221https://github.com/microsoft, https://github.com/madlag222 223### Licensing Information224 225Computational Use of Data Agreement (C-UDA) License.226 227### Citation Information228 229```230@article{DBLP:journals/corr/abs-2102-04664,231  author    = {Shuai Lu and232               Daya Guo and233               Shuo Ren and234               Junjie Huang and235               Alexey Svyatkovskiy and236               Ambrosio Blanco and237               Colin B. Clement and238               Dawn Drain and239               Daxin Jiang and240               Duyu Tang and241               Ge Li and242               Lidong Zhou and243               Linjun Shou and244               Long Zhou and245               Michele Tufano and246               Ming Gong and247               Ming Zhou and248               Nan Duan and249               Neel Sundaresan and250               Shao Kun Deng and251               Shengyu Fu and252               Shujie Liu},253  title     = {CodeXGLUE: {A} Machine Learning Benchmark Dataset for Code Understanding254               and Generation},255  journal   = {CoRR},256  volume    = {abs/2102.04664},257  year      = {2021}258}259@article{tufano2019empirical,260  title={An empirical study on learning bug-fixing patches in the wild via neural machine translation},261  author={Tufano, Michele and Watson, Cody and Bavota, Gabriele and Penta, Massimiliano Di and White, Martin and Poshyvanyk, Denys},262  journal={ACM Transactions on Software Engineering and Methodology (TOSEM)},263  volume={28},264  number={4},265  pages={1--29},266  year={2019},267  publisher={ACM New York, NY, USA}268}269```270 271### Contributions272 273Thanks to @madlag (and partly also @ncoop57) for adding this dataset.