AmazonScience/migration-bench-java-selected

1---2license: apache-2.03dataset_info:4  features:5  - name: repo6    dtype: string7  - name: base_commit8    dtype: string9  - name: num_java_files10    dtype: int3211  - name: num_loc12    dtype: int3213  - name: num_pom_xml14    dtype: int3215  - name: num_src_test_java_files16    dtype: int3217  - name: num_test_cases18    dtype: int3219  - name: license20    dtype: string21  splits:22  - name: test23    num_examples: 30024programming_languages:25- java26tags:27- coding28- code-migration29- java30- amazon31- amazon-science32- aws33task_categories:34- text-generation35---36 37# MigrationBench38<table>39  <tr>40    <td style="padding: 0;">41      <a href="https://huggingface.co/collections/AmazonScience/migrationbench-68125452fc21a4564b92b6c3">42        <img src="https://img.shields.io/badge/-🤗 MigrationBench-4d5eff?style=flatten&labelColor" alt="MigrationBench (Hugging Face)">43      </a>44    </td>45    <td style="padding: 0;">46      <a href="https://github.com/amazon-science/MigrationBench">47        <img src="https://img.shields.io/badge/MigrationBench-000000?style=flatten&logo=github" alt="MigrationBench (GitHub)">48      </a>49    </td>50    <td style="padding: 0;">51      <a href="https://github.com/amazon-science/SDFeedback">52        <img src="https://img.shields.io/badge/SDFeedback-000000?style=flatten&logo=github&logoColor=white" alt="SDFeedback (GitHub)">53      </a>54    </td>55    <td style="padding: 0;">56      <a href="https://arxiv.org/abs/2505.09569">57        <img src="https://img.shields.io/badge/arXiv-2505.09569-b31b1b.svg?style=flatten" alt="MigrationBench (arXiv)">58      </a>59    </td>60    <td style="padding: 0; padding-left: 10px; vertical-align: middle;">61      <a href="https://huggingface.co/datasets/AmazonScience/migration-bench-java-full">62        <img src="https://img.shields.io/badge/-🤗 java--full-8a98ff?style=flat&labelColor" alt="java-full">63      </a>64    </td>65    <td style="padding: 0; vertical-align: middle;">66      <a href="https://huggingface.co/datasets/AmazonScience/migration-bench-java-selected">67        <img src="https://img.shields.io/badge/-🤗 java--selected-8a98ff?style=flat&labelColor" alt="java-selected">68      </a>69    </td>70    <td style="padding: 0; vertical-align: middle;">71      <a href="https://huggingface.co/datasets/AmazonScience/migration-bench-java-utg">72        <img src="https://img.shields.io/badge/-🤗 java--utg-8a98ff?style=flat&labelColor" alt="java-utg">73      </a>74    </td>75  </tr>76</table>77 78 79## 1. 📖 Overview80 81[🤗 MigrationBench](https://huggingface.co/collections/AmazonScience/migrationbench-68125452fc21a4564b92b6c3)82is a large-scale code migration benchmark dataset at the **repository** level,83across multiple programming languages.84 85- Current and initial release includes `java 8` repositories with the `maven` build system, as of May 2025.86 87 88## 2. Datasets89 90There are three datasets in [🤗 MigrationBench](https://huggingface.co/collections/AmazonScience/migrationbench-68125452fc21a4564b92b6c3):91- All repositories included in the datasets are available on GitHub, under the `MIT` or `Apache-2.0` license.92 93| Index | Dataset                                       | Size  | Notes                                                                                               |94|-------|-----------------------------------------------|-------|-----------------------------------------------------------------------------------------------------|95| 1     | [🤗 `AmazonScience/migration-bench-java-full`](https://huggingface.co/datasets/AmazonScience/migration-bench-java-full)         | 5,102 | Each repo has a test directory or at least one test case                              |96| 2     | [🤗 `AmazonScience/migration-bench-java-selected`](https://huggingface.co/datasets/AmazonScience/migration-bench-java-selected) |   300 | A **subset** of [🤗 `migration-bench-java-full`](https://huggingface.co/datasets/AmazonScience/migration-bench-java-full)                                          |97| 3     | [🤗 `AmazonScience/migration-bench-java-utg`](https://huggingface.co/datasets/AmazonScience/migration-bench-java-utg)           | 4,814 | The unit test generation (utg) dataset, **disjoint** with [🤗 `migration-bench-java-full`](https://huggingface.co/datasets/AmazonScience/migration-bench-java-full)|98 99 100## 3. Metadata101 102Metadata is provided in the `csv` file for each dataset:103 1041. `repo (str)`: The original repo URL without the `https://github.com/` prefix1051. `base_commit (str)`: Base commit id106    - At this commit with `java 8` and `maven`, the repository is able to (1) compile and (2) pass existing unit tests and integration tests if any107    - It is the starting point for code migration from `java 8`1081. `num_java_files (int)`: Number of `*.java` files in the repository at `base_commit`, similarly for all other `num_*` columns1091. `num_loc (int)`: Lines of code for the repository1101. `num_pom_xml (int)`: Number of modules (`pom.xml` files) in the repository1111. `num_src_test_java_files (int)`: Number of `*.java` files in the dedicated `src/test/` directory1121. `num_test_cases (int)`: Number of test cases, based on running the `mvn -f test .` command in the root directory113    - Non negative values indicate number of test cases is parsed correctly from the output114    - Negative values means it's unable to parse the output: `[INFO] Results:` (`-2`) or `[INFO] Tests run:` (`-1`) regex is missing1151. `license (str)`: The license of the repository, either `MIT` or `Apache2.0` for the whole dataset116 117### 3.1 Parsing Java Test Cases118 119Here is a sample `maven -f test .` command output to get valid `num_test_cases`:120 121```122...123[INFO] Results:124[INFO]125[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0126...127```128 129 130## 4. 📚 Citation131 132```bibtex133@misc{liu2025migrationbenchrepositorylevelcodemigration,134      title={MigrationBench: Repository-Level Code Migration Benchmark from Java 8},135      author={Linbo Liu and Xinle Liu and Qiang Zhou and Lin Chen and Yihan Liu and Hoan Nguyen and Behrooz Omidvar-Tehrani and Xi Shen and Jun Huan and Omer Tripp and Anoop Deoras},136      year={2025},137      eprint={2505.09569},138      archivePrefix={arXiv},139      primaryClass={cs.SE},140      url={https://arxiv.org/abs/2505.09569},141}142```143