AmazonScience/migration-bench-java-selected

MigrationBench

<table> <tr> <td style="padding: 0;"> <a href="https://huggingface.co/collections/AmazonScience/migrationbench-68125452fc21a4564b92b6c3"> <img src="https://img.shields.io/badge/-🤗 MigrationBench-4d5eff?style=flatten&labelColor" alt="MigrationBench (Hugging Face)"> </a> </td> <td style="padding: 0;"> <a href="https://github.com/amazon-science/MigrationBench"> <img src="https://img.shields.io/badge/MigrationBench-000000?style=flatten&logo=github" alt="MigrationBench (GitHub)"> </a> </td> <td style="padding: 0;"> <a href="https://github.com/amazon-science/SDFeedback"> <img src="https://img.shields.io/badge/SDFeedback-000000?style=flatten&logo=github&logoColor=white" alt="SDFeedback (GitHub)"> </a> </td> <td style="padding: 0;"> <a href="https://arxiv.org/abs/2505.09569"> <img src="https://img.shields.io/badge/arXiv-2505.09569-b31b1b.svg?style=flatten" alt="MigrationBench (arXiv)"> </a> </td> <td style="padding: 0; padding-left: 10px; vertical-align: middle;"> <a href="https://huggingface.co/datasets/AmazonScience/migration-bench-java-full"> <img src="https://img.shields.io/badge/-🤗 java--full-8a98ff?style=flat&labelColor" alt="java-full"> </a> </td> <td style="padding: 0; vertical-align: middle;"> <a href="https://huggingface.co/datasets/AmazonScience/migration-bench-java-selected"> <img src="https://img.shields.io/badge/-🤗 java--selected-8a98ff?style=flat&labelColor" alt="java-selected"> </a> </td> <td style="padding: 0; vertical-align: middle;"> <a href="https://huggingface.co/datasets/AmazonScience/migration-bench-java-utg"> <img src="https://img.shields.io/badge/-🤗 java--utg-8a98ff?style=flat&labelColor" alt="java-utg"> </a> </td> </tr> </table>

1. 📖 Overview

🤗 MigrationBench is a large-scale code migration benchmark dataset at the repository level, across multiple programming languages.

• —Current and initial release includes java 8 repositories with the maven build system, as of May 2025.

2. Datasets

There are three datasets in 🤗 MigrationBench:

• —All repositories included in the datasets are available on GitHub, under the MIT or Apache-2.0 license.

3. Metadata

Metadata is provided in the csv file for each dataset:

1. 1.repo (str): The original repo URL without the https://github.com/ prefix
2. 2.base_commit (str): Base commit id
3. 3.At this commit with java 8 and maven, the repository is able to (1) compile and (2) pass existing unit tests and integration tests if any
4. 4.It is the starting point for code migration from java 8
5. 5.num_java_files (int): Number of *.java files in the repository at base_commit, similarly for all other num_* columns
6. 6.num_loc (int): Lines of code for the repository
7. 7.num_pom_xml (int): Number of modules (pom.xml files) in the repository
8. 8.num_src_test_java_files (int): Number of *.java files in the dedicated src/test/ directory
9. 9.num_test_cases (int): Number of test cases, based on running the mvn -f test . command in the root directory
10. 10.Non negative values indicate number of test cases is parsed correctly from the output
11. 11.Negative values means it's unable to parse the output: [INFO] Results: (-2) or [INFO] Tests run: (-1) regex is missing
12. 12.license (str): The license of the repository, either MIT or Apache2.0 for the whole dataset

3.1 Parsing Java Test Cases

Here is a sample maven -f test . command output to get valid num_test_cases:

...
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
...

4. 📚 Citation

@misc{liu2025migrationbenchrepositorylevelcodemigration,
      title={MigrationBench: Repository-Level Code Migration Benchmark from Java 8},
      author={Linbo Liu and Xinle Liu and Qiang Zhou and Lin Chen and Yihan Liu and Hoan Nguyen and Behrooz Omidvar-Tehrani and Xi Shen and Jun Huan and Omer Tripp and Anoop Deoras},
      year={2025},
      eprint={2505.09569},
      archivePrefix={arXiv},
      primaryClass={cs.SE},
      url={https://arxiv.org/abs/2505.09569},
}