barty/EvalRepair-Java

Dataset Description

• —Repository: MORepair
• —Paper: MORepair: Teaching LLMs to Repair Code via Multi-Objective Fine-tuning
• —Point of Contact: Boyang Yang

Dataset Summary

EvalRepair-Java is a benchmark for evaluating Java program repair performance, derived from HumanEval. It contains 163 single-function repair tasks, each with a buggy implementation and its corresponding fixed version.

Supported Tasks

• —Program Repair: Fixing bugs in Java functions
• —Code Generation: Generating correct implementations from buggy code

Dataset Structure

Each row contains:

• —task_id: Unique identifier for the task (same as HumanEval)
• —buggy_code: The buggy implementation
• —fixed_code: The correct implementation
• —unit_test: Unit tests for verifying the correctness of the implementation

Source Data

This dataset is derived from HumanEval, a benchmark for evaluating code generation capabilities. We manually introduced bugs into the original implementations and verified the fixes.

Citation

@article{morepair,
author = {Yang, Boyang and Tian, Haoye and Ren, Jiadong and Zhang, Hongyu and Klein, Jacques and Bissyande, Tegawende and Le Goues, Claire and Jin, Shunfu},
title = {MORepair: Teaching LLMs to Repair Code via Multi-Objective Fine-Tuning},
year = {2025},
publisher = {Association for Computing Machinery},
issn = {1049-331X},
url = {https://doi.org/10.1145/3735129},
doi = {10.1145/3735129},
journal = {ACM Trans. Softw. Eng. Methodol.},
}