lca
Datasets
All datasets matching “lca”lca-results
Long Code Arena (raw results)
These are the raw results from the Long Code Arena benchmark suite, as well as the corresponding model predictions.
Please use the subset dropdown menu to select the necessary data relating to our six benchmarks:
🤗 Library-based code generation
🤗 CI builds repair
🤗 Project-level code completion
🤗 Commit message generation🤗 Bug localization
🤗 Module summarization
lca-bug-localization
🏟️ Long Code Arena (Bug localization)
This is the benchmark for the Bug localization task as part of the
🏟️ Long Code Arena benchmark.
The bug localization problem can be formulated as follows: given an issue with a bug description and a repository snapshot in a state where the bug is reproducible, identify the files within the repository that need to be modified to address the reported bug.
The dataset provides all the required components for evaluation of bug localization… See the full description on the dataset page: https://huggingface.co/datasets/JetBrains-Research/lca-bug-localization.lca-project-level-code-completion
🏟️ Long Code Arena (Project-level code completion)
This is the benchmark for Project-level code completion task as part of the 🏟️ Long Code Arena benchmark.
Each datapoint contains the file for completion, a list of lines to complete with their categories (see the categorization below), and a repository snapshot that can be used to build the context.
All the repositories are published under permissive licenses (MIT, Apache-2.0, BSD-3-Clause, and BSD-2-Clause). The datapoints can… See the full description on the dataset page: https://huggingface.co/datasets/JetBrains-Research/lca-project-level-code-completion.GPTspeech_encodec_v2
Dataset Card for "GPTspeech_encodec_v2"
More Information needed
text_interference_vocalsoundlca-commit-message-generation
🏟️ Long Code Arena (Commit message generation)
This is the benchmark for the Commit message generation task as part of the
🏟️ Long Code Arena benchmark.
The dataset is a manually curated subset of the Python test set from the 🤗 CommitChronicle dataset, tailored for larger commits.
All the repositories are published under permissive licenses (MIT, Apache-2.0, and BSD-3-Clause). The datapoints can be removed upon request.
How-to
from datasets import load_dataset… See the full description on the dataset page: https://huggingface.co/datasets/JetBrains-Research/lca-commit-message-generation.