icmlbenchname/library-based-code-generation
BenchName (Library-based code generation) This is the benchmark for Library-based code generation task as part of the BenchName benchmark. The current version includes 150 manually curated instructions asking the model to generate Python code using a particular library. The samples come from 62 Python repositories. All the samples in the dataset are based on reference example programs written by authors of the respective libraries. All the repositories are published under… See the full description on the dataset page: https://huggingface.co/datasets/icmlbenchname/library-based-code-generation.
BenchName (Library-based code generation)
This is the benchmark for Library-based code generation task as part of the BenchName benchmark.
The current version includes 150 manually curated instructions asking the model to generate Python code using a particular library. The samples come from 62 Python repositories. All the samples in the dataset are based on reference example programs written by authors of the respective libraries. All the repositories are published under permissive licenses (MIT, Apache-2.0, BSD-3-Clause, and BSD-2-Clause). The datapoints can be removed upon request.
How-to
Load the data via `load_dataset`:
from datasets import load_dataset
dataset = load_dataset("icmlbenchname/library-based-code-generation", split="test")Note that all the data we have is considered to be in the test split.
Dataset Structure
Samples structure
The samples in the dataset have the following structure: