CoolFace
Datasetpublic

jenyag/repo-code-completion

Repository Level Code Completion Dataset for Evaluation This is a dataset of repository snapshots before a commit where a python file has been added. One needs to complete added file with given content of repository composed in different ways. How to load the data via load_dataset: from datasets import load_dataset data_files = # choose from the table below dataset = load_dataset("jenyag/repo-code-completion", data_files=data_files, split="train")… See the full description on the dataset page: https://huggingface.co/datasets/jenyag/repo-code-completion.

sourceHugging Faceapache-2.0updated 3y agoView on Hugging Face
3likes123downloads
Dataset Card

Repository Level Code Completion Dataset for Evaluation

This is a dataset of repository snapshots before a commit where a python file has been added. One needs to complete added file with given content of repository composed in different ways.

How to load the data

  1. 1.via `load_dataset`:
    from datasets import load_dataset

    data_files =  # choose from the table below
    dataset = load_dataset("jenyag/repo-code-completion", data_files=data_files, split="train")
Options for data_files:
**all_context****non_py_context****py_context**
function class mask half composerdata/functionclassmaskhalfcomposer/all_context/test-*data/functionclassmaskhalfcomposer/nonpycontext/test-*data/functionclassmaskhalfcomposer/py_context/test-*
imports first composerdata/importsfirstcomposer/all_context/test-*data/importsfirstcomposer/nonpycontext/test-*data/importsfirstcomposer/py_context/test-*
alphabetical composerdata/alphabeticalcomposer/allcontext/test-*data/alphabeticalcomposer/nonpy_context/test-*data/alphabeticalcomposer/pycontext/test-*
naive composerdata/naivecomposer/allcontext/test-*data/naivecomposer/nonpy_context/test-*data/naivecomposer/pycontext/test-*
path distance composerdata/pathdistancecomposer/all_context/test-*data/pathdistancecomposer/nonpycontext/test-*data/pathdistancecomposer/py_context/test-*
file length composerdata/filelengthcomposer/all_context/test-*data/filelengthcomposer/nonpycontext/test-*data/filelengthcomposer/py_context/test-*
half memory composerdata/halfmemorycomposer/all_context/test-*data/halfmemorycomposer/nonpycontext/test-*data/halfmemorycomposer/py_context/test-*
function class mask one composerdata/functionclassmaskonecomposer/all_context/test-*data/functionclassmaskonecomposer/nonpycontext/test-*data/functionclassmaskonecomposer/py_context/test-*

How to get the full context for the specific line

for datapoint in dataset:
  project_context = datapoint['project_context']  # The project context may be quite long
  for file_context_dict, ground_truth in zip(datapoint['file_context'], datapoint['gt']):
    file_context = file_context_dict['content']
    full_context = project_context + file_context