bigcode/commitpack

Dataset Card for CommitPack

Table of Contents

• —Table of Contents
• —Dataset Description
• —Dataset Summary
• —Languages
• —Dataset Structure
• —Data Instances
• —Data Fields
• —Data Splits
• —Dataset Creation
• —Curation Rationale
• —Source Data
• —Annotations
• —Additional Information
• —Licensing Information
• —Citation Information
• —Contributions

Dataset Description

• —Repository: https://github.com/bigcode-project/octopack
• —Paper: OctoPack: Instruction Tuning Code Large Language Models
• —Point of Contact: Niklas Muennighoff

Dataset Summary

CommitPack is is a 4TB dataset of commits scraped from GitHub repositories that are permissively licensed.

• —Creation: The dataset can be recreated using instructions available here.
• —Languages: 350
• —OctoPack🐙🎒:

<table> <tr> <th>Data</t> <td><a href=https://huggingface.co/datasets/bigcode/commitpack>CommitPack</a></td> <td>4TB of GitHub commits across 350 programming languages</td> </tr> <tr> <th></t> <td><a href=https://huggingface.co/datasets/bigcode/commitpackft>CommitPackFT</a></td> <td>Filtered version of CommitPack for high-quality commit messages that resemble instructions</td> </tr> <tr> <th>Model</t> <td><a href=https://huggingface.co/bigcode/octocoder>OctoCoder</a></td> <td>StarCoder (16B parameters) instruction tuned on CommitPackFT + OASST</td> </tr> <tr> <th></t> <td><a href=https://huggingface.co/bigcode/octogeex>OctoGeeX</a></td> <td>CodeGeeX2 (6B parameters) instruction tuned on CommitPackFT + OASST</td> </tr> <tr> <th>Evaluation</t> <td><a href=https://huggingface.co/datasets/bigcode/humanevalpack>HumanEvalPack</a></td> <td>Extension of OpenAI's HumanEval to cover 3 scenarios across 6 languages</td> </tr> </table>

Dataset Structure

Data Instances

An example looks as follows:

{
  'commit': '0c17311f7fd511f5dae8f8e4acc2dce1a2de3cf5',
  'old_file': 'main.py',
  'new_file': 'main.py',
  'old_contents': "import numpy as np\nimport matplotlib.pyplot as plt\n\n# generate sample data\nx_data = np.linspace(-5, 5, 20)\ny_data = np.random.normal(0.0, 1.0, x_data.size)\n\nplt.plot(x_data, y_data, 'o')\nplt.show()\n",
  'new_contents': "import math\nimport numpy as np\nimport matplotlib.pyplot as plt\n\n# generate sample data\nx_data = np.linspace(-math.pi, math.pi, 30)\ny_data = np.sin(x_data) + np.random.normal(0.0, 0.1, x_data.size)\n\nplt.plot(x_data, y_data, 'o')\nplt.show()\n\n",
  'subject': 'Change to sin() function with noise',
  'message': 'Change to sin() function with noise\n',
  'lang': 'Python',
  'license': 'mit',
  'repos': 'MorganR/basic-gaussian-process',
  'returncode': 0,
  'stderr': ''
}

Data Fields

The data fields are the same among all splits:

• —commit: unique commit id
• —old_file: name of the file before the commit
• —new_file: name of the file after the commit
• —old_contents: contents of the file before the commit
• —new_contents: contents of the file after the commit
• —subject: subject of the commit (this is used for all experiments in the paper)
• —message: message of the commit (commonly the same as the subject)
• —lang: programming language
• —license: license of the repository the code stems from, one of ['mit', 'artistic-2.0', 'isc', 'cc0-1.0', 'epl-1.0', 'mpl-2.0', 'unlicense', 'unknown', 'apache-2.0', 'bsd-3-clause', 'agpl-3.0', 'lgpl-2.1', 'bsd-2-clause']
• —repos: name of the the repository the code stems from (if multiple, they are comma separated)
• —returncode: if applicable errorcode during scraping (0 = no error)
• —'stderr': if applicable the error that occured during scraping (empty = no error)

Data Splits

Additional Information

Licensing Information

Each sample comes from a code repository with a permissive license. The license is provided by the license field for each sample.

Citation Information

@article{muennighoff2023octopack,
      title={OctoPack: Instruction Tuning Code Large Language Models}, 
      author={Niklas Muennighoff and Qian Liu and Armel Zebaze and Qinkai Zheng and Binyuan Hui and Terry Yue Zhuo and Swayam Singh and Xiangru Tang and Leandro von Werra and Shayne Longpre},
      journal={arXiv preprint arXiv:2308.07124},
      year={2023}
}