bigcode/starcoder2data-extras

StarCoder2 Extras

This is the dataset of extra sources (besides Stack v2 code data) used to train the StarCoder2 family of models. It contains the following subsets:

• —Kaggle (kaggle): Kaggle notebooks from Meta-Kaggle-Code dataset, converted to scripts and prefixed with information on the Kaggle datasets used in the notebook. The file headers have a similar format to Jupyter Structured but the code content is only one single script.
• —StackOverflow (stackoverflow): stackoverflow conversations from this StackExchange dump.
• —Issues (issues): processed GitHub issues, same as the Stack v1 issues.
• —OWM (owm): the Open-Web-Math dataset.
• —LHQ (lhq): Leandro's High quality dataset, it is a compilation of high quality code files from: APPS-train, CodeContests, GSM8K-train, GSM8K-SciRel, DeepMind-Mathematics, Rosetta-Code, MultiPL-T, ProofSteps, ProofSteps-lean.
• —Wiki (wikipedia): the English subset of the Wikipedia dump in RedPajama.
• —ArXiv (arxiv): the ArXiv subset of RedPajama dataset, further processed the dataset only to retain latex source files and remove preambles, comments, macros, and bibliographies from these files.
• —IRlanguage (`ircpp, irlowresource, irpython`, `irrust`): these are intermediate representations of Python, Rust, C++ and other low resource languages.
• —Documentation (documentation): documentation of popular libraries.

For more details on the processing of each subset, check the StarCoder2 paper or The Stack v2 GitHub repository.

Usage

from datasets import load_dataset

# replace `kaggle` with one of the config names listed above
ds = load_dataset("bigcode/starcoder2data-extras", "kaggle", split="train")

Citation

@article{lozhkov2024starcoder,
  title={Starcoder 2 and the stack v2: The next generation},
  author={Lozhkov, Anton and Li, Raymond and Allal, Loubna Ben and Cassano, Federico and Lamy-Poirier, Joel and Tazi, Nouamane and Tang, Ao and Pykhtar, Dmytro and Liu, Jiawei and Wei, Yuxiang and others},
  journal={arXiv preprint arXiv:2402.19173},
  year={2024}
}