Datasetpublic

mponty/code_tutorials

Coding Tutorials This comprehensive dataset consists of 500,000 documents, summing up to around 1.5 billion tokens. Predominantly composed of coding tutorials, it has been meticulously compiled from various web crawl datasets like RefinedWeb, OSCAR, and Escorpius. The selection process involved a stringent filtering of files using regular expressions to ensure the inclusion of content that contains programming code (most of them). These tutorials offer more than mere code… See the full description on the dataset page: https://huggingface.co/datasets/mponty/code_tutorials.

sourceHugging Faceupdated 3y agoView on Hugging Face

9likes186downloads

Dataset Card

Coding Tutorials

This comprehensive dataset consists of 500,000 documents, summing up to around 1.5 billion tokens. Predominantly composed of coding tutorials, it has been meticulously compiled from various web crawl datasets like RefinedWeb, OSCAR, and Escorpius. The selection process involved a stringent filtering of files using regular expressions to ensure the inclusion of content that contains programming code (most of them).

These tutorials offer more than mere code snippets. They provide an extensive context, including the rationale behind the code, the problem being addressed, and detailed step-by-step instructions. This layered context is helpful for training a code-LM model, enabling it to discern the user intent behind a piece of code and thus facilitating more contextually relevant assistance.

Programming Language Distribution

       cpp ▏   39% █████████████████████████
    python ▏   25% ████████████████
      java ▏   16% ██████████
    csharp ▏    3% ██
javascript ▏    1% ▋
    kotlin ▏    1% ▋
     other ▏   14% █████████

Natural language distribution

en ▏   80% █████████████████████████
ru ▏   16% █████
zh ▏    2% ▋
es ▏    2% ▋