CoolFace
Datasetpublic

ruediste/codeparrot-github-code-10G

This is data is derived from the Codeparrot Dataset by taking the first 10GB of text from each language, and splitting it into individual configs. This results in a download size of about 3GB per language. Sample usage: from datasets import load_dataset dataset = load_dataset("ruediste/codeparrot-github-code-10G", "java") List of Languages: languages = { 'HTML': 'html', 'Java': 'java', 'JavaScript': 'js', 'CSS': 'css', 'C#': 'cs', 'TypeScript': 'ts', "Batchfile":… See the full description on the dataset page: https://huggingface.co/datasets/ruediste/codeparrot-github-code-10G.

sourceHugging Faceupdated 2y agoView on Hugging Face
2likes2.5kdownloads
Dataset Card

This is data is derived from the Codeparrot Dataset by taking the first 10GB of text from each language, and splitting it into individual configs. This results in a download size of about 3GB per language.

Sample usage:

python
from datasets import load_dataset
dataset = load_dataset("ruediste/codeparrot-github-code-10G", "java")

List of Languages:

languages = {
    'HTML': 'html',
    'Java': 'java',
    'JavaScript': 'js',
    'CSS': 'css',
    'C#': 'cs',
    'TypeScript': 'ts',
    "Batchfile": "bat",
    "C": 'c',
    "C++": 'cpp',
    "CMake": "cmake",
    "Dockerfile": "dockerfile",
    "FORTRAN": 'f',
    "GO": "go",
    "Haskell": "hs",
    "Julia": "jl",
    "Lua": "lua",
    "Makefile": "makefile",
    "Markdown": "md",
    "PHP": "php",
    "Perl": "perl",
    "PowerShell": 'ps1',
    "Python": "py",
    "Ruby": "rb",
    "Rust": "rs",
    "SQL": "sql",
    "Scala": "scala",
    "Shell": "sh",
    "TeX": "tex",
    "Visual Basic": "vb"
}

Please note that assembly to lost in the conversion.