Datasetpublic

KoalaAI/GitHub-CC0

Public Domain GitHub Repositories Dataset This dataset contains metadata and source code of 9,000 public domain (cc0 or unlicense) licensed GitHub repositories that have more than 25 stars. The dataset was created by scraping the GitHub API and downloading the repositories, so long as they are under 100mb. The dataset can be used for various natural language processing and software engineering tasks, such as code summarization, code generation, code search, code analysis, etc.… See the full description on the dataset page: https://huggingface.co/datasets/KoalaAI/GitHub-CC0.

sourceHugging Facecc0-1.0updated 3y agoView on Hugging Face

6likes525downloads

Dataset Card

Public Domain GitHub Repositories Dataset

This dataset contains metadata and source code of 9,000 public domain (cc0 or unlicense) licensed GitHub repositories that have more than 25 stars. The dataset was created by scraping the GitHub API and downloading the repositories, so long as they are under 100mb. The dataset can be used for various natural language processing and software engineering tasks, such as code summarization, code generation, code search, code analysis, etc.

Dataset Summary

—Number of repositories: 9,000
—Size: 2.4 GB (compressed)
—Languages: Python, JavaScript, Java, C#, C++, Ruby, PHP, Go, Swift, and Rust
—License: Public Domain (cc0 or unlicense)

Dataset License

This dataset is released under the Public Domain (cc0 or unlicense) license. The original repositories are also licensed under the Public Domain (cc0 or unlicense) license. You can use this dataset for any purpose without any restrictions.

Reproducing this dataset

This dataset was produced by modifiying the "github-downloader" from EleutherAI. You can access our fork on our GitHub page Replication steps are included in it's readme there.