andstor/the_pile_github
Dataset Card for The Pile GitHub Dataset Summary This is the GitHub subset of EleutherAi/The Pile dataset and contains GitHub repositories. The programming languages are identified using the guesslang library. A total of 54 programming languages are included in the dataset. Supported Tasks and Leaderboards [More Information Needed] Languages The following languages are covered by the dataset: 'Assembly', 'Batchfile', 'C', 'C#', 'C++'… See the full description on the dataset page: https://huggingface.co/datasets/andstor/the_pile_github.
Delete data/validation.parquet
Delete data/test.parquet
Update README.md
Upload 110 files
Upload 2 files
Update README.md
Update README.md
Delete the_pile_github.py
Update README.md
Update the_pile_github.py
URL encode data links
Update the_pile_github.py
Minor fix
Fix datafile counts
Update the_pile_github.py
Update data
Update test and validation data
Cleanup
Remove shuffle
Shuffle the data
Fix encodings
Fix train urls
Add validation and test data + restructure train files
Fix split names
Fix split names
Fix split names
Add all config
Add data
initial commit