Datasetpublic

Aniket200325/coder-pretrain-60gb

Coding LLM Pretraining Corpus Built with build_pretrain_dataset.ipynb + finalize_local.py (DuckDB). Composition Source Collected rows Collected size Final rows Final size code 8,415,197 30.00 GB 8,413,850 29.88 GB web 3,492,492 15.00 GB 3,390,372 14.50 GB math 659,720 3.00 GB 658,089 2.95 GB wiki 519,584 2.00 GB 519,535 1.99 GB docs 4,504,069 10.00 GB 4,289,545 9.74 GB Total final corpus: 59.06 GB of raw text (17,271,391 documents) across… See the full description on the dataset page: https://huggingface.co/datasets/Aniket200325/coder-pretrain-60gb.

sourceHugging Faceotherupdated 3mo agoView on Hugging Face

0likes146downloads

root.gitattributes

Nothing at this path on main. The folder may be empty, or the revision may not exist.

Aniket200325/coder-pretrain-60gb · main · files are served by the source, never re-hosted here