Gugu8/Pretrain
pretrain-60GB A 60GB, knowledge-dense pretraining corpus built as a clean, superior alternative to TinyStories. TinyStories teaches grammar. This teaches knowledge, reasoning, and code. File: pretrain.csv - 60GB, ~15M rows, single column text Why vs TinyStories? TinyStories is fiction for kids. This is textbook / encyclopedia / reasoning / code. Every row is 3500-5200 chars packed with 2-3 fused concepts, not a story. Physics, Chem, Bio, Math, CS, History, Geo… See the full description on the dataset page: https://huggingface.co/datasets/Gugu8/Pretrain.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face