Gugu8/Pretrain

pretrain-60GB A 60GB, knowledge-dense pretraining corpus built as a clean, superior alternative to TinyStories. TinyStories teaches grammar. This teaches knowledge, reasoning, and code. File: pretrain.csv - 60GB, ~15M rows, single column text Why vs TinyStories? TinyStories is fiction for kids. This is textbook / encyclopedia / reasoning / code. Every row is 3500-5200 chars packed with 2-3 fused concepts, not a story. Physics, Chem, Bio, Math, CS, History, Geo… See the full description on the dataset page: https://huggingface.co/datasets/Gugu8/Pretrain.