Datasetpublic

Gugu8/Pretrain

pretrain-60GB A 60GB, knowledge-dense pretraining corpus built as a clean, superior alternative to TinyStories. TinyStories teaches grammar. This teaches knowledge, reasoning, and code. File: pretrain.csv - 60GB, ~15M rows, single column text Why vs TinyStories? TinyStories is fiction for kids. This is textbook / encyclopedia / reasoning / code. Every row is 3500-5200 chars packed with 2-3 fused concepts, not a story. Physics, Chem, Bio, Math, CS, History, Geo… See the full description on the dataset page: https://huggingface.co/datasets/Gugu8/Pretrain.

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

0likes24downloads

settings

This repository belongs to Gugu8 on Hugging Face.

CoolFace never edits a repository it does not host. Visibility, licence, collaborators and gating are all managed at the source.

namePretrain

visibilitypublic

licenceother

gatedno

ownerGugu8

Account settings