Datasetpublic

KristianS7/prepacked-fineweb-edu-llama2-32K-T2048

prepacked-fineweb-edu-llama2-32K-T2048 Pre-tokenized and BOS-aligned best-fit packed version of FineWeb-Edu for training with looped nanochat. Tokenized with the Llama 2 tokenizer (32,000 base vocab + 8 special tokens = 32,008). Stats Train split Source karpathy/fineweb-edu-100b-shuffle (1,821 shards) Total tokens 63.26B Total docs 97.1M Rows 30,873,598 Shards 2,059 (train-00000 to train-02058) Rows per shard ~15,000… See the full description on the dataset page: https://huggingface.co/datasets/KristianS7/prepacked-fineweb-edu-llama2-32K-T2048.

sourceHugging Faceodc-byupdated 6mo agoView on Hugging Face

0likes1.3kdownloads

settings

This repository belongs to KristianS7 on Hugging Face.

CoolFace never edits a repository it does not host. Visibility, licence, collaborators and gating are all managed at the source.

nameprepacked-fineweb-edu-llama2-32K-T2048

visibilitypublic

licenceodc-by

gatedno

ownerKristianS7

Account settings