JamesConley/fineweb-sample-5.97B-512

FineWeb-Sample-5.97B-512 Dataset Description This dataset contains approximately 5.97 billion tokens (5,968,954,880 tokens) sampled from the FineWeb-Edu dataset. Each text sample is capped at a maximum of 512 tokens. Dataset Statistics Total Tokens: ~5.97B (5,968,954,880) Max Tokens per Sample: 512 Max Characters per Sample: 5,120 (10 chars/token estimate) Source Dataset: FineWeb-Edu 350BT Random Seed: 42 Dataset Structure The… See the full description on the dataset page: https://huggingface.co/datasets/JamesConley/fineweb-sample-5.97B-512.