karpathy/fineweb-edu-100B-gpt2-token-shards
FineWeb Edu 100B dataset tokenized with GPT-2 tokenizer using the code in llm.c repo.
84.9k
the rest of the shards
add next 100 shards
add 100 train shards
add the val shard as a test first
Update README.md
initial commit