Datasetpublic

karpathy/fineweb-edu-100B-gpt2-token-shards

FineWeb Edu 100B dataset tokenized with GPT-2 tokenizer using the code in llm.c repo.

sourceHugging Faceodc-byupdated 2y agoView on Hugging Face

8likes4.8kdownloads

Dataset Card

FineWeb Edu 100B dataset tokenized with GPT-2 tokenizer using the code in llm.c repo.