Datasetpublic

karpathy/fineweb-edu-100B-gpt2-token-shards

FineWeb Edu 100B dataset tokenized with GPT-2 tokenizer using the code in llm.c repo.

sourceHugging Faceodc-byupdated 2y agoView on Hugging Face

8likes4.9kdownloads

discussions and pull requests

Conversations for this repository live on Hugging Face.

CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.

Open discussions on Hugging Face