Datasetpublic

pt-sk/fineweb_edu_10B

This is the finewebedu dataset 10 Billion tokens. Tokenized using gpt-2 tokenizer. Each shards are a numpy file and contains 100 million tokens.

sourceHugging Facemitupdated 2y agoView on Hugging Face

0likes148downloads

discussions and pull requests

Conversations for this repository live on Hugging Face.

CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.

Open discussions on Hugging Face