Datasetpublic

pt-sk/fineweb_edu_10B

This is the finewebedu dataset 10 Billion tokens. Tokenized using gpt-2 tokenizer. Each shards are a numpy file and contains 100 million tokens.

sourceHugging Facemitupdated 2y agoView on Hugging Face

0likes78downloads

Dataset Card

This is the finewebedu dataset 10 Billion tokens. Tokenized using gpt-2 tokenizer. Each shards are a numpy file and contains 100 million tokens.