Datasetpublic

BEE-spoke-data/fineweb-1M_longish

fineweb "longish" 1M 1m samples w/ random seed w.r.t. previous samples. min 512 GPT-4 tiktoken tokens max 8192 GPT-4 tiktoken tokens BEE-spoke-data/claude-tokenizer token count: token_count count 1000000.000000 mean 1218.231641 std 935.733312 min 139.000000 25% 683.000000 50% 905.000000 75% 1350.000000 max 9550.000000 Total count: 1218.23 M tokens

sourceHugging Faceodc-byupdated 9mo agoView on Hugging Face

4likes97downloads

root.gitattributes

Nothing at this path on main. The folder may be empty, or the revision may not exist.

BEE-spoke-data/fineweb-1M_longish · main · files are served by the source, never re-hosted here