BEE-spoke-data/fineweb-1M_longish
fineweb "longish" 1M 1m samples w/ random seed w.r.t. previous samples. min 512 GPT-4 tiktoken tokens max 8192 GPT-4 tiktoken tokens BEE-spoke-data/claude-tokenizer token count: token_count count 1000000.000000 mean 1218.231641 std 935.733312 min 139.000000 25% 683.000000 50% 905.000000 75% 1350.000000 max 9550.000000 Total count: 1218.23 M tokens
497
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face