BEE-spoke-data/falcon-refinedweb-1M_en_medium
BEE-spoke-data/falcon-refinedweb-1M_en_medium A sample from falcon-refinedweb: more than 512 & less than 8192 gpt4 tiktoken tokens en only (via fasttext-langdetect) 1M samples GPT-4 tiktoken token count: token_count count 1000000.000000 mean 1197.179246 std 964.177338 min 513.000000 25% 653.000000 50% 871.000000 75% 1315.000000 max 8191.000000 Total count: 1197.18 M tokens
236
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face