Datasetpublic

BEE-spoke-data/falcon-refinedweb-1M_en_medium

BEE-spoke-data/falcon-refinedweb-1M_en_medium A sample from falcon-refinedweb: more than 512 & less than 8192 gpt4 tiktoken tokens en only (via fasttext-langdetect) 1M samples GPT-4 tiktoken token count: token_count count 1000000.000000 mean 1197.179246 std 964.177338 min 513.000000 25% 653.000000 50% 871.000000 75% 1315.000000 max 8191.000000 Total count: 1197.18 M tokens

sourceHugging Faceodc-byupdated 9mo agoView on Hugging Face

2likes36downloads

settings

This repository belongs to BEE-spoke-data on Hugging Face.

CoolFace never edits a repository it does not host. Visibility, licence, collaborators and gating are all managed at the source.

namefalcon-refinedweb-1M_en_medium

visibilitypublic

licenceodc-by

gatedno

ownerBEE-spoke-data

Account settings