Datasetpublic

EliMC/TxT360-5M-sample-en

BEE-spoke-data/TxT360-5M-sample-en english only sample from LLM360/TxT360: min length 256 GPT-4 tokens max length 24576 GPT-4 tokens GPT-4 tiktoken token count: token_count count 5.000000e+06 mean 1.003614e+03 std 1.424231e+03 min 2.570000e+02 25% 4.020000e+02 50% 6.220000e+02 75% 1.050000e+03 max 2.457400e+04 Total count: 5018.07 M tokens

sourceHugging Faceodc-byupdated 10mo agoView on Hugging Face

0likes155downloads

Dataset Card

BEE-spoke-data/TxT360-5M-sample-en

english only sample from LLM360/TxT360:

—min length 256 GPT-4 tokens
—max length 24576 GPT-4 tokens

GPT-4 tiktoken token count:

        token_count
count  5.000000e+06
mean   1.003614e+03
std    1.424231e+03
min    2.570000e+02
25%    4.020000e+02
50%    6.220000e+02
75%    1.050000e+03
max    2.457400e+04

—Total count: 5018.07 M tokens