Datasetpublic

kjj0/fineweb10B-gpt2

fineweb10B-gpt2 This repo contains the GPT-2 tokens for fineweb10B, just as would be generated by https://github.com/KellerJordan/modded-nanogpt/tree/master (or llm.c). You can download from this repo instead of re-tokenizing to save a couple hours of setup on a new machine.

sourceHugging Facemitupdated 2y agoView on Hugging Face

11likes11kdownloads

root

file.gitattributes2 KBdownload

filefineweb_train_000001.bin190.7 MBdownload

filefineweb_train_000002.bin190.7 MBdownload

filefineweb_train_000003.bin190.7 MBdownload

filefineweb_train_000004.bin190.7 MBdownload

filefineweb_train_000005.bin190.7 MBdownload

filefineweb_train_000006.bin190.7 MBdownload

filefineweb_train_000007.bin190.7 MBdownload

filefineweb_train_000008.bin190.7 MBdownload

filefineweb_train_000009.bin190.7 MBdownload

filefineweb_train_000010.bin190.7 MBdownload

filefineweb_train_000011.bin190.7 MBdownload

filefineweb_train_000012.bin190.7 MBdownload

filefineweb_train_000013.bin190.7 MBdownload

filefineweb_train_000014.bin190.7 MBdownload

filefineweb_train_000015.bin190.7 MBdownload

filefineweb_train_000016.bin190.7 MBdownload

filefineweb_train_000017.bin190.7 MBdownload

filefineweb_train_000018.bin190.7 MBdownload

filefineweb_train_000019.bin190.7 MBdownload

filefineweb_train_000020.bin190.7 MBdownload

filefineweb_train_000021.bin190.7 MBdownload

filefineweb_train_000022.bin190.7 MBdownload

filefineweb_train_000023.bin190.7 MBdownload

filefineweb_train_000024.bin190.7 MBdownload

filefineweb_train_000025.bin190.7 MBdownload

filefineweb_train_000026.bin190.7 MBdownload

filefineweb_train_000027.bin190.7 MBdownload

filefineweb_train_000028.bin190.7 MBdownload

filefineweb_train_000029.bin190.7 MBdownload

filefineweb_train_000030.bin190.7 MBdownload

filefineweb_train_000031.bin190.7 MBdownload

filefineweb_train_000032.bin190.7 MBdownload

filefineweb_train_000033.bin190.7 MBdownload

filefineweb_train_000034.bin190.7 MBdownload

filefineweb_train_000035.bin190.7 MBdownload

filefineweb_train_000036.bin190.7 MBdownload

filefineweb_train_000037.bin190.7 MBdownload

filefineweb_train_000038.bin190.7 MBdownload

filefineweb_train_000039.bin190.7 MBdownload

filefineweb_train_000040.bin190.7 MBdownload

filefineweb_train_000041.bin190.7 MBdownload

filefineweb_train_000042.bin190.7 MBdownload

filefineweb_train_000043.bin190.7 MBdownload

filefineweb_train_000044.bin190.7 MBdownload

filefineweb_train_000045.bin190.7 MBdownload

filefineweb_train_000046.bin190.7 MBdownload

filefineweb_train_000047.bin190.7 MBdownload

filefineweb_train_000048.bin190.7 MBdownload

filefineweb_train_000049.bin190.7 MBdownload

filefineweb_train_000050.bin190.7 MBdownload

filefineweb_train_000051.bin190.7 MBdownload

filefineweb_train_000052.bin190.7 MBdownload

filefineweb_train_000053.bin190.7 MBdownload

filefineweb_train_000054.bin190.7 MBdownload

filefineweb_train_000055.bin190.7 MBdownload

filefineweb_train_000056.bin190.7 MBdownload

filefineweb_train_000057.bin190.7 MBdownload

filefineweb_train_000058.bin190.7 MBdownload

filefineweb_train_000059.bin190.7 MBdownload

filefineweb_train_000060.bin190.7 MBdownload

filefineweb_train_000061.bin190.7 MBdownload

filefineweb_train_000062.bin190.7 MBdownload

filefineweb_train_000063.bin190.7 MBdownload

filefineweb_train_000064.bin190.7 MBdownload

filefineweb_train_000065.bin190.7 MBdownload

filefineweb_train_000066.bin190.7 MBdownload

filefineweb_train_000067.bin190.7 MBdownload

filefineweb_train_000068.bin190.7 MBdownload

filefineweb_train_000069.bin190.7 MBdownload

filefineweb_train_000070.bin190.7 MBdownload

filefineweb_train_000071.bin190.7 MBdownload

filefineweb_train_000072.bin190.7 MBdownload

filefineweb_train_000073.bin190.7 MBdownload

filefineweb_train_000074.bin190.7 MBdownload

filefineweb_train_000075.bin190.7 MBdownload

filefineweb_train_000076.bin190.7 MBdownload

filefineweb_train_000077.bin190.7 MBdownload

filefineweb_train_000078.bin190.7 MBdownload

filefineweb_train_000079.bin190.7 MBdownload

filefineweb_train_000080.bin190.7 MBdownload

filefineweb_train_000081.bin190.7 MBdownload

filefineweb_train_000082.bin190.7 MBdownload

filefineweb_train_000083.bin190.7 MBdownload

filefineweb_train_000084.bin190.7 MBdownload

filefineweb_train_000085.bin190.7 MBdownload

filefineweb_train_000086.bin190.7 MBdownload

filefineweb_train_000087.bin190.7 MBdownload

filefineweb_train_000088.bin190.7 MBdownload

filefineweb_train_000089.bin190.7 MBdownload

filefineweb_train_000090.bin190.7 MBdownload

filefineweb_train_000091.bin190.7 MBdownload

filefineweb_train_000092.bin190.7 MBdownload

filefineweb_train_000093.bin190.7 MBdownload

filefineweb_train_000094.bin190.7 MBdownload

filefineweb_train_000095.bin190.7 MBdownload

filefineweb_train_000096.bin190.7 MBdownload

filefineweb_train_000097.bin190.7 MBdownload

filefineweb_train_000098.bin190.7 MBdownload

filefineweb_train_000099.bin190.7 MBdownload

filefineweb_train_000100.bin190.7 MBdownload

filefineweb_train_000101.bin190.7 MBdownload

filefineweb_train_000102.bin190.7 MBdownload

filefineweb_train_000103.bin105.5 MBdownload

filefineweb_val_000000.bin190.7 MBdownload

fileREADME.md302 Bdownload

kjj0/fineweb10B-gpt2 · main · files are served by the source, never re-hosted here