kjj0/fineweb10B-gpt2
fineweb10B-gpt2 This repo contains the GPT-2 tokens for fineweb10B, just as would be generated by https://github.com/KellerJordan/modded-nanogpt/tree/master (or llm.c). You can download from this repo instead of re-tokenizing to save a couple hours of setup on a new machine.
Upload fineweb_train_000084.bin with huggingface_hub
Upload fineweb_train_000080.bin with huggingface_hub
Upload fineweb_train_000058.bin with huggingface_hub
Upload fineweb_train_000089.bin with huggingface_hub
Upload fineweb_train_000101.bin with huggingface_hub
Upload fineweb_train_000049.bin with huggingface_hub
Upload fineweb_train_000075.bin with huggingface_hub
Upload fineweb_train_000059.bin with huggingface_hub
Upload fineweb_train_000099.bin with huggingface_hub
Upload fineweb_train_000055.bin with huggingface_hub
Upload fineweb_train_000094.bin with huggingface_hub
Upload fineweb_train_000052.bin with huggingface_hub
Upload fineweb_train_000074.bin with huggingface_hub
Upload fineweb_train_000103.bin with huggingface_hub
Upload fineweb_train_000056.bin with huggingface_hub
Upload fineweb_train_000061.bin with huggingface_hub
Upload fineweb_train_000067.bin with huggingface_hub
Upload fineweb_train_000069.bin with huggingface_hub
Upload fineweb_train_000076.bin with huggingface_hub
Upload fineweb_train_000077.bin with huggingface_hub
Upload fineweb_train_000053.bin with huggingface_hub
Upload fineweb_train_000102.bin with huggingface_hub
Upload fineweb_train_000063.bin with huggingface_hub
Upload fineweb_train_000050.bin with huggingface_hub
Upload fineweb_train_000073.bin with huggingface_hub
Upload fineweb_train_000060.bin with huggingface_hub
Upload fineweb_train_000079.bin with huggingface_hub
Upload fineweb_train_000081.bin with huggingface_hub
Upload fineweb_train_000087.bin with huggingface_hub
Upload fineweb_train_000054.bin with huggingface_hub
Upload fineweb_train_000072.bin with huggingface_hub
Upload fineweb_train_000085.bin with huggingface_hub
Upload fineweb_train_000065.bin with huggingface_hub
Upload fineweb_train_000057.bin with huggingface_hub
Upload fineweb_train_000090.bin with huggingface_hub
Upload fineweb_train_000051.bin with huggingface_hub
Upload fineweb_train_000091.bin with huggingface_hub
Upload fineweb_train_000092.bin with huggingface_hub
Upload fineweb_train_000093.bin with huggingface_hub
Upload fineweb_train_000078.bin with huggingface_hub
Upload fineweb_train_000071.bin with huggingface_hub
Upload fineweb_train_000086.bin with huggingface_hub
Upload fineweb_train_000070.bin with huggingface_hub
Upload fineweb_train_000062.bin with huggingface_hub
Upload fineweb_train_000083.bin with huggingface_hub
Upload fineweb_train_000064.bin with huggingface_hub
Upload fineweb_train_000096.bin with huggingface_hub
Upload fineweb_train_000097.bin with huggingface_hub
Upload fineweb_train_000068.bin with huggingface_hub
Upload fineweb_train_000100.bin with huggingface_hub