leo20240112/pile-neox-uint16-parts
Tokenized uint16 shard parts for language-model pretraining. Original source: The Pile / NeoX-style preprocessing.
06
Upload folder using huggingface_hub (part 8)
Upload folder using huggingface_hub (part 7)
Upload folder using huggingface_hub (part 6)
Upload folder using huggingface_hub (part 5)
Upload folder using huggingface_hub (part 4)
Upload folder using huggingface_hub (part 3)
Upload folder using huggingface_hub (part 2)
Upload folder using huggingface_hub
initial commit