Datasetpublic

leo20240112/pile-neox-uint16-parts

Tokenized uint16 shard parts for language-model pretraining. Original source: The Pile / NeoX-style preprocessing.

sourceHugging Faceotherupdated 3mo agoView on Hugging Face

0likes6downloads

Dataset Card

Tokenized uint16 shard parts for language-model pretraining.

Original source: The Pile / NeoX-style preprocessing.