datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
chess_owtowt-gpt2bpe-9B
owt-gpt2bpe-9B
OpenWebText (from apollo-research/Skylion007-openwebtext-tokenizer-gpt2), pre-tokenized with the gpt2bpe tokenizer (vocab 50,257)
and packaged as flat uint16 token-id .bin files for fast memmap training.
file
split
tokens
train.bin
train
9,015,870,208
val.bin
val
20,000,000
train and val are disjoint held-out partitions. Each .bin is a raw
little-endian uint16 stream (no header); token count = filesize / 2, and
train.meta.json/ val.meta.json carry… See the full description on the dataset page: https://huggingface.co/datasets/alexkstern/owt-gpt2bpe-9B.