Ryukijano/parameter-golf-sp4096
FineWeb10B SP4096 Dataset for Parameter Golf This is the SP4096 tokenized version of the FineWeb10B dataset, prepared for the OpenAI Parameter Golf competition. Contents fineweb_train_*.bin — Training shards (144 files, ~2.8GB total) fineweb_val_*.bin — Validation shards (10 files) tokenizers/fineweb_4096_bpe.model — SentencePiece tokenizer model (4096 vocab) Format Binary .bin files contain raw token IDs as uint16 arrays, tokenized with the… See the full description on the dataset page: https://huggingface.co/datasets/Ryukijano/parameter-golf-sp4096.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face