Datasetpublic

Ryukijano/parameter-golf-sp4096

FineWeb10B SP4096 Dataset for Parameter Golf This is the SP4096 tokenized version of the FineWeb10B dataset, prepared for the OpenAI Parameter Golf competition. Contents fineweb_train_*.bin — Training shards (144 files, ~2.8GB total) fineweb_val_*.bin — Validation shards (10 files) tokenizers/fineweb_4096_bpe.model — SentencePiece tokenizer model (4096 vocab) Format Binary .bin files contain raw token IDs as uint16 arrays, tokenized with the… See the full description on the dataset page: https://huggingface.co/datasets/Ryukijano/parameter-golf-sp4096.

sourceHugging Faceupdated 5mo agoView on Hugging Face

0likes152downloads

Dataset Card

FineWeb10B SP4096 Dataset for Parameter Golf

This is the SP4096 tokenized version of the FineWeb10B dataset, prepared for the OpenAI Parameter Golf competition.

—fineweb_train_*.bin — Training shards (144 files, ~2.8GB total)
—fineweb_val_*.bin — Validation shards (10 files)
—tokenizers/fineweb_4096_bpe.model — SentencePiece tokenizer model (4096 vocab)

Format

Binary .bin files contain raw token IDs as uint16 arrays, tokenized with the SP4096 SentencePiece model.

Usage

bash

pip install huggingface-hub
python3 download_hf_data.py  # or use huggingface-cli

Source

—Original dataset: HuggingFaceFW/fineweb
—Tokenizer: 4096-token BPE trained on FineWeb sample
—Derived from: kevclark/parameter-golf

License

Same as FineWeb (ODC-BY 1.0)