parameter-golf
parameter-golf-data
Parameter Golf Competition Data
Pre-tokenized FineWeb shards for the OpenAI Parameter Golf competition. Multiple SentencePiece vocab sizes plus a corrected byte-exact Scylla (TokenMonster) tokenization. Free checkpoint persistence API. Zero setup friction.
⚠️ Important: Scylla v1 Deprecated
The original fineweb_scylla/ directory uses the 998-token vocab from PR #1143. That vocab's byte-accounting metadata treated TokenMonster tokens as context-free, which overcounts… See the full description on the dataset page: https://huggingface.co/datasets/LightSpeedUp/parameter-golf-data.parameter-golf
Parameter Golf FineWeb Export
This repository hosts tokenizer-matched export artifacts derived from HuggingFaceFW/fineweb, specifically a 30B subset pulled from the 100B FineWeb cut used for parameter-golf experiments.
The repository contains:
pretokenized training and validation shards under datasets/datasets/
tokenizer artifacts under datasets/tokenizers/
the export manifest at datasets/manifest.json
selected-document metadata at datasets/docs_selected.jsonl
License… See the full description on the dataset page: https://huggingface.co/datasets/willdepueoai/parameter-golf.parameter-golf-sp4096
FineWeb10B SP4096 Dataset for Parameter Golf
This is the SP4096 tokenized version of the FineWeb10B dataset, prepared for the OpenAI Parameter Golf competition.
Contents
fineweb_train_*.bin — Training shards (144 files, ~2.8GB total)
fineweb_val_*.bin — Validation shards (10 files)
tokenizers/fineweb_4096_bpe.model — SentencePiece tokenizer model (4096 vocab)
Format
Binary .bin files contain raw token IDs as uint16 arrays, tokenized with the SP4096… See the full description on the dataset page: https://huggingface.co/datasets/Ryukijano/parameter-golf-sp4096.Parameter-Golf-V17-512Cube-XYZ-Priority-Orbital-Compression
Parameter Golf V17 — 512Cube Solution Bank
This is an English research-control dataset for OpenAI Parameter Golf work. It is not a replacement for FineWeb and must not be used as a substitute training or validation corpus. FineWeb remains the canonical data path for contest scoring.
The dataset captures three things:
V17 512Cube routing concepts translated into English.
Contest and submission guardrails for legal, reproducible BPB reduction.
Screenshot-derived scouting observations… See the full description on the dataset page: https://huggingface.co/datasets/8Planetterraforming/Parameter-Golf-V17-512Cube-XYZ-Priority-Orbital-Compression.parameter-golf-sp4096Parameter-Golf-V15-Patent-Time-Halving
Parameter Golf V15 — Patent Time Halving
English-only working dataset for the 100,000,000-patent runtime audit and 50% time-reduction target.
Status
This package does not claim a measured 0.81 BPB result.
It gives exact calculation rows, audit rows, and Parameter Golf safety rows that can be used for:
clean documentation,
ablation planning,
throughput math,
reproducibility checks,
avoiding invalid val_bpb claims.
Core fixed assumptions
Patent corpus size:… See the full description on the dataset page: https://huggingface.co/datasets/8Planetterraforming/Parameter-Golf-V15-Patent-Time-Halving.