parameter-golf
parameter-golf-data
Parameter Golf Competition Data
Pre-tokenized FineWeb shards for the OpenAI Parameter Golf competition. Multiple SentencePiece vocab sizes plus a corrected byte-exact Scylla (TokenMonster) tokenization. Free checkpoint persistence API. Zero setup friction.
⚠️ Important: Scylla v1 Deprecated
The original fineweb_scylla/ directory uses the 998-token vocab from PR #1143. That vocab's byte-accounting metadata treated TokenMonster tokens as context-free, which overcounts… See the full description on the dataset page: https://huggingface.co/datasets/LightSpeedUp/parameter-golf-data.parameter-golf
Parameter Golf FineWeb Export
This repository hosts tokenizer-matched export artifacts derived from HuggingFaceFW/fineweb, specifically a 30B subset pulled from the 100B FineWeb cut used for parameter-golf experiments.
The repository contains:
pretokenized training and validation shards under datasets/datasets/
tokenizer artifacts under datasets/tokenizers/
the export manifest at datasets/manifest.json
selected-document metadata at datasets/docs_selected.jsonl
License… See the full description on the dataset page: https://huggingface.co/datasets/willdepueoai/parameter-golf.Parameter-Golf-V17-512Cube-XYZ-Priority-Orbital-Compression
Parameter Golf V17 — 512Cube Solution Bank
This is an English research-control dataset for OpenAI Parameter Golf work. It is not a replacement for FineWeb and must not be used as a substitute training or validation corpus. FineWeb remains the canonical data path for contest scoring.
The dataset captures three things:
V17 512Cube routing concepts translated into English.
Contest and submission guardrails for legal, reproducible BPB reduction.
Screenshot-derived scouting observations… See the full description on the dataset page: https://huggingface.co/datasets/8Planetterraforming/Parameter-Golf-V17-512Cube-XYZ-Priority-Orbital-Compression.parameter-golf-sp4096
FineWeb10B SP4096 Dataset for Parameter Golf
This is the SP4096 tokenized version of the FineWeb10B dataset, prepared for the OpenAI Parameter Golf competition.
Contents
fineweb_train_*.bin — Training shards (144 files, ~2.8GB total)
fineweb_val_*.bin — Validation shards (10 files)
tokenizers/fineweb_4096_bpe.model — SentencePiece tokenizer model (4096 vocab)
Format
Binary .bin files contain raw token IDs as uint16 arrays, tokenized with the SP4096… See the full description on the dataset page: https://huggingface.co/datasets/Ryukijano/parameter-golf-sp4096.parameter-golf-sp4096Parameter-Golf-V8-WebSignal-BPE-Entropy-MicroMix
Parameter-Golf-V8-WebSignal-BPE-Entropy-MicroMix
Short Description
Auxiliary micro-mix dataset for OpenAI Parameter Golf: FineWeb-style web-signal filtering, BPE-aware payload compression, boilerplate suppression, privacy-safe minimization, and exact compact-state reasoning for BPB-oriented probes.
Extended Description
This is a V8 auxiliary micro-mix dataset for OpenAI Parameter Golf experiments. It is built from the strongest useful themes in the… See the full description on the dataset page: https://huggingface.co/datasets/8Planetterraforming/Parameter-Golf-V8-WebSignal-BPE-Entropy-MicroMix.