Apppublic

ProCreations/repro-prescriptive-scaling-lm-capabilities

sourceHugging Faceupdated 2mo agoView on Hugging Face

0likes

App README

Reproduction bundle

Reproduction of "Prescriptive Scaling Reveals the Evolution of Language Model Capabilities" (arXiv:2602.15327, OpenReview IkjsHRpuYY), Zhang, Jin, Syrgkanis & Kakade.

The paper estimates capability boundaries — high conditional quantiles of benchmark score as a function of log pre-training FLOPs — by smoothed pinball quantile regression with a monotone saturating sigmoid.

No language model is trained or evaluated here. All six claims are settled by re-deriving the paper's own statistics from its released evaluation tables, using an independent re-implementation of the estimator written from the equations in Section 2.1. The authors' released code was additionally re-run end to end as a cross-check; both sets of numbers appear side by side on every claim page.

bash

pip install -r requirements.txt
python3 run_analysis.py     # -> outputs/*.json  (~15 min, CPU only)
python3 validate_evidence.py
python3 verify_manifest.py

Layout

path	contents
`psl_core.py`	independent re-implementation of the estimator (paper Section 2.1)
`run_analysis.py`	all six claims + correctness gates + negative controls
`authors/`	vendored slices of the authors' released evaluation tables
`outputs/`	every number quoted in the logbook, as JSON
`pages/`	the logbook

Data provenance

authors/oll_v2_slim.csv is a column subset of the Open LLM Leaderboard table from the pinned prescriptive-scaling repository (4576 leaderboard rows, 3663 with a compute proxy). authors/aime-2025.csv and authors/math-500.csv are the Artificial Analysis exports shipped in the same repository. authors/manifests/ are the budgeted design manifests emitted by the authors' pinned sweep-alpha implementation.