Pawitt/stockfish-policy

Stockfish Policy Teacher (100k)

Root-policy supervision for 100,000 chess positions, produced by a fixed-node MultiPV Stockfish search. Each row carries a full probability distribution over every legal move at the root — not just the best move — so it can be used as a soft-label teacher for policy distillation.

What makes a row

For every legal root move the generator first evaluates the child position with go depth 0, negating the child score into the root side-to-move perspective. It then searches the root with fixed-node MultiPV; searched scores override the one-ply fallback. Every legal move therefore keeps a positive probability, and moves outside the MultiPV window are still scored rather than dropped.

The stored distribution is:

q(move) = tanh(score_cp / 600)       for centipawn scores
q(move) = sign(mate_distance)        for mate scores
policy(move) = softmax(q / 0.12)

Raw centipawn and mate scores are retained alongside the probabilities, so you can re-apply a different temperature without rerunning Stockfish.

Splits

Splits are stratified over the source plan and drawn from 76,242 distinct source games. Positions are split by game, so no game spans two splits.

Composition

Fields

teacher_moves entries hold move_uci, rank, score_cp, mate_distance, nodes, depth, seldepth, searched, and policy_probability. searched distinguishes a move that received a real MultiPV search from one carrying the depth-0 fallback score; on average 7.49 of the legal moves per position were searched (MultiPV 8). Exactly one of score_cp / mate_distance is non-null. 3,136 positions contain at least one mate score.

Positions have 2–67 legal moves, mean 29.2.

Sample weights

sample_weight = 0 marks a row kept for alignment but carrying no trainable policy signal. Filter with `sample_weight > 0` for training.

In this release exactly 3 rows are zero-weighted, all dead draws by insufficient material. They are labeled normally — they have legal moves and a well-defined static evaluation — but a drawn position teaches nothing about move choice:

8/8/8/8/2b4K/4k3/8/8 w - - 0 1     K vs KB
8/8/7k/8/8/8/8/6K1 b - - 0 1       K vs K
8/3k4/8/8/2K5/3N4/8/8 b - - 0 1    K vs KN

The generator also zero-weights forced single-move positions (none occur here) and writes schema-complete placeholder rows with empty legal_moves for positions with no legal move at all (none occur here).

Generation

python scripts/generate_policy_targets.py \
  --input-plan policy-plan-100k.parquet \
  --output stockfish-policy-100k \
  --stockfish src/stockfish \
  --nodes 10000 --multipv 8 --temperature 0.12 --pv-plies 12 \
  --threads 4 --hash-mb 256 \
  --checkpoint-rows 250 --shard-rows 25000

manifest.json records the full provenance, including a SHA-256 for every shard and for the input plan.

Reproducibility caveat

The run was generated with --threads 4. Stockfish's lazy SMP is not deterministic above one thread, so regenerating this dataset with identical settings will not reproduce it move for move. Measured on a 250-position sample, two runs with identical settings on identical hardware agree on best_move for only ~63% of positions. Treat the labels as one sample from the teacher's search distribution, not as a canonical target.

The run was also produced in two segments by two builds of the same engine source, after the first segment was interrupted:

The two commits are byte-identical in source; only the compile target differs. Cross-platform divergence was measured to be below the run-to-run noise floor of lazy SMP (~67% cross-platform agreement vs ~63% same-platform agreement), so the segments are statistically interchangeable. Both builds are listed in manifest.json under engine_builds.

Note that the Parquet schema metadata embedded in the shards names only the second build for all rows; engine_builds in manifest.json is the accurate record.

Loading

from datasets import load_dataset

ds = load_dataset("Pawitt/stockfish-policy")
train = ds["train"].filter(lambda r: r["sample_weight"] > 0)

row = train[0]
policy = {m["move_uci"]: m["policy_probability"] for m in row["teacher_moves"]}

Provenance and licensing

Labels are generated by Stockfish, which is licensed GPL-3.0, using the nn-1a298aa575a0.nnue network. Root positions are drawn from engine, human, and self-play corpora. If you redistribute this data, check that your use is compatible with the licenses of both Stockfish and the underlying position corpora.