Pawitt/zero-evaluator

Zero Evaluator High-Variance Chess Positions

This dataset collects 3,809,201 chess positions from three distinct styles of play — engine tournament games, neural-network self-play, and strong human online games. Positions are stored as normalized six-field FEN records for immediate board reconstruction without replaying a game, and every collection balances opening, middlegame, and endgame coverage.

Two of the collections additionally carry static, depth-zero win/draw/loss labels. Variance audits are in variance-report.json, lc0-selfplay-2m-variance.json, lichess-elite-300k-variance.json, and RESULTS.md.

Five configurations are available.

Positions with labels:

• —stockfish_zero_wdl: 1,509,201 CCRL positions with immediate static Stockfish NNUE WDL labels and per-row engine provenance;
• —consensus_wdl: a one-million-position subset carrying calibrated static WDL from both Lc0 and Stockfish, a blended consensus target, and a per-row agreement weight.

Positions without labels:

• —default: 1,509,201 positions from CCRL engine tournament games;
• —lc0_selfplay: 2,000,000 positions from Leela Chess Zero self-play, split by the generation strength of the network that produced them;
• —lichess_elite: 300,000 positions from strong human games on Lichess.

The three unlabeled collections are near-disjoint: of 3,809,201 rows, 3,784,379 FENs are unique and 24,822 appear in more than one collection, almost entirely common opening positions.

Data layout

The release consists of six Zstandard-compressed Parquet files partitioned by:

• —source_split: train or test
• —phase: opening, middlegame, or endgame

data/_manifest.json contains file sizes, row counts, ID bounds, and SHA-256 checksums.

The labeled derivative mirrors the same six partitions under stockfish-zero-wdl/. Its _manifest.json records the source Hub revision, engine identity, binary checksum, row counts, file sizes, and checksums.

Load the dataset

Hugging Face Datasets:

from datasets import load_dataset

positions = load_dataset("Pawitt/zero-evaluator")
print(positions["train"][0]["fen"])

Load the Stockfish-labeled configuration with:

from datasets import load_dataset

positions = load_dataset(
    "Pawitt/zero-evaluator",
    "stockfish_zero_wdl",
)
row = positions["train"][0]
print(row["fen"], row["wdl_win"], row["wdl_draw"], row["wdl_loss"])

For Hive partition columns and streaming Arrow batches, use PyArrow directly:

import pyarrow.dataset as ds

positions = ds.dataset(
    "data",
    format="parquet",
    partitioning="hive",
)
scanner = positions.scanner(
    filter=ds.field("source_split") == "train",
    columns=["fen", "result", "phase"],
    batch_size=8192,
)
for batch in scanner.to_batches():
    pass

When downloading from the Hub first, point ds.dataset at the downloaded data/ directory.

Columns

Each record includes normalized fen, source-game provenance, ply and result, side to move, piece and material statistics, legal-move count, check state, castling mask, and halfmove clock. See FORMAT.md for exact semantics.

The source-game result is provenance metadata. It is not an lc0 depth-zero WDL label.

The stockfish_zero_wdl configuration adds:

WDL is from the perspective of the side to move and always sums to 1000.

Stockfish zero-depth labeling

The labeled configuration was produced with a patched Stockfish command, go depth 0. It performs one immediate NNUE evaluation without tree search, then applies Stockfish's calibrated WDL conversion. This is a static evaluator label, not a searched game-theoretic result or a native three-output neural head.

Provenance:

• —engine: Stockfish dev-20260822-d95a3013-zero-wdl;
• —NNUE: nn-1a298aa575a0.nnue;
• —engine binary SHA-256: 57ca9adcf657338ac907b3c0c667f1f705885c06865603134f23c6e6e402682d;
• —source dataset revision: 7e7d453311882b4b8686aeb885e1c2eb9e2911f9;
• —terminal adjudication: python-chess outcome(claim_draw=True).

Across all rows, the mean WDL is 186.773 / 578.242 / 234.985. Stockfish's static calibration is draw-heavy: the median draw value is 875/1000.

Calibrated consensus WDL

The consensus_wdl configuration holds 1,000,000 positions drawn from the same corpus and labeled independently by two static evaluators, then reconciled into a single target. It is the only configuration with a validation split.

Both engines are read at depth zero, so each label is one immediate network evaluation rather than a search result. Their raw outputs disagree in scale as well as in content, so each is temperature-calibrated before blending:

The consensus target is the equal arithmetic mean of the two calibrated distributions. Stockfish's much larger temperature reflects how sharply its raw static evaluation is distributed compared to Lc0's.

Selection

Positions were admitted only where the two evaluators broadly agree, on two independent criteria:

• —absolute difference in expected score at most 0.5;
• —Jensen-Shannon divergence between the two distributions at most 0.4 bits.

That left 1,383,243 eligible positions, from which 1,000,000 were sampled to a fixed phase balance with seed 91.

Sample weight

Agreement is graded rather than binary. Each row carries

sample_weight = clip((1 - JS) * (1 - |delta_q| / 2), 0.4, 1.0)

so positions where the evaluators concur closely count fully, and marginal ones are retained at reduced weight. No row falls below 0.4. Consumers training on soft targets should weight by this column.

Splits and balance

Added columns

Every WDL triple is from the side to move and sums to 1000.

zero-consensus/manifest.json records the calibration temperatures, engine identities, filter thresholds, sampling seed, per-split and per-phase counts, and a SHA-256 for every shard.

Load it with:

from datasets import load_dataset

positions = load_dataset("Pawitt/zero-evaluator", "consensus_wdl")
row = positions["train"][0]
print(row["fen"], row["wdl_win"], row["sample_weight"])

Self-play and human position collections

Two further collections extend the corpus beyond CCRL engine games. Both hold positions only — no evaluations — in the same schema as the default configuration, so they can be read the same way and labeled independently.

lc0_selfplay

2,000,000 positions sampled from Leela Chess Zero self-play training data published at storage.lczero.org/files/training_data, drawn from 252,686 distinct games across 56 archives and 9 training runs spanning 2018-09 to 2026-08.

Splits correspond to the strength of the network that generated the games, since a collection drawn only from the strongest run would be narrow in exactly the way a varied corpus should not be:

Records were decoded with the Lc0 rescorer without tablebase rescoring or deblundering, and without the position filtering that the Stockfish NNUE conversion path applies. Up to twelve positions were taken per game, allocated across phases in the same 15/60/25 ratio the splits hold.

The test71 run is excluded: it is the Chess960 run, measured at 30-36% Chess960 against at most 1.2% in every other run. Chess960 positions are excluded throughout, so every FEN is legal standard chess.

Source game results are 583,358 white wins, 893,297 draws, and 523,345 black wins — a 44.7% draw rate.

lichess_elite

300,000 positions from 180,510 games in the Lichess Elite Database, which filters the Lichess standard database to games where a 2400+ player faced a 2200+ player. Twelve months are sampled in equal share, spread across 2020-06 to 2025-10, at three positions per game.

This is strong human blitz, not considered classical play: roughly 88% of eligible games are 3+0 or 3+2, 5% rapid, and under 1% classical, with White Elo median 2550. Bullet and ultrabullet are excluded.

Draws are 12.3% of source games here, against 44.7% in lc0_selfplay. Human blitz is substantially more decisive than engine self-play, so this collection supplies sharper and less balanced positions than the other two.

Loading

from datasets import load_dataset

selfplay = load_dataset("Pawitt/zero-evaluator", "lc0_selfplay")
print(selfplay["strong"][0]["fen"])

human = load_dataset("Pawitt/zero-evaluator", "lichess_elite")
print(human["train"][0]["fen"])

Both use the same columns as default; see FORMAT.md. As there, the source game result is provenance metadata and not a position label.

Validation

• —All 1,509,201 source rows were reproduced in Parquet.
• —All six partition counts match the source database.
• —All 26 row groups use Zstandard compression.
• —All six file checksums match the manifest.
• —6,000 sampled FEN records were reconstructed successfully with python-chess.
• —The Stockfish derivative contains exactly 1,509,201 rows in the same six partitions.
• —Every labeled partition matches its manifest row count, byte size, and SHA-256 checksum.
• —Every labeled row has WDL values summing to 1000.
• —All labeled files carry zero_wdl_complete=true and consistent engine provenance.
• —The consensus configuration contains exactly 1,000,000 rows across six shards, each matching its manifest row count and SHA-256 checksum.
• —Every consensus row carries three WDL triples that each sum to 1000, and a sample weight within [0.4, 1.0].
• —The self-play and human collections contain exactly 2,000,000 and 300,000 rows, with no duplicate FEN within either.
• —Their Parquet forms hold FEN sets identical to the SQLite databases they were built from.
• —Sampled rows from both were reconstructed with python-chess: every FEN parses as a legal standard-chess position, and side_to_move, piece_count, in_check, and legal_moves were recomputed and matched.
• —Every split in both holds opening, middlegame, and endgame in a 15/60/25 ratio.

Source

The source is the Leela Chess Zero standard CCRL dataset, published as 2.5 million CCRL 40/40 and 40/4 engine games with an original 80/20 train/test split.

The self-play collection derives from Lc0 training data, decoded with the Lc0 rescorer. The human collection derives from the Lichess Elite Database, itself filtered from the Lichess open database.

The extraction and conversion scripts are included for reproducibility.