avewright/local-wdl

Local WDL

Value-only chess dataset. Each row is a unique board labeled with official Stockfish 19 `UCI_ShowWDL`. Use wdl as the value target. Do not treat this as MultiPV policy data.

22,305,000 rows on this repo (append-only waves). Source id 4. Compact move vocab (1968). Shards keep a global index: wave 1 is data/shard_000000–000199. Later waves continue.

This is SF19's fishtest-LTC self-play WDL model (eval + remaining material). It is not FIDE/Lichess Elo and not a sigmoid of cp. Official WDL is much more drawish: a start-like +27cp is about 5% White win / 94% draw.

Load

from datasets import load_dataset

ds = load_dataset("avewright/local-wdl", split="train")
train = ds.filter(lambda r: int(r["split"]) == 0)
holdout = ds.filter(lambda r: int(r["split"]) == 1)

# White-absolute value target: [P(White wins), P(draw), P(White loses)]
wdl = train[0]["wdl"]

Honor split. split=1 is a 5% holdout. Do not invent a new hash holdout.

For value training, use wdl with KL / cross-entropy against a 3-class head ordered win/draw/loss. Drop or keep wdl_source==2 terminals explicitly.

# Skip anything that is not official UCI WDL (should be none in this pack)
uci = train.filter(lambda r: int(r["wdl_source"]) == 1)

soft_indices / soft_probs are a single best-move slot (n_soft=1) so the row stacks with the project's soft-cache schema. This pack is not MultiPV. Do not train a policy from these probs as if they were an 8-move distribution.

Columns

Reconstruct a FEN

import chess

ID_TO_SYMBOL = {
    1: "P", 2: "N", 3: "B", 4: "R", 5: "Q", 6: "K",
    7: "p", 8: "n", 9: "b", 10: "r", 11: "q", 12: "k",
}
CASTLE = ((8, "K"), (4, "Q"), (2, "k"), (1, "q"))

def row_to_fen(row) -> str:
    ranks = []
    ba = row["board_array"]
    for rank in range(7, -1, -1):
        empty = 0
        cells = []
        for file in range(8):
            pid = int(ba[rank * 8 + file])
            if pid <= 0:
                empty += 1
                continue
            if empty:
                cells.append(str(empty))
                empty = 0
            cells.append(ID_TO_SYMBOL[pid])
        if empty:
            cells.append(str(empty))
        ranks.append("".join(cells))
    castle = "".join(ch for bit, ch in CASTLE if int(row["castling"]) & bit) or "-"
    ep_i = int(row["ep_square"])
    ep = chess.square_name(ep_i) if 0 <= ep_i <= 63 else "-"
    stm = "b" if int(row["turn"]) else "w"
    return f"{'/'.join(ranks)} {stm} {castle} {ep} 0 1"

Clocks and repetition are unknown. The identity key is the 4-field position (board, side, castling, ep).

Teacher

• —Stockfish 19, EvalFile nn-1a298aa575a0.nnue
• —Binary SHA-256 a18534ee5eab7de770692a1219e477b31a98f90555431df44e16ea2373a0a6e1
• —Full strength, Threads=1, Hash=64, UCI_ShowWDL=true
• —Label: 25,000 nodes / MultiPV=1
• —Play on unlabeled plies: 2,000 nodes
• —Rows without official WDL were dropped. analyze_fail=0, no_wdl=0

ECO starts from Lichess openings: 7,852 unique (A 817 / B 772 / C 1,250 / D 614 / E 357). Then SF19 vs SF19 with epsilon noise.

Do not

• —Convert cp through a sigmoid and call it WDL
• —Mix these shards with MultiPV-8 soft-target out-dirs
• —Treat UCI_Elo on the teacher dump as a human rating (LimitStrength was off)
• —Train policy as if soft_probs were an 8-move teacher
• —Ignore split

Files

• —data/shard_XXXXXX.parquet — 5,000 rows each
• —teacher.json, openings.json, manifest.json