CoolFace
Modelpublic

thomas-schweich/pawn-small-legacy

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face
0likes7downloads
Model Card

PAWN-Small

PAWN (Playstyle-Agnostic World-model Network for Chess) is a causal transformer trained on random chess games. It learns legal moves, board state representations, and game dynamics purely from uniformly random legal move sequences -- no strategic play, no hand-crafted features, no external game databases.

This is the small variant (~9.5M parameters). PAWN is designed as a frozen backbone for parameter-efficient finetuning into player models with arbitrary playstyles.

[GitHub Repository](https://github.com/thomas-schweich/PAWN) -- full source code, training scripts, adapter implementations, and documentation.

All Variants

VariantParametersLink
PAWN-Small~9.5Mthomas-schweich/pawn-small
PAWN (Base)~35.8Mthomas-schweich/pawn-base
PAWN-Large~68.4Mthomas-schweich/pawn-large

Headline Metrics

MetricValue
Legal move rate99.18%
Top-1 accuracy6.75%
Top-5 accuracy27.40%
Val loss3.159

Accuracy Ratios

PAWN is trained on uniformly random chess games, so top-1 accuracy has a hard theoretical ceiling. Ratios above 100% on the unconditioned ceiling indicate the model exploits the outcome token to make non-uniform predictions. The MC conditioned ceiling is an estimate reported as a bracket \[corrected, naive\]; see Accuracy Ceiling Analysis for methodology.

CeilingRatio
Unconditioned (E\[1/N_legal\] = 6.52%)103%
Bayes-optimal conditioned (MC, 128 rollouts = \[6.67, 7.34\]%)92–101%

Probe Results

Linear probes trained on frozen hidden states measure how well the model's internal representations encode board-level features.

ProbeAccuracyDescription
Piece type89.1%Per-square piece type (13 classes x 64 squares)
Side to move100.0%Whose turn it is
Is check94.3%Whether the side to move is in check
Castling rights96.5%KQkq castling availability
En passant square99.8%En passant target square (64 + none)
Material count86.5% (MAE 4.9)Piece counts per type per color
Legal move count30.7% (MAE 7.4)Number of legal moves available
Halfmove clock13.3% (MAE 3.9)Plies since last capture or pawn move
Game phase91.1%Opening / middlegame / endgame

Diagnostic Results

Edge-case diagnostics measure the model's legal move rate in specific tactical situations.

CategoryPositionsLegal Rate
In check100082.4%
Double check7165.1%
Pin restricts movement100086.2%
En passant available94097.1%
Castling legal (kingside)100098.8%
Castling legal (queenside)100098.2%
Castling blocked by check89295.7%
Promotion available100096.2%
Checkmate (terminal)27666.4%
Stalemate (terminal)4153.8%

Architecture

ParameterValue
ArchitectureDecoder-only transformer
d_model256
Layers8
Attention heads4
Head dimension64
d_ff1024
Parameters~9.5M
Vocabulary4,284 tokens
Context length256 tokens
NormalizationPre-norm RMSNorm
FFNSwiGLU (4x expansion)
Positional encodingRotary (RoPE, base 10000)
EmbeddingsFactored (src + dst + promo)
Dropout0.0

Training Details

ParameterValue
Training dataOn-the-fly uniformly random legal games (no external dataset)
ObjectiveNext-token cross-entropy (non-padding positions only)
Total steps100,000
Batch size256
Games seen25,600,000
Learning rate3e-4 (cosine decay with 1,000-step warmup)
OptimizerAdamW (weight decay 0.01)
PrecisionMixed (AMP)
HardwareNVIDIA H200

Usage

Loading the model

python
import torch
from safetensors.torch import load_file
from pawn.config import CLMConfig
from pawn.model import PAWNCLM

cfg = CLMConfig.small()
model = PAWNCLM(cfg).cuda().eval()
weights = load_file("model.safetensors", device="cuda")
model.load_state_dict(weights)

Or load directly from HuggingFace:

python
from pawn.checkpoint import load_backbone_weights
from pawn.config import CLMConfig
from pawn.model import PAWNCLM

weights, config = load_backbone_weights("thomas-schweich/pawn-small")
cfg = CLMConfig.small()
model = PAWNCLM(cfg).eval()
model.load_state_dict(weights)

Finetuning with an adapter

bash
uv run python scripts/train_bottleneck.py \
    --checkpoint thomas-schweich/pawn-small \
    --pgn thomas-schweich/pawn-lichess-full \
    --bottleneck-dim 32 --lr 1e-4 --local-checkpoints

Acknowledgments

PAWN builds on ideas and tools from the following projects and publications:

ComponentReference
TransformerVaswani et al., "Attention Is All You Need", NeurIPS 2017
RMSNormZhang & Sennrich, "Root Mean Square Layer Normalization", NeurIPS 2019
RoPESu et al., "RoFormer: Enhanced Transformer with Rotary Position Embedding", 2021
SwiGLUShazeer, "GLU Variants Improve Transformer", 2020
AdamWLoshchilov & Hutter, "Decoupled Weight Decay Regularization", ICLR 2019
Cosine scheduleLoshchilov & Hutter, "SGDR: Stochastic Gradient Descent with Warm Restarts", ICLR 2017
Mixed precisionMicikevicius et al., "Mixed Precision Training", ICLR 2018
Bottleneck adaptersHoulsby et al., "Parameter-Efficient Transfer Learning for NLP", ICML 2019
LoRAHu et al., "LoRA: Low-Rank Adaptation of Large Language Models", ICLR 2022
FiLMPerez et al., "FiLM: Visual Reasoning with a General Conditioning Layer", AAAI 2018
RoSANikdan et al., "RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation", 2024
Linear probesAlain & Bengio, "Understanding Intermediate Layers Using Linear Classifier Probes", ICLR Workshop 2017
Intrinsic dimensionalityAghajanyan et al., "Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning", ACL 2021
MAIAMcIlroy-Young et al., "Aligning Superhuman AI with Human Behavior: Chess as a Model System", KDD 2020
AlphaZeroSilver et al., "A General Reinforcement Learning Algorithm that Masters Chess, Shogi, and Go through Self-Play", Science 2018
Leela Chess Zerogithub.com/LeelaChessZero/lc0
shakmatygithub.com/niklasf/shakmaty
PyO3github.com/PyO3/pyo3
Lichesslichess.org / database.lichess.org

Citation

bibtex
@software{schweich2026pawn,
  author = {Schweich, Thomas},
  title = {{PAWN}: Playstyle-Agnostic World-model Network for Chess},
  year = {2026},
  url = {https://github.com/thomas-schweich/PAWN},
  license = {Apache-2.0}
}

License

Apache 2.0. See LICENSE.