Modelpublic

chorcat/rukh-qwen3-pgn-qlora

sourceHugging Faceapache-2.0updated 5d agoView on Hugging Face

0likes23downloads

Model Card

chorcat/rukh-qwen3-pgn-qlora

A QLoRA adapter that fine-tunes `Qwen/Qwen3-0.6B` on chess games written as PGN text. It exists to answer one question with a number: is a general language model, fine-tuned on the same games, a substitute for a model built for the job?

It is part of Rukh, a course that builds a chess language model end to end. The model it is compared against, and how: https://lab.rukh.borjaglez.com.

How it was trained


Base model	`Qwen/Qwen3-0.6B` (600,637,440 parameters)
Precision	4-bit NF4 (QLoRA)
Rank `r`	16, `alpha` 32
Adapted matrices	`k_proj, o_proj, q_proj, v_proj`
Trainable parameters	4,587,520 (0.764 %)
Training samples	100,000 games
Steps	1,500
Weights on the device	851 MB
Peak memory while training	2361 MB

The games are the same ones the project's own decoder trains on, from the same month, held out against the same validation split. Each is rendered as the movetext a human would paste into a chess site, with the two ratings as PGN tags:

[WhiteElo "2015"] [BlackElo "2028"]
1. e4 c6 2. d4 d5 3. exd5 cxd5 4. Bd3 Nf6 ... 1-0

Results

Measured with the project's own harness -- the same validation positions, the same puzzle split and the same Stockfish ladder as every other stage -- so the two sides differ in their representation and in nothing else.

Metric	Value
Legal moves written, no mask	62.50 %
Top-1 next move	12.5 %
Puzzles solved	0.9 %
Estimated Elo	< 807 (one-sided 95 % bound; every game lost)

How its answers failed

A model whose vocabulary is the set of legal moves can only be wrong one way: a legal move in the wrong position. A model writing SAN has three more.

Outcome	Share
legal	62.50 %
illegal here	33.70 %
not a move at all	2.90 %
ambiguous SAN	0.90 %
nothing written	0.00 %

In the course

—Built in M4 · Fine-tuning: los labs, the lesson that runs every command behind this repository.
—Measured in the single results table as the stage qwen3-pgn-qlora: every stage of the course, the same suite, the same day.
—Bring it to the paths the configs read: uv run rukh pull qwen3-pgn-qlora.

Licence

apache-2.0.