pkloats/bishop-chess-dataset

Bishop Chess Concepts Dataset

Cleaned, training-ready chess text focused on bishop concepts and strategy, distilled from the `Waterhorse/chess_data` dataset.

At a glance

• —15,677 records · ~18.59M tokens (measured with the GPT-4 cl100k BPE).
• —Game-level-disjoint train/test split (no game leaks across splits), seed 20260708, test fraction 0.02.

Sources (origin):

Schema

One JSON object per line:

{"source":"annotated_pgn","origin":"gameknot","id":"gameknot_game_45108.pgn/1","n_tokens":365,"text":"1. d4 d5 2. c3 Qd6 ..."}

Files

• —`bishop_train.jsonl` / `bishop_test.jsonl` — the loadable split (referenced by the dataset viewer).
• —bishop_train_clean.jsonl — the full cleaned corpus before splitting (train ∪ test).
• —clean_stats.json — per-source record/token counts (raw vs cleaned).
• —quality_eval.json — per-source quality heuristics (token size, explanation/chess density, %English).
• —bishop_counts.json — the "bishop" filter counts per source that defined the selection.
• —split_manifest.json — split parameters, per-source counts, and benchmark-game coverage.

Selection & cleaning

• —Records mentioning "bishop" ≥ 2×, drawn from the 3 highest-quality sources (lichess studies + gameknot games in annotated_pgn, and chess stackexchange Q&A), deduped (the gameknot folder is a subset of annotated_pgn).
• —Cleaning stripped admin PGN headers / URLs / tooling / promo, folded real master-game headers into a compact caption, kept setup FENs / study titles / diagram labels / analysis prose, and converted NAG codes to symbols.
• —Verified by 3 adversarial-review passes and 2 over-removal audits (final residual-noise scan = 0; move content fully preserved).

Usage

from datasets import load_dataset

ds = load_dataset("pkloats/bishop-chess-dataset")
print(ds["train"][0]["text"])

License & provenance

This dataset is a derivative of Waterhorse/chess_data, which aggregates content from multiple sources with different licenses — notably StackExchange content is CC-BY-SA 4.0 (share-alike), while lichess-study and gameknot game annotations carry their own terms. It is released here for research use; downstream users must honor the upstream source licenses. Attribution:

• —Waterhorse/chessdata — https://huggingface.co/datasets/Waterhorse/chessdata
• —Chess StackExchange — https://chess.stackexchange.com (CC-BY-SA)