datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
chess-evaluations
Chess Evaluations Dataset
This dataset contains chess positions represented in FEN (Forsyth-Edwards Notation) along with their evaluations and next moves for tactical evals. The dataset is divided into three configurations:
tactics: Includes chess positions, their evaluations, and the best move in the position.
randoms: Contains random chess positions and their evaluations.
chess_data: General chess positions with evaluations.
This is an in progress dataset which contains millions… See the full description on the dataset page: https://huggingface.co/datasets/ssingh22/chess-evaluations.ChessQA-Benchmark
ChessQA-Benchmark
CSSLab, Department of Computer Science, University of Toronto
Code: GitHub
Abstract
Chess provides an ideal testbed for evaluating the reasoning, modeling, and abstraction capabilities of large language models (LLMs), as it has well-defined structure and objective ground truth while admitting a wide spectrum of skill levels. However, existing evaluations of LLM ability in chess are ad hoc and narrow in scope, making it difficult to accurately… See the full description on the dataset page: https://huggingface.co/datasets/wieeii/ChessQA-Benchmark.chess_checkmate_in_one_big_bench
Dataset Card for BIG-Bench Checkmate In One Move
This is an adapted version of the BIG-Bench Checkmate in One Move task originally made by Nitish Keskar (nkeskar@salesforce.com).
There also exists a dataset with the image representation of the board positions before the checkmate move.
The original task description is the following:
The goal of this task is to probe the ability of language models to play chess in standard algebraic notation (SAN). The input to the model is a… See the full description on the dataset page: https://huggingface.co/datasets/alonsosilva/chess_checkmate_in_one_big_bench.ChessAlpacaPrediction
Chess Position Evaluation Dataset
This dataset contains chess positions in FEN format along with their Stockfish evaluations. It is formatted in the Alpaca instruction-following format.
Dataset Structure
Each example contains:
instruction: A prompt to evaluate the chess position
input: The FEN string representing the chess position
output: The evaluation score in pawn units (positive values favor White, negative values favor Black)
Example
{… See the full description on the dataset page: https://huggingface.co/datasets/dsaunders23/ChessAlpacaPrediction.chess-debate-puzzles
Chess Debate Puzzles
A stratified sample of Lichess mid/endgame chess puzzles annotated with Stockfish-evaluated
moves across ten centipawn-quality bands. Designed for experiments in the spirit of
AI Safety via Debate (Irving et al., 2018), where two AI
agents argue for different moves and a judge must identify the objectively better one.
Motivation
Debate as an alignment technique asks whether a human (or AI) judge can identify the correct
answer when two agents argue… See the full description on the dataset page: https://huggingface.co/datasets/kvoudouris/chess-debate-puzzles.