homerquan/boardgamebench-answer-grpo

BoardGameBench Answer GRPO Dataset

This dataset contains 10,000 BoardGameBench prompt/reward examples generated for GRPO-style reinforcement learning on board-game move selection.

The final nemotron-boardgame-answer-lora-b4-safe-final adapter used this reviewed GRPO corpus after SFT and DPO. For that final pilot run, training used the first 512 examples from grpo_train.jsonl; the full 10k reviewed set is published here for reproducibility and follow-up training.

Format

The main training file is:

• —grpo_train.jsonl

Each row includes:

• —id: stable example id.
• —prompt: board-game decision prompt ending at ### Response:.
• —reward_map_json: serialized reward map over legal move labels.
• —source: dataset source id.
• —split: split name.
• —metadata_json: serialized source metadata.

The reward map records legal moves, move ranks, engine/search signals, rollout survival signals, and normalized reward values. It is intended for trainers that sample completions and score exact legal move prefixes.

Additional Files

• —grpo_prompts.jsonl: prompt-only view.
• —grpo_rewards.jsonl: reward-map view.
• —states_with_rewards.jsonl: source state and reward records.
• —manifest.json: generation settings and row counts.
• —validation_report.json: validation summary.
• —progress.json: generation progress metadata.

Included Games

The corpus balances examples across:

• —Connect Four
• —Gomoku 19x19
• —Breakthrough 6x6
• —Dots and Boxes 3x3
• —Othello 6x6
• —Othello 8x8
• —Hex 7x7

Example

{
  "id": "9b7ca5f114e15bf786c86be7",
  "prompt": "Below is an instruction that describes a board-game reasoning task...",
  "reward_map_json": "{\"game_id\": \"connect_four\", \"legal_moves\": [\"C1\", \"C2\", \"C3\", \"C4\", \"C5\", \"C6\", \"C7\"]}"
}

Loading

from datasets import load_dataset

ds = load_dataset("homerquan/boardgamebench-answer-grpo", data_files="grpo_train.jsonl")
print(ds["train"][0])

Training Lineage

The final adapter was trained as:

1. 1.SFT on homerquan/boardgamebench-answer-sft.
2. 2.DPO on homerquan/boardgamebench-answer-dpo.
3. 3.GRPO on this prompt/reward dataset.

Source Project

The board positions, legal move sets, and engine-guided reward maps come from:

https://github.com/homerquan/BoardGameBench

Notes

The GRPO reward parser used exact legal move prefixes. This keeps the objective focused on choosing legal and stronger board-game moves rather than producing long free-form explanations.