homerquan/nemotron-boardgame-answer-lora-b4-safe-final

Nemotron Boardgame Answer LoRA B4 Safe Final

Overview

nemotron-boardgame-answer-lora-b4-safe-final is a specialist PEFT/LoRA adapter for board-game move selection and BoardGameBench-style answer generation. It continues the earlier `homerquan/nemotron-boardgame-answer-lora-b4-safe-2000` adapter with preference optimization: first DPO on reviewed move pairs, then GRPO on prompt+reward maps generated from BoardGameBench states.

The headline change is better benchmark stability after DPO + GRPO: on the 22-game BoardGameBench default curriculum, this adapter reached 286.0 BRI, up from 271.9 BRI for the earlier b4-safe-2000 run, while reducing forfeits from 3 to 0.

Author: Homer Quan Project: BoardGameBench

Training Datasets

The data used in this training lineage is published here:

• —homerquan/boardgamebench-answer-sft: the full answer-only SFT corpus.
• —homerquan/boardgamebench-answer-dpo: a reviewed DPO sample/pilot set used for the preference continuation stage.
• —homerquan/boardgamebench-answer-grpo: a reviewed GRPO sample/pilot prompt+reward set; the final GRPO run used a 512-example pilot slice from this release.

DPO and GRPO are intentionally published as sample/pilot datasets only, while the SFT release is the large answer-only corpus.

What This Is

This is a LoRA adapter for nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16. It is not a standalone model. Load it with the compatible Nemotron 3 Nano base model through PEFT.

The adapter is narrow by design:

• —It is intended for compact deterministic board-game prompts.
• —It should be given rules, board state, side to move, and legal moves whenever possible.
• —It is optimized for concise move-label answers, not broad assistant behavior.

DPO + GRPO Improvement

The earlier adapter already learned the board-game answer format through SFT. The final adapter adds preference and reward optimization:

• —DPO: preference continuation from the most recent SFT-only adapter.
• —GRPO: reward-map training on BoardGameBench prompt+reward data.

In the benchmark snapshot below, DPO + GRPO preserved the single win, removed all move-format forfeits, and improved BRI.

Benchmark details for the final adapter are included in this repository at:

benchmarks/boardgamebench_grpo_2026-05-08.json

Per-Game Snapshot

These are benchmark-specific results, not a general-purpose reasoning claim. Scores can vary with prompt format, decoding settings, base model revision, hardware, and benchmark harness version.

Loading Example

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"
adapter_id = "homerquan/nemotron-boardgame-answer-lora-b4-safe-final"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    device_map="auto",
    torch_dtype="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

Example prompt shape:

Game: <game name>
Rules: <rules>
You are side <X/O>. Choose one legal move label exactly as written.
Legal moves: <comma-separated legal moves>
Current state:
<board>

Return the best move only.

Intended Use

Use this adapter for:

• —BoardGameBench evaluations
• —board-game move-label generation
• —compact deterministic game experiments
• —comparing SFT, DPO, and GRPO behavior on game prompts

Out of scope:

• —general chat or instruction following
• —factual QA, medical, legal, financial, or safety advice
• —autonomous real-world decisions
• —games where rules or legal moves are omitted or ambiguous

Training Details

• —Base model: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
• —Adapter type: LoRA
• —PEFT task type: causal language modeling
• —Rank: 8
• —LoRA alpha: 16
• —LoRA dropout: 0.0
• —Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• —Optimization stages: SFT adapter startpoint, then DPO, then GRPO
• —PEFT version: 0.19.1

Limitations

• —The adapter depends on the compatible Nemotron 3 Nano base model.
• —It may choose weak or illegal moves when the prompt is underspecified.
• —It is benchmark-specialized and should not be treated as a general-purpose assistant.
• —The included benchmark is a snapshot, not a proof of solved game play.

Citation

If you use this adapter in evaluations or derivative work, please cite BoardGameBench:

@software{quan_boardgamebench_2026,
  author = {Quan, Homer},
  title = {BoardGameBench},
  year = {2026},
  url = {https://github.com/homerquan/BoardGameBench}
}

Author

Author: Homer Quan