ianlee1996/pokerbench-qwen3-14b-lora-mixed-v2

PokerBench Qwen3-14B + LoRA (Mixed v2 — sized)

Successor to `pokerbench-qwen3-14b-lora-mixed`. Same training recipe (50/50 paper-format + production-PE-format, 250 steps on top of the Stage 1 paper LoRA), with one critical bug fix:

Fixed: PE training labels for raise/bet actions previously dropped the sizing — every raise label was the bare string "raise". The model trained on this never learned to emit a raise amount, so production deployments saw default_pct: None whenever the model chose to raise. The fixed PE jsonl recovers the BB amount from the prompt's raise pct=N(Mbb,...) line and rewrites the label as "raise N".

Why this matters

The action-accuracy headline is unchanged because the eval scoring folds raise/bet/all-in into a single "aggressive" kind. The sizing is what production deployment cares about.

What changed in the data

gen-pe-format-dataset.ts used row.correctDecision (action kind only) as the label. The fix is post-hoc — scripts/fix_pe_raise_sizing.py reads each prompt, extracts BB amount from the raise pct=N(Mbb, ~M) / raise pct=N(Bet M, ~M) line, and rewrites the label.

After fix:

• —3804 raise/bet records → 0 bare "raise" remaining
• —109 distinct raise sizes (top: raise 14.0, raise 13.0, raise 25.0, raise 2.5, raise 10.0)

Sample outputs

prompt: "...preflop SB facing UTG raise 2.5..."
v1 (old):    "raise"            ← parser sees no number, default_pct: None
v2 (this):   "raise 14.5"       ← parser gets 14.5 BB, action complete

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-14B",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model = PeftModel.from_pretrained(base, "ianlee1996/pokerbench-qwen3-14b-lora-mixed-v2")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-14B")

system_prompt = (
    "You are a specialist in playing 6-handed No Limit Texas Holdem. "
    "Output ONLY the optimal action with no explanation. "
    "Valid formats: 'fold', 'check', 'call', 'bet N', 'raise N', 'all-in'."
)

# Inference at temp 0.1, top-p 0.95, max_tokens=16 (raise output is 5-6 tokens)

Training

• —Base: Qwen/Qwen3-14B
• —Stage 1 LoRA: 04_qwen3_14b_lora_full_data_1epoch/checkpoint-4375 (60k+500k paper-format SFT, 1 epoch, EM 90.07%)
• —Stage 2 LoRA (this): 250 steps on 50/50 mix of paper + production-PE-fixed jsonl
• —LoRA: r=32, alpha=64, dropout=0.05, target=all-linear
• —Optimizer: pagedadamw8bit, LR 5e-5, cosine schedule, warmup 0.03
• —Batch: effective 128 (4 per device × 32 grad accumulation)
• —Hardware: 1× NVIDIA RTX PRO 6000 Blackwell (96 GB)
• —Time: ~1h 38min

Reproducibility

Full pipeline at https://github.com/IanLiYi1996/PokerBench:

# 1. Fix the existing PE jsonl in-place
.venv/bin/python -m scripts.fix_pe_raise_sizing \
    data/pe_format/mixed_paper_pe_50_50.jsonl \
    data/pe_format/mixed_paper_pe_50_50_fixed.jsonl

# 2. Re-train the SFT mixed checkpoint on the fixed data
.venv/bin/python -m scripts.train --config configs/experiments/09_qwen3_14b_lora_mixed_v2_sized.yaml

Citation

@inproceedings{zhuang2025pokerbench,
  title={PokerBench: Training Large Language Models to become Professional Poker Players},
  author={Zhuang, Richard and Gupta, Akshat and Yang, Richard and Rahane, Aniket and Li, Zhengyu and Anumanchipalli, Gopala},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2025},
  url={https://arxiv.org/abs/2501.08328}
}

License

Apache-2.0, matching Qwen/Qwen3-14B and the PokerBench dataset.