jevonmao/llama31-8b-poker-mix-v1-step10k

PokerLlama-4

PokerLlama-4 is a supervised fine-tune of Llama-3.1-8B-Instruct for heads-up no-limit Texas hold'em at 200 big blinds. It distills the GTO Wizard equilibrium solver's mixed strategies into a chat-template language model and supports both direct action emission and chain-of-thought reasoning at inference time.

It reaches 83.96% top-1 action-type accuracy on a 31,105-decision held-out evaluation split — a new project record and the highest of any 8B model we evaluated, outperforming both open-source poker fine-tunes and zero-shot frontier reasoning models on postflop action prediction.

• —Project landing page: https://jevonmao.github.io/cs153-project/
• —Code & evaluation harness: https://github.com/jevonmao/cs153-project
• —Training corpus: `jevonmao/poker-sft-mix-v1`

PokerLlama-4: When Distilling a Nash Policy Beats the Benchmark but Loses the Game. Stanford CS 153 Final Project, 2026.

Model details

This repository contains the merged FP16 weights at the converged checkpoint. No LoRA adapter file is included — the merge is in-place, so the model loads exactly like the base via AutoModelForCausalLM.

Intended use

PokerLlama-4 is a research artifact accompanying the project's final report. It is intended for:

• —Reproducing the project's evaluation results on the held-out HoldemEval-31k split and live-play benchmarks.
• —Studying GTO-policy distillation into small language models — the trade-off between matching the equilibrium action distribution offline and achieving expected value against opponents who deviate from equilibrium.
• —Tool-use / function-call experiments in a poker domain — the model emits well-formed preflop_gto tool calls with 99.9% argument correctness.

It is not intended for gambling, real-money play, or any context in which the output is acted on without human oversight.

How to use

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "jevonmao/llama31-8b-poker-mix-v1-step10k"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

SYSTEM_PROMPT = (
    "You are an expert No Limit Texas Hold'em poker assistant. "
    "Read the hand history carefully and decide the next action. "
    "Respond with a single action wrapped in <action></action> tags."
)

USER_PROMPT = """\
Game: HUNL 200BB
Your position: SB
Your hand: As Kh
Pot: 150
Effective stack: 19,900
Action so far: SB posts 50, BB posts 100.

Legal actions: fold, call 100, raise [to] [200..19900]
"""

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user",   "content": USER_PROMPT},
]

inputs = tokenizer.apply_chat_template(
    messages, return_tensors="pt", add_generation_prompt=True
).to(model.device)

with torch.inference_mode():
    out = model.generate(inputs, max_new_tokens=32, do_sample=False)

print(tokenizer.decode(out[0, inputs.shape[1]:], skip_special_tokens=True))
# e.g. <action>raise to 225</action>

For chain-of-thought inference, replace the system prompt's final sentence with the project's CoT mode marker (see eval/ in the source repository).

Greedy decoding is recommended

All reported results use greedy decoding (do_sample=False). Temperature sampling at T = 1.0 recovers the mixed-strategy character of the distilled policy at the cost of action-label accuracy and is appropriate for live play against opponents who themselves play balanced strategies.

Training data

PokerLlama-4 is fine-tuned on `jevonmao/poker-sft-mix-v1`, a 606,630-record four-way mix of equilibrium-solver decisions and audited chain-of-thought reasoning traces:

A teacher-rationale hand-class audit rejects 21.7% of solver-source CoT records (15,054 / 69,438) for categorical hand-class errors before training.

Evaluation

Offline action accuracy (HoldemEval-31k)

The +22.3 pt margin over the closest 8B baseline and the +53.3 pt margin over the published PokerBench-8B fine-tune are isolated to the corpus and training recipe — every model shares the Llama-3.1-8B backbone.

Against frontier reasoning models (postflop split, n = 2,044)

A separate predecessor checkpoint (PokerLlama-2, a postflop specialist on the same project line) was evaluated against three frontier reasoning baselines on a 2,044-decision postflop-only split:

Specialized distillation clears the frontier zero-shot ceiling (≈ 69%) on postflop action prediction at full parse-rate reliability.

Live self-play (rlcard heads-up 200 BB)

Greedy decoding of the distilled policy approximates the modal action of the solver's mixed strategy. Against a balanced opponent this is approximately Nash; against opponents who deviate from equilibrium (PokerBench-8B over-folds, base Llama rarely folds) the modal-of-mixed-strategy policy does not maximize EV. This trade-off is the central finding of the accompanying report — see §Diagnostics on the project landing page for the full analysis.

Limitations

• —Mixed-strategy mean targeting. The training objective drives the model toward the equilibrium policy. Greedy decoding therefore picks the modal equilibrium action, which is not always the EV-maximizing response to a non-equilibrium opponent. This is a feature for Nash-quality evaluation and a constraint for live-play exploitation.
• —Bet-space discretization. The live-play simulator discretizes bets into five canonical sizes; the training distribution covers a wider continuous range. Behavior on bet sizes outside the simulator's discretization may not reflect what the model would do given a continuous-bet interface.
• —Game format scope. The model is trained exclusively on heads-up no-limit Texas hold'em at 200 BB depth. It has not been evaluated at other stack depths, other player counts, or other variants.
• —Teacher rationales on the GTO Wizard slice. The hand-class audit applies only to the solver chain-of-thought source; the GTO Wizard chain-of-thought slice has no per-spot hand-class ground truth and is left unaudited. A residual teacher-error rate is assumed.

Responsible use

This model is a research artifact for a coursework project. It is not intended for use in real-money gambling or in any setting where a mis-prediction could cause financial harm. The Llama-3.1 community license governs downstream use of the base weights and applies to this fine-tune in full.

Citation

@misc{mao2026pokerllama,
  title  = {PokerLlama-4: When Distilling a Nash Policy Beats the Benchmark but Loses the Game},
  author = {Mao, Jevon},
  year   = {2026},
  note   = {Stanford CS 153 Final Project},
  url    = {https://jevonmao.github.io/cs153-project/}
}

Acknowledgements

• —GTO Wizard for the live solver-decision data interface.
• —Meta for releasing Llama-3.1-8B-Instruct.
• —Stanford CS 153 course staff for compute access on the H100 cluster.