ianlee1996/pokerbench-8max-reasoning-traces

PokerBench 8-max — teacher-distilled reasoning traces

Reasoning traces for 8-max No-Limit Hold'em decisions, distilled from Claude Sonnet 5 on Bedrock in the STaR style, for training small models to reason about poker prices rather than pattern-match to an action.

Method

The teacher is not told the answer. It reasons freely from the same prompt production sends, and a trace is kept only if its conclusion matches the target label. Telling the teacher the target would produce contorted arguments for conclusions it disagrees with, teaching the student to justify rather than to decide.

Two further constraints shape the data:

• —A hard word budget (130 words). Reasoning length sets evaluation cost: a 1000-hand match at 20000 characters per decision runs 30-40 hours. Raw untrained 4B output was 15000-22667 characters with no conclusion inside a 6144-token budget.
• —A mechanical price floor (v3 only). See below.

Fields

Variants

A (`variant_a_heuristic_labels`) — only traces whose conclusion agrees with the hand-written heuristic labeler, trained on that label. Isolates one variable: does reasoning help, holding the label source fixed?

B (`variant_b_teacher_labels`) — every usable trace, trained on the teacher's own decision. Higher ceiling in principle, but changes two variables at once. Trace and label are consistent by construction: the label is the conclusion the reasoning argues for.

traces_all is the unfiltered pool both variants are derived from, so you can re-cut them yourself. Note that in traces_all the label field is the heuristic label and disagreeing rows are retained — filter on agrees before using it for training, or the trace argues for a different action than the label.

v1 vs v3 — the price floor

v1 trained a Qwen3.5-4B that reasoned about prices and then violated them: 3 of 40 calls were below break-even, against 0 of 41 for a 14B DPO adapter on the same spots. (Only 1 of the 3 exceeds a 2pp tolerance.) The cause was the data, not the reasoning: 13% of v1's call rows were themselves below break-even (80 of 593 in A, 130 of 997 in B). The student's 7.5% is below its data's 13.5%, so it did not simply copy the rate -- but it learned some of what was there, and the 14B's data had none of it.

Prompting alone did not fix it — the teacher was already instructed to weigh pot odds and still concluded otherwise — so v3 adds a mechanical check: compute to_call / (pot + to_call) from the prompt and discard any trace concluding "call" when the stated equity is below it, whatever the teacher argued. v3 also front-loads the arithmetic, requiring the first sentence to state required equity against actual equity.

The floor does not merely delete calls — it redistributes them. v3 A holds 354 calls against v1's 593 while raises rose from 1508 to 1786: marginal calls became folds or raises rather than disappearing.

Companion models

• —variant A adapter
• —variant B adapter

Both v1. On a 161-decision regression set drawn from live human-table play they agree with human play 45.3% of the time, against 64.0% for a 14B DPO adapter — these are ablation artifacts, not deployment candidates.

Caveat on the labels

The heuristic labels are not solver output. They come from a hand-written labeler over a preflop strength table, so they encode that labeler's opinions. An earlier revision of the strength table was built at 500 Monte Carlo samples per hand, whose ±3-5pp noise mis-ordered hands differing by 0.5-1pp and so produced wrong top-N% range compositions; it has since been rebuilt at 20000 samples. Traces here were generated after that fix.