Datasetpublic

dannyobito/arena-pokerkit-hands

Arena PokerKit Hands (S8 archive — offline practice data) A real-hand dataset of 6-max No-Limit Texas Hold'em poker played by 19 AI agents on the dev.fun Arena Beta — frontier LLMs, OSS solver bots, and equity heuristics — captured during the S8 benchmark run (May 5–6, 2026). What this is and is not. This is a derived, settled-hand archive in a normalized snake_case schema. It is not a mirror of the live /texas/benchmark/status.table (or /texas/pending-actions[].) shape. Use it… See the full description on the dataset page: https://huggingface.co/datasets/dannyobito/arena-pokerkit-hands.

sourceHugging Facecc-by-4.0updated 4mo agoView on Hugging Face

0likes31downloads

Dataset Card

Arena PokerKit Hands (S8 archive — offline practice data)

A real-hand dataset of 6-max No-Limit Texas Hold'em poker played by 19 AI agents on the dev.fun Arena Beta — frontier LLMs, OSS solver bots, and equity heuristics — captured during the S8 benchmark run (May 5–6, 2026).

What this is and is not. This is a derived, settled-hand archive in a normalized snake_case schema. It is not a mirror of the live /texas/benchmark/status.table (or /texas/pending-actions[].) shape. Use it for offline practice, eval-style sanity tests, and reward modeling — the live Poker Eval S5 benchmark generates its own scoring on-server. If you want to ship to the live leaderboard, run examples/agent.py from chenziz/arena-pokerkit against the live competition cmpdk0pt00eawvcaf1es8plw2. This dataset is the offline development companion, not the benchmark itself.

Pair this dataset with the Arena PokerKit GitHub repo to develop and tune your decide() function without burning live-arena API credits. The same decide(table, deadline_s, research_context) signature that runs against this offline file also runs unchanged against the live /texas/benchmark/start flow — but the table shape is reconstructed in eval/local_eval.py, not copied verbatim from a live API row.

Quick start

bash

# 1. Clone the agent skeleton
git clone https://github.com/chenziz/arena-pokerkit
cd arena-pokerkit
uv sync

# 2. Pull this dataset (Path 2a — local data)
huggingface-cli download dannyobito/arena-pokerkit-hands --repo-type dataset --local-dir ./hands

# 3. Synthetic settled-hand replay against agent.decide() — see "Adapter
#    boundary" below for what this does and does NOT verify.
uv run python ../arena-pokerkit-hf/eval/local_eval.py \
  --agent examples/agent.py \
  --dataset ./hands/data/hands.jsonl

# Output:
# Agent: arena-pokerkit-starter
# Hands evaluated:     500
# Decisions evaluated: ~3,000  (~6 seats × 500 hands)
# bb/100:              -1.42  (95% CI: -2.81, +0.04)
# Showdown freq:   0.34
# Win at SD:       0.51
# Decisions/hand:  1.0  (S8 was 1-decision-per-table)

When ready, same code, live Arena (Poker Eval S5):

bash

uv run examples/agent.py --competition-id cmpdk0pt00eawvcaf1es8plw2

Dataset structure

One JSONL row per settled table (one full hand). Each row contains the final board, every seat's hole cards (revealed at settle), final stacks, winners, and per-seat profit in big blinds.

The row shape is normalized snake_case (board_cards, agent_handle, hole_cards, …). It is a derived archive convenient for offline training and analysis — not a verbatim mirror of the live /texas/benchmark/status.table or /texas/pending-actions[]. JSON, which use camelCase and include allowedActions, actionDeadlineAt, recentEvents, etc. that don't apply to a settled (post-hoc) row.

See `data/README.md` for the full data dictionary.

Adapter boundary

eval/local_eval.py reconstructs an Arena-shaped table dict from each settled row (build_pseudo_table) so your decide(table, deadline_s) sees the same field names it would in live play. This is a coarse adapter:

—allowedActions is fabricated (single-decision "what would you do here" probe) since the S8 archive has no per-action stream.
—potChips is approximated from final winner amounts.
—actionDeadlineAt is None (no live deadline pressure).

Treat the offline eval as a sanity test on the shape of your decide() output, not as proof of live API compatibility. For that, run examples/agent.py --dry-run (in-process mock that matches the live endpoint contract) or just run against the live Poker Eval S5 competition.

What's inside

Stat	Value
Source competition	S8 archive (`cmot36zjk00urvu10qxbkz5ig`) — NOT Poker Eval S5
Total hands	500 (balanced sample of S8 snapshot 0427)
Total seat-decisions	~3,000 (~6 seats × 500 hands)
Agents	19 (2 LLM, 17 reference bots / OSS solvers)
Format	6-max No-Limit Hold'em
Buy-in / blinds	2000 chips / 10–20
Sample method	Round-robin balanced (≥20 hands per agent)
Date captured	2026-05-05 → 2026-05-06
Hand-history standard	Arena native JSONL (PHH export available)

Why S8 and not Poker Eval S5?

The current live Poker Eval S5 benchmark (cmpdk0pt00eawvcaf1es8plw2) is the on-server scoring engine for the leaderboard. It does not export per-hand dumps in the same way S8 did. The S8 archive is shipped here as practice data so builders can iterate on decide() without paying live API tax.

When the Arena ships per-action streams (arena_texas_events.snapshot) in a future season, we'll publish a matching dataset. Until then: develop here, score live there.

Agents in this archive

LLMs: GPT-5-5-XHigh, Claude-Opus-4-7.

Bots: Bot-PokerKit-MC, Bot-Slumbot, Bot-DCFR-Leduc, Bot-NFSP-Leduc-Deep, Bot-NFSP-Leduc, Bot-XFP-Leduc, Bot-MCCFR-Leduc, Bot-CFR-Leduc, Bot-Eval7-Equity, Bot-Treys-MC, Bot-PyPoker-MC, Bot-RLCard-NLHE, Bot-RsPoker-CFR, Bot-PHEval-Heuristic, Bot-NeuronPoker-Equity, Bot-PokerBench (Llama-3.1-8B SFT), Bot-Deuces-Equity.

Note: Arena run logs append a short random suffix to each registered handle (Bot-Deuces-Equity-tl, Bot-PokerKit-MC-m6, etc.). The canonical handle is recoverable by stripping a trailing -[a-z0-9]{2,3} segment.

Cleanness

Anchor / null-strategy bots (Anchor-Fold, Anchor-CheckCall, Anchor-AllIn, Anchor-RandomA, Anchor-RandomB) were excluded from this snapshot — they pollute style stats (VPIP / PFR / aggression) and serve no learning signal.

The exporter (scripts/export_from_arena.py) still applies the anchor filter defensively.

Known limitations

—One decision per hand: S8 used handCount=1 per table, so action sequences are not captured. Use this dataset for outcome-level training (Q-learning targets, bb/100 leaderboards, reward modeling) — not for action-level imitation. A future S9+ dump will include per-action streams.
—Fabricated `allowedActions` in offline eval: see "Adapter boundary" above.
—No per-position labels: dealer button is not emitted by the current server. Position-aware metrics are blocked until upstream ships.
—No AIVAT: variance reduction requires unblocked upstream fields.
—Snapshot only: this is the S8 frozen snapshot. Live leaderboards update at arena.dev.fun and may diverge from this static slice.

Citation

bibtex

@dataset{arena_pokerkit_hands_2026,
  author       = {Huang, Dichen and {dev.fun Arena Team}},
  title        = {Arena PokerKit Hands: 6-max NLHE hand histories from 19 AI agents (S8 archive)},
  year         = {2026},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/datasets/dannyobito/arena-pokerkit-hands}
}

License

—Data (data/): CC-BY-4.0
—Scripts (scripts/, eval/): MIT