sach0312/qwen3-0.6b-nhl-polymarket-grpo-v2

Qwen3-0.6B NHL Polymarket GRPO (v2)

RL-tuned (TRL GRPO) trading policy on NHL moneyline markets, trained from the merged SFT-v2 model (sach0312/qwen3-0.6b-nhl-polymarket-sft-v2-merged) with a fee-net realized-P&L reward on replayed Polymarket NHL games. LoRA r=32 on all attention + MLP projections.

Walk-forward results (the only metric used — never perplexity)

All numbers: identical shares-based fee-net accounting (1.5% taker fee both fills), temporally held-out windows, fills at displayed 1-minute candle prices.

• —This model beats no-trade pooled across windows (≈ +$0.021/game, z ≈ 2.7) but the simple lookup-table teacher stays clearly ahead (+$0.11–0.13/game with edge staking).
• —Edge-proportional staking applied at decode time more than doubles this model's test P&L (+$0.016 → +$0.036) with no retraining; val transfer is weaker (+$0.0213).
• —Parse-fail rate ≈ 9% (intrinsic, not truncation; invalid outputs default to no-trade).
• —Fill realism not modeled (no book depth/latency) — treat as upper bounds.

GRPO v2b (stake-scaled reward) — negative result

Six documented attempts to retrain GRPO with edge-proportional stake in the reward (script). The best run trained to ~step 337/500 before collapse: frac_reward_zero_std → 1.0, grad_norm → 0, ~98% of completions truncated at max length, zero advantage signal. Cancelled per pre-agreed kill criteria. The v2b adapter is not usable. The stake-alignment hypothesis (aligning the RL reward with the winning staking scheme) remains untested end-to-end: TRL GRPO on this 0.6B model with this degenerate-prone reward structure did not train stably at any explored configuration.

Usage

from peft import AutoPeftModelForCausalLM
model = AutoPeftModelForCausalLM.from_pretrained(
    "sach0312/qwen3-0.6b-nhl-polymarket-grpo-v2", torch_dtype="bfloat16")

Prompt format: the sft_v2 state schema (see sach0312/nhl-polymarket-trading), response = JSON {"action": "buy_team0|buy_team1|unwind|no_trade", "stake": 0..1, "reason": "..."}.

Training

• —TRL GRPOTrainer, 500 steps, 8 generations/state, temp 1.0, adaptive entropy (target 1.2), lr 1e-5, a10g-small (~62 min)
• —Reward: fee-net counterfactual P&L per action vs game settlement; malformed −0.05, buy at price >0.95 −0.05, invalid unwind −0.02
• —Dashboard: sach0312/qwen3-nhl-grpo-v2-trackio

Limitations

• —Research prototype; not financial advice; Polymarket NHL markets are thin
• —Price-only settlement replay, 1-minute candles, no order-book depth
• —Policy hedges via unwind only; no partial fills or latency modeling