ash1402/ai-sports-prediction-index
AI Sports Prediction Index Line-blind LLM forecasts of every MLB game and UFC fight, graded in public against the final result and the closing market. One row per model per graded event: win probabilities, score projections, picks, results, closing consensus prices, and per-pick closing-line value. This is a contamination-impossible LLM evaluation. Every event happened after every model's training cutoff. Every prediction carries a timestamp from before first pitch. The models… See the full description on the dataset page: https://huggingface.co/datasets/ash1402/ai-sports-prediction-index.
AI Sports Prediction Index
Line-blind LLM forecasts of every MLB game and UFC fight, graded in public against the final result and the closing market. One row per model per graded event: win probabilities, score projections, picks, results, closing consensus prices, and per-pick closing-line value.
This is a contamination-impossible LLM evaluation. Every event happened after every model's training cutoff. Every prediction carries a timestamp from before first pitch. The models never saw a betting line: each one receives the identical stats-only data packet and returns probabilities. Then the games are played. There is no way for any of this to leak into training data before the forecast is made, which makes it one of the few real-world, time-stamped, leakage-free forecasting benchmarks available, and it grows every day.
The live leaderboard built from these rows: https://predictedsports.com/leaderboard. Methodology: https://predictedsports.com/accuracy. Any two pickers head-to-head: https://predictedsports.com/vs.
What models are in it
Frontier models from Anthropic, OpenAI, Google, xAI, DeepSeek, Moonshot and Z.ai, called via OpenRouter with the same prompt and packet. models.csv lists each model's active window and graded-game count. Run windows differ per model (the roster changes on the 1st of each month; new models are provisional until their first full month), so always window comparisons using models.csv.
Grading, in one paragraph
The winner grade is the higher-probability side against the final score. Brier and log loss score p_home (MLB) or p_a (UFC) against the result. ROI is one flat unit on the pick at the closing implied probability (win pays 1/p - 1, loss is -1, pushes skipped); closing prices include vig, so 0% is the market's bar. The closing market is the latest pre-game capture, a multi-book consensus where available, otherwise exchange prices. clv_pp is how far the market moved toward the model's side between pick time and close, in percentage points; positive means the pick beat the close.
A standard-library Python script that recomputes the whole leaderboard from these CSVs and matches the live site is included in the GitHub mirror: https://github.com/ashjanderson/ai-sports-prediction-index
Honest caveats
The dataset starts 2026-06-30 and is young; several differences between models are not yet statistically separable. Not included, on purpose: the input data packet (that pipeline is ours), pick-time price series (they grade our subscription products; clv_pp is the published derivative), scraped expert picks, and anything user-related.
Refresh and citation
Refreshed weekly, versioned release on the 1st of each month. License CC BY 4.0: use it for anything, link back. If you want the predictions live rather than weekly, we run a real-time prediction API for every sport on the site: hello@predictedsports.com
@misc{predictedsports2026index,
title = {AI Sports Prediction Index: line-blind LLM forecasts of MLB and UFC, graded},
author = {{PredictedSports}},
year = {2026},
url = {https://predictedsports.com/leaderboard}
}