licensedbets-open/probability-calibration-lab

Probability Calibration Lab

An original, dependency-free browser workbench for evaluating binary probability forecasts. Paste a labeled CSV batch, inspect rejected rows, compare Brier score and clipped log loss, and examine equal-width reliability bins.

The default 16 observations are entirely synthetic, manually constructed for demonstration. They are not measured model performance, real odds, betting predictions or a recommendation to gamble. No model is trained or hosted.

Input and validation

Use two unquoted CSV fields: probability,outcome. An optional header is supported. Probabilities must be finite numbers from 0 to 1; outcomes must be exactly 0 or 1. Invalid rows are excluded with explicit reasons and original line numbers. Limits are 10,000 nonblank data rows and 1 MB of text. Computation stays in your browser; the app has no network requests or persistent storage.

Metrics

• —Brier score: mean (p-y)^2, with the original probabilities.
• —Log loss: natural logarithms, clipping probabilities to [1e-15, 1-1e-15] so boundary forecasts have finite loss.
• —Accuracy: class threshold p >= 0.5. It discards confidence and is not a calibration measure.
• —Reliability: 5, 10 or 20 equal-width bins, with counts, mean probability and observed outcome rate. Empty bins are shown explicitly. The final bin includes 1.

The chart has a numeric table equivalent. Accepted observations can be exported with per-row scores and bin indices. Editing input clears stale results. Small batches, binning choices, class balance and label quality limit interpretation. No confidence intervals or significance tests are supplied.

Publisher and context

Published by Licensed Bets, an affiliate-funded publisher. This standalone educational app contains no bookmaker affiliate links or account integration. The odds-data freshness explanation discusses a separate data-quality issue; it does not validate this synthetic example or establish model calibration.

18+ only. Evaluation is not a guarantee of profit or a reason to gamble. Responsible-gambling support resources.

Related separate project: Accumulator Reality Check. That repository is not this lab's code provenance. This Space contains the complete source in index.html.

License

MIT. See LICENSE.