castcheck/temperature-verification

CastCheck — daily station-level verification of public weather forecasts

Independent, automated verification of raw 2 m temperature forecasts from operational NWP (ECMWF IFS HRES, NCEP GFS) and AI models (ECMWF AIFS Single; NOAA/CIRA operational runs of GraphCast, Pangu-Weather, FourCastNet v2 and Aurora from both GFS and IFS initial conditions) at 23 U.S. first-order stations — 22 major airports plus New York Central Park. The headline metric is the instantaneous 2 m temperature at 00/06/12/18 UTC against the ASOS observation at the same instants; daily extremes are published as secondary variables, both like-for-like and against the NWS Daily Climate Report (CLI).

• —Site and permanent links: https://castcheck.zifanzhang.com
• —Methodology v0.3.1: https://castcheck.zifanzhang.com/methodology/
• —Code: https://github.com/ZzZTripleZzZ/castcheck
• —Archived: https://doi.org/10.5281/zenodo.22212363
• —Updated daily by an automated pipeline. Last push: 2026-09-21 17:01 UTC

Fairness statement

These are raw model outputs at 0.25°, without MOS, bias correction, or downscaling. They are not equivalent to the post-processed products end users receive, and scores here understate operational forecast quality. Every model is sampled at the same four instants, and the headline metric compares those samples with observations at the same instants, so no part of it depends on a model's own diurnal amplitude. The extremes verified against the CLI report (tmax_cli, tmin_cli) carry a sampling penalty whose size does differ between models, so they are secondary and are never used to rank (see methodology §2.3).

Files (schema v0.3)

• —data/forecast_values/model_id=*/year_month=*.parquet — extracted station values (long format)
• —data/truth_instant/year=*.parquet — observed 2 m temperature at 00/06/12/18 UTC, with QC flags
• —data/truth_daily/year=*.parquet — NWS CLI/CF6/observation truth with first-final policy and QC flags
• —data/daily_forecasts/model_id=*/year=*.parquet — derived sampled/native daily extremes per lead day
• —data/scores/latest.parquet, data/scores/pairwise_latest.parquet — published aggregates with bootstrap CIs

from datasets import load_dataset
ds = load_dataset("castcheck/temperature-verification", data_files="data/scores/latest.parquet")

Sources and licences

ECMWF Open Data (CC-BY-4.0) · NOAA/NCEP GFS (public domain) · NOAA/CIRA AIWP (open data; cite the AIWP BAMS paper) · NWS climate reports (public domain) · Iowa Environmental Mesonet AFOS archive. ECMWF data is © ECMWF, licensed under CC-BY-4.0 and used unmodified apart from interpolation to the station; ECMWF does not endorse this work. This dataset is released under CC-BY-4.0; please cite as CastCheck, methodology v0.3.1, doi:10.5281/zenodo.22212363, https://castcheck.zifanzhang.com (accessed YYYY-MM-DD).

Models

• —ifs_hres — ECMWF IFS HRES
• —aifs_single — ECMWF AIFS Single
• —gfs — NCEP GFS
• —graphcast_ifs — GraphCast (IFS initial conditions)
• —graphcast_gfs — GraphCast (GFS initial conditions)
• —pangu_ifs — Pangu-Weather (IFS initial conditions)
• —pangu_gfs — Pangu-Weather (GFS initial conditions)
• —fourcastnet_ifs — FourCastNet v2 (IFS initial conditions)
• —fourcastnet_gfs — FourCastNet v2 (GFS initial conditions)
• —aurora_ifs — Aurora (IFS initial conditions)
• —aurora_gfs — Aurora (GFS initial conditions)

Stations

• —KNYC New York Central Park (40.783, -73.967, 46.9 m)
• —KEWR Newark Liberty (40.682, -74.169, 4.9 m)
• —KPHL Philadelphia Intl (39.873, -75.227, 2.1 m)
• —KTTN Trenton Mercer (40.276, -74.816, 64.0 m)
• —KBOS Boston Logan (42.361, -71.011, 6.1 m)
• —KDCA Washington Reagan (38.848, -77.034, 4.0 m)
• —KATL Atlanta Hartsfield (33.640, -84.427, 313.0 m)
• —KMIA Miami Intl (25.791, -80.316, 3.0 m)
• —KORD Chicago O'Hare (41.980, -87.904, 203.0 m)
• —KMSP Minneapolis-St Paul (44.883, -93.229, 256.0 m)
• —KDFW Dallas-Fort Worth (32.897, -97.022, 164.9 m)
• —KIAH Houston Bush (29.984, -95.361, 27.1 m)
• —KAUS Austin Bergstrom (30.183, -97.680, 148.1 m)
• —KSAT San Antonio Intl (29.533, -98.464, 246.0 m)
• —KMSY New Orleans Intl (29.993, -90.251, 0.9 m)
• —KOKC Oklahoma City (35.389, -97.600, 394.1 m)
• —KDEN Denver Intl (39.847, -104.656, 1647.1 m)
• —KPHX Phoenix Sky Harbor (33.428, -112.003, 339.9 m)
• —KLAS Las Vegas Harry Reid (36.072, -115.163, 664.5 m)
• —KLAX Los Angeles Intl (33.938, -118.389, 38.1 m)
• —KSAN San Diego Lindbergh (32.734, -117.183, 4.0 m)
• —KSFO San Francisco Intl (37.620, -122.366, 3.0 m)
• —KSEA Seattle-Tacoma (47.445, -122.314, 130.1 m)