kacpnowak/hclimrep-ocean-oceanbench

HCLIMREP Ocean — OceanBench challenger forecasts

OceanBench challenger submissions produced with the WeatherGenerator machine-learning Earth system model, together with the executed evaluation notebooks that turn each forecast store into an OceanBench report.

Each subdirectory is one evaluated model. Scores are tied to a specific OceanBench version and are not comparable across versions.

ft0818

Ten-day global ocean forecasts from WeatherGenerator run glory_finetuning_20260818_105654, initialised weekly through 2024 and evaluated with OceanBench 0.4.0 over the global region.

Contents

ft0818/
  wg_ft0818_B1.zarr/                        forecast store (Zarr v2, ~39 GB on disk)
  challenger_ft0818.py                      opens the store as `challenger_dataset`
  challenger_ft0818_global_report.ipynb     executed OceanBench report, region "global"

Store schema

Dimensions:  first_day_datetime: 52, lead_day_index: 10, depth: 9,
             latitude: 672, longitude: 1440

Data variables (float32)
  thetao  sea_water_potential_temperature   degrees_C   (time, lead, depth, lat, lon)
  so      sea_water_salinity                PSU         (time, lead, depth, lat, lon)
  uo      eastward_sea_water_velocity       m/s         (time, lead, depth, lat, lon)
  vo      northward_sea_water_velocity      m/s         (time, lead, depth, lat, lon)
  zos     sea_surface_height_above_geoid    m           (time, lead, lat, lon)

• —Forecast starts: 52 weekly Wednesdays, 2024-01-03 through 2024-12-25. Each forecast is initialised from the GLORYS nowcast of the preceding Tuesday.
• —Lead days: lead_day_index 0–9, one daily mean per step, ten days total.
• —Grid: 0.25° regular, latitude −78.0 to 89.75, longitude −180.0 to 179.75.
• —Depths (m): 0.494, 47.374, 92.326, 155.851, 222.475, 318.127, 380.213, 453.938, 541.089. These are the nine OceanBench reference levels, taken directly from the model's own GLORYS depth axis, so no vertical interpolation or nearest-neighbour substitution is involved.
• —Chunking: one forecast start per chunk, all leads and depths in that chunk.
• —Written with Zarr v2. OceanBench pins zarr==2.18.4 and cannot read v3 stores. Consolidated metadata is present, so the store opens either way; the shipped challenger script passes consolidated=False so that it keeps working if the store is ever extended with further forecast starts.

Usage

hf download kacpnowak/hclimrep-ocean-oceanbench --repo-type dataset \
  --include "ft0818/*" --local-dir ./hclimrep-ocean-oceanbench

import xarray as xr

challenger_dataset = xr.open_zarr(
    "hclimrep-ocean-oceanbench/ft0818/wg_ft0818_B1.zarr", consolidated=False
)

challenger_ft0818.py reads the store path from the WG_CHALLENGER_STORE environment variable, so the shipped file works unchanged once that points at your local copy:

export WG_CHALLENGER_STORE=$PWD/hclimrep-ocean-oceanbench/ft0818/wg_ft0818_B1.zarr

To reproduce the report, drop that script into the OceanBench challenger notebook template and execute it, or re-run the shipped notebook in an environment with oceanbench==0.4.0 and the OceanBench reference data staged locally.

Results (OceanBench 0.4.0, region global)

RMSD against GLORYS reanalysis, surface fields:

RMSD against GLORYS reanalysis, temperature and salinity at depth:

RMSD against observations (Class-4):

Geostrophic currents against GLORYS, and Lagrangian trajectory deviation:

Lagrangian deviation is defined for lead days 2 through 9; it grows from 9.70 km at lead day 2 to 67.54 km at lead day 9.

The notebook also contains the GLO12-analysis counterparts of the GLORYS-reanalysis metrics. See the executed report for every row.

Provenance

• —Model: WeatherGenerator run glory_finetuning_20260818_105654, an aggregation-only, ten-step finetune of the preceding ft0808 checkpoint.
• —Inference: 52 single-forecast runs driven by config/config_forecasting_glorys_obench_v2.yml, using the 133-channel GLORYS v2 ocean stream and an ERA5 atmospheric stream. Atmospheric forcing is supplied only over the initial-condition window; the model sees no future atmosphere during the ten-day rollout.
• —Export: export_glorys_nc.py, selecting the nine reference depth labels from the model's native output levels.
• —Evaluation: OceanBench 0.4.0, region global, no post-processing or bias correction applied to the forecasts.

Caveats

• —Mixed layer depth is computed from nine output levels rather than a full vertical profile, which discretises the diagnostic and inflates its RMSD.
• —Scores are specific to OceanBench 0.4.0. Do not compare them against numbers produced by other versions of the benchmark.
• —The store is Zarr v2 for compatibility with the zarr==2.18.4 pin in OceanBench. It cannot be read by a Zarr v3-only reader.
• —Values are raw model output with no clipping. A small number of salinity points fall marginally outside physically plausible bounds, which the scores already reflect.