htmoges/chaosnetbench-cml

Dataset Card - ChaosNetBench-CML

Primary dataset file: data/chaosnetbench_cml.h5

Release

• —Version: 1.0.0
• —Public release date: 2026-05-09
• —Creator and maintainer: H. T. Moges
• —Corresponding contact: ht.moges@gmail.com
• —Homepage: https://htmoges.github.io
• —Code: https://github.com/htmoges/ChaosNetBench
• —License: CC-BY 4.0

Purpose

ChaosNetBench-CML is a benchmark dataset and evaluation framework for systematically comparing spatio-temporal graph neural networks on controlled chaotic lattice dynamics. Built on coupled standard maps with known ring topology and independently tunable local chaos (K), coupling (epsilon), and system size (N), it supports regime-aware comparisons between graph-aware and purely temporal baselines across 96 system instances and 9,600 trajectories.

Contents

CSV schema

Metric definitions

Metric definitions are implemented in chaosnetbench/metrics.py in the public code repository:

• —VPT (Valid Prediction Time): first autoregressive step where NRMSE > 1.0. NRMSE is RMSE normalized by the standard deviation of the true trajectory.
• —Convergence filter: runs with test_mse_mean >= 0.95 are degenerate (near-constant output) and are excluded from VPT head-to-head comparisons.
• —3-seed aggregation: each (model, K, rho, N) configuration is trained with 3 random seeds; ar_vpt_mean and test_mse_mean are the means, while *_std fields are across-seed standard deviations.

Generation protocol

System: Coupled Standard Map (Chirikov-Taylor map, ring topology, nearest-neighbor coupling).

Parameter grid:

• —K in {0.5, 0.97, 2.0, 6.5}
• —rho in {0.05, 0.075, 0.10, 0.15, 0.20, 0.30, 0.40, 0.50}; epsilon = rho * K
• —N in {8, 16, 32}

Protocol:

• —100 initial conditions per configuration (70 train / 10 validation / 20 test)
• —IC-based split to avoid temporal leakage
• —1000 transient steps discarded + 10000 recorded steps
• —SALI orbit classification with 1000 tangent-map iterations and early termination at SALI < 1e-8

Code for trajectory generation and benchmark evaluation is available in the public repository: https://github.com/htmoges/ChaosNetBench

Results preview

The lightweight multiseed_aggregated.csv preview was produced by:

1. 1.Training each benchmark model on the full trajectory dataset for 50 epochs with 3 random seeds.
2. 2.Evaluating on the IC-held-out test split with 20 initial conditions per configuration.
3. 3.Computing per-seed VPT and test MSE.
4. 4.Aggregating the results into one CSV row per (model, K, rho, N) with cross-seed means and standard deviations.

The preview contains post-aggregation metrics only. It is intended to make the benchmark outputs easy to inspect without downloading the full HDF5 release or retraining models.

Responsible AI

• —Data limitations: covers only coupled standard map lattice dynamics; results may not generalize to dissipative systems, continuous-time chaotic flows, or spatiotemporal PDEs.
• —Data biases: entirely synthetic; no human subjects. The parameter grid intentionally samples critical transition regimes for scientific analysis.
• —Personal or sensitive information: none.
• —Validated uses: model comparison under controlled chaos; STGNN-versus-temporal baseline evaluation; analysis of rollout robustness.
• —Synthetic data: yes.