chaosnetbench-team/chaosnetbench-cml

Dataset Card — ChaosNetBench-CML

Primary dataset file: data/chaosnetbench_cml.h5

Dataset Name

ChaosNetBench-CML

Version

1.0.0

License

CC-BY 4.0

Purpose

ChaosNetBench-CML is a benchmark dataset and evaluation framework for systematically comparing spatio-temporal graph neural networks on controlled chaotic lattice dynamics. Built on coupled standard maps with known ring topology and independently tunable local chaos (K), coupling (epsilon), and system size (N), it supports regime-aware comparisons between graph-aware and purely temporal baselines across 96 system instances and 9,600 trajectories. ---

Contents

CSV schema

Metric Definitions

Metric definitions are implemented in chaosnetbench/metrics.py in the anonymous code repository:

• —VPT (Valid Prediction Time): first AR step where NRMSE > 1.0 (VPTNRMSETHRESHOLD). NRMSE is RMSE normalised by signal std.
• —Convergence filter: runs with test_mse_mean >= 0.95 are degenerate (near-constant output). Excluded from VPT head-to-head comparisons.
• —3-seed aggregation: each (model, K, rho, N) config is trained with 3 random seeds; arvptmean and testmsemean are the means; *_std are cross-seed standard deviations.

Generation Protocol

System: Coupled Standard Map (Chirikov-Taylor map, N sites, nearest-neighbour coupling).

Reference: Chirikov (1979), Physics Reports 52(5), 263–379.

Parameter grid:

• —K ∈ {0.5, 0.97, 2.0, 6.5} (ordered → hyperchaotic)
• —ρ ∈ {0.05, 0.075, 0.10, 0.15, 0.20, 0.30, 0.40, 0.50}; ε = ρ × K, filtered to [0.01, 5.0]
• —N ∈ {8, 16, 32} sites

Trajectories per config: 100 ICs (70 train / 10 val / 20 test), IC-based split (no temporal leakage).

Steps: 1000 transient (discarded) + 10000 recorded.

Initial conditions: Uniform on [0, 2π) × (−π, π) per site; hash-based per-config seeds (base_seed=42).

SALI orbit classification: 1000 tangent-map iterations, early termination at SALI < 1e-8.

Code: trajectory generation is implemented in chaosnetbench/dataset.py and chaosnetbench/systems/standard_map.py in the anonymous code repository.

Reviewer Sample — How It Was Created

The reviewer sample (multiseed_aggregated.csv) was produced by:

1. 1.Training each model on the full trajectory dataset for 50 epochs with 3 seeds.
2. 2.Evaluating on the IC-held-out test set (20 ICs per config).
3. 3.Computing per-seed VPT and MSE; taking mean and std across the 3 seeds.
4. 4.Aggregating into a single CSV with one row per (model, K, rho, N).

The sample contains post-aggregation metrics only — no raw trajectories or model weights. It supports result verification (reproducing Table 3 and Figure 5 in the paper) without requiring model retraining.

Responsible AI

Data limitations: Covers only CSM lattice dynamics; results may not generalize to dissipative systems, continuous-time chaotic flows, or spatiotemporal PDEs. The IC-based split does not test out of distribution K/rho generalisation.

Data biases: Entirely synthetic; no human subjects. The parameter grid intentionally densifies near K=0.97 (critical transition) for scientific reasons.

Personal/sensitive information: None.

Use cases (validated): (1) Model comparison under controlled chaos. (2) STGNN vs temporal baseline evaluation. Not recommended as a general-purpose time-series benchmark unrelated to chaotic dynamics.

Social impact: Positive (reproducible benchmark, scientific transparency). No known misuse vectors.

Synthetic data: Yes — numerically simulated from a deterministic mathematical system.