GilpinLab/dataset-panda72M-repro

Dataset Card for _dataset-panda72M-repro_

These are all the datasets we used for training our scaled-up Panda-72M model (https://huggingface.co/GilpinLab/panda-72M) of Panda: Patched Attention for Nonlinear Dynamics.

We have made all parquet files here streamable by using chunked row groups, instead of the single row group containing all data (as done for our other Panda datasets on HF).

_NOTE (on historical changelog):_

base_mixedp_ic16 The dataset in basemixedpic16/train-part1.parquet contains only the first 7 initial conditions present in the full base-mixedp-ic16 train data (https://huggingface.co/datasets/GilpinLab/base-mixedp-ic16). Specifically, these are ics 1-7, and this was due to an indexing error when we made this dataset.

Moreover, there are actually only 109 “ic1” trajectories in the entire “train” dataset; these are all for “pp0” i.e. the unperturbed original base systems.

The table below gives the number of trajectories for each initial condition (ic{1-7}) in basemixedpic16/train-part1.parquet. Each ic{1-7} has fewer than the number of perturbed systems because some trajectories may not have passed our fixed integration time-limit, or our filters.

skew_mixedp_ic16 The dataset in skewmixedpic16/train-part1.parquet contains only the first 8 initial conditions present in the full skew-mixedp-ic16 train data (https://huggingface.co/datasets/GilpinLab/skew-mixedp-ic16). Specifically, these are ics 0-7.

By "distinct skew system parent pairs" we refer to {SystemA driving SystemB} as a parent pair, and perturbations of the parameters of this system make the perturbed skew systems. Another distinct parent pair would be {SystemC driving System A} for example.

The table below gives the number of trajectories for each initial condition (ic{0-7}) in skewmixedpic16/train-part1.parquet. Each ic{0-7} has fewer than the number of perturbed systems because some trajectories may not have passed our fixed integration time-limit, or our filters.

base40 Lastly, base40/train.parquet is exactly the same as https://huggingface.co/datasets/GilpinLab/base40 but with chunked row groups so it is streamable.

NOTE: the traininginfo.json files from our model training runs may mention "finalbase40/trainz5z10"; this was merged into "final_base40/train" which is uploaded here as base40/train.parquet, containing the union of the files.

Summary As can be seen in our uploaded wandb log file from our original Panda-72M training run (panda72M-wandb-log.log), we trained on 190006 datasets i.e. arrow files (trajectories). 151,197 + 32,996 + 5,813 = 190,006.

There are many benefits to making these parquet files have chunked row groups. The schema, column order, dtypes, and row order remain unchanged; pq.read_table() returns a table identical to what it would return with a single row group. The benefits:

1. 1.HF viewer works because the Parquet file contains a page index to enable random access without having to load the entire row group
2. 2.Partial reads are possible. With multiple (hundreds) of row groups, datasets.loaddataset(..., streaming=True) and pq.ParquetFile.iterbatches() become usable instead of OOMing or doing a slow crawl on the full dataset
3. 3.Row-group statistics enable skipping. Since rows are sorted by sourcedirectory, each row group's min/max on that column is precise. A reader who wants just Lorenz-pp0 can now read a handful of row groups instead of reading the whole file.
4. 4.Parallel/distributed reads. pyarrow, dask, spark, polars all parallelize at row-group granularity. One row group = one thread, no matter how many cores.

Paper abstract:

Chaotic systems are intrinsically sensitive to small errors, challenging efforts to construct predictive data-driven models of real-world dynamical systems such as fluid flows or neuronal activity. Prior efforts comprise either specialized models trained separately on individual time series, or foundation models trained on vast time series databases with little underlying dynamical structure. Motivated by dynamical systems theory, we present Panda, Patched Attention for Nonlinear DynAmics. We train Panda on a novel synthetic, extensible dataset of 2 \times 10^4 chaotic dynamical systems that we discover using an evolutionary algorithm. Trained purely on simulated data, Panda exhibits emergent properties: zero-shot forecasting of unseen real world chaotic systems, and nonlinear resonance patterns in cross-channel attention heads. Despite having been trained only on low-dimensional ordinary differential equations, Panda spontaneously develops the ability to predict partial differential equations without retraining. We demonstrate a neural scaling law for differential equations, underscoring the potential of pretrained models for probing abstract mathematical domains like nonlinear dynamics.

• —Preprint: arXiv:2505.13755
• —Repository: https://github.com/abao1999/panda

Citation

BibTeX:

If you find our work valuable for your research, please cite us:

@inproceedings{
    lai2026panda,
    title={Panda: A pretrained forecast model for chaotic dynamics},
    author={Jeffrey B. Lai and Anthony Bao and William Gilpin},
    booktitle={The Fourteenth International Conference on Learning Representations},
    year={2026},
    url={https://openreview.net/forum?id=DgnsohAUMn}
}