BruceYuan/KoopmanDA

Koopman Data Assimilation Dataset

Dataset accompanying the benchmark for Data Assimilation (DA) tasks with Koopman-inspired models.

Code repository: github.com/Wenxuan52/KoopmanDABench (Currently private, will release codebase when paper finished)

What Is Hosted Where

The project is split across two locations:

• —GitHub repo — source code, plus a few small files under data/ (e.g. data-generation scripts, small reference files, configs).
• —This Hugging Face dataset — the large pre-computed .npy / .h5 files that are too big to track in Git.

On a new machine, cloning the GitHub repo and then downloading this dataset into the same data/ folder gives you the complete setup. The two sets of files do not overlap.

Final data/ Layout

After both git clone and hf download, the data/ folder should look like:

data/
├── kol_generate_data/            # from GitHub
├── ERA5_High/
│   ├── process_weatherbench.py   # from GitHub
│   ├── test.py                   # from GitHub
│   └── raw_data/                 # from Hugging Face
├── ERA5/                         # from Hugging Face
├── dam/                          # from Hugging Face
├── cylinder/                     # from Hugging Face
└── kol/                          # from Hugging Face

Quick Start on a New Machine

1. Clone the code from GitHub

git clone https://github.com/Wenxuan52/KoopmanDA.git
cd KoopmanDA

This already populates data/ with the small files tracked in Git (e.g. generation scripts).

2. Set up the Python environment

conda create -n koopmanda python=3.10 -y
conda activate koopmanda
pip install -r requirements.txt        # or: pip install -e .
pip install -U huggingface_hub

3. Pull the large data files from Hugging Face into data/

Download into the existing data/ folder — files from GitHub are not affected:

hf download BruceYuan/KoopmanDA \
    --repo-type dataset \
    --local-dir ./data

Equivalent Python call:

from huggingface_hub import snapshot_download
snapshot_download(
    repo_id="BruceYuan/KoopmanDA",
    repo_type="dataset",
    local_dir="./data",
)

The project should now run exactly as it did on the original machine.

License

Released under the MIT License.