aryana-haghjoo/jades-dr4-dr5-multimodal

JADES DR4/DR5 Multimodal Dataset

Cold-storage archive of the data products for the `multimodal_superresolution` project: JADES DR4 NIRSpec spectra matched to DR5 NIRCam/HST imaging cutouts, used for multimodal galaxy super-resolution and redshift estimation.

Archived 2026-08-28. The local working copy was deleted to free disk. Code and trained model weights are on GitHub; everything Git ignored is here.

👉 [`RECONSTRUCT.md`](RECONSTRUCT.md) is the full recipe for rebuilding the project from scratch. Read it first.

What this is

2937 galaxies with both a NIRSpec spectrum and NIRCam imaging, across the GOODS-S and GOODS-N fields:

• —Spectra — DR4 prism + R1000 grating, 4702 wavelength points each
• —Images — DR5 224×224 cutouts in up to 13 bands, with science, error, exposure, weight and nim planes
• —Manifests + splits — the exact train/test partitions the published models were trained on

The expensive part is the imaging: the cutouts were extracted from JADES DR5 mosaics totalling 757 GB of downloads (~29 hours of transfer). This archive exists so that never has to be repeated — pulling 21 GB from here takes under an hour.

Quick start

pip install huggingface_hub

# everything (~21 GB)
hf download aryana-haghjoo/jades-dr4-dr5-multimodal \
    --repo-type dataset --local-dir ./jades_data

# or just the manifests and one band
hf download aryana-haghjoo/jades-dr4-dr5-multimodal --repo-type dataset \
    --local-dir ./jades_data \
    --include "data/multimodal/*" "data/JADES/cutouts_shared_bands/goods-s/f090w.tar"

To rebuild the full repo layout:

git clone git@github.com:aryana-haghjoo/multimodal_superresolution.git
bash ./jades_data/restore.sh ./jades_data ./multimodal_superresolution

Reading a single cutout:

import numpy as np
d = np.load("goods-s/f090w/dr4id_1603_dr5id_94222.npz")
print(list(d))       # science, error, exposure, weight, nim, ra, dec,
                     # dr4_id, dr5_id, match_sep_arcsec, band, field
print(d["science"].shape)   # (224, 224)

Layout

Cutouts are bundled as uncompressed .tar, one per band/field. .npz is already DEFLATE-compressed internally, so gzip/zstd gained only 0.3% — not worth the CPU on either end.

Bands

f070w exists only in goods-s — it was excluded from the project because it has no matched coverage in goods-n. The trained models use 11 bands: f090w f115w f150w f200w f277w f356w f444w f410m f335m f182m f210m. f070w and f162m are archived but unused.

Companion models

Trained weights are on GitHub, not here:

Not included: DR5 photometry catalogues

The pipeline that produced these cutouts also reads the JADES DR5 photometry catalogues (HDU FLAG) to resolve object positions and real per-band coverage:

hlsp_jades_jwst_nircam_goods-s_photometry_v5.0_catalog.fits   6.17 GB
hlsp_jades_jwst_nircam_goods-n_photometry_v5.0_catalog.fits   3.12 GB

They are not mirrored here — large, re-downloadable, and only needed to redo the DR4↔DR5 matching or re-extract cutouts from the mosaics. Everything they were used to produce is archived: the match tables, the coverage tables, and the cutouts themselves.

BASE=https://slate.ucsc.edu/~brant/jades-dr5
wget -c "$BASE/GOODS-S/hlsp/catalogs/hlsp_jades_jwst_nircam_goods-s_photometry_v5.0_catalog.fits"
wget -c "$BASE/GOODS-N/hlsp/catalogs/hlsp_jades_jwst_nircam_goods-n_photometry_v5.0_catalog.fits"

⚠️ That source is a collaboration server under a ~brant user path, not MAST — verified live 2026-08-28, but it may move when DR5 goes public. Fall back to <https://archive.stsci.edu/hlsp/jades>. See `RECONSTRUCT.md` for full detail.

Known issues

• —multimodal_dataset_qualitycut*.pkl are empty (0 rows). The strict line-S/N≥5 cut selected nothing on its last run, though splits_qualitycut/ retains 130 rows from an earlier working generation.
• —splits/multimodal_dataset_val.pkl (315 rows) is a stale leftover from an abandoned 3-way split. The models used train.pkl (2190) and test.pkl (730). split_counts.csv and split_field_summary.csv disagree for the same reason.
• —Foundation-v2 checkpoints r2/r3/r4 were already lost before archiving; only their inference outputs survive. Check W&B cloud if needed.

RECONSTRUCT.md documents all of these in detail.

Provenance and citation

Derived from public JWST/JADES data — DR4 NIRSpec spectra and DR5 NIRCam/HST mosaics from https://slate.ucsc.edu/~brant/jades-dr5/. The original survey data is released by the JADES Collaboration; please cite JADES when using this derived product. Derived cutouts, manifests and splits are shared under CC-BY-4.0.