Datasetpublic

OneAstronomy/provabgs-galaxy-estimation

PROVABGS desi_legacy_fusion Dataset Summary 109,991 BGS galaxies with multimodal data from DESI (spectra + photometry) × Legacy Survey (imaging + photometry), matched within 1 arcsec. Split Samples train 87,992 validation 10,999 test 11,000 total 109,991 Labels (Galaxy Parameters from PROVABGS SED fitting) Column Description Units z_hp Redshift — log_mstar Stellar mass log(M☉) tage_mw Mass-weighted age Gyr… See the full description on the dataset page: https://huggingface.co/datasets/OneAstronomy/provabgs-galaxy-estimation.

sourceHugging Facecc-by-4.0updated 4mo agoView on Hugging Face

0likes23downloads

Dataset Card

PROVABGS desilegacyfusion Dataset

Summary

109,991 BGS galaxies with multimodal data from DESI (spectra + photometry) × Legacy Survey (imaging + photometry), matched within 1 arcsec.

Split	Samples
train	87,992
validation	10,999
test	11,000
total	109,991

Labels (Galaxy Parameters from PROVABGS SED fitting)

Column	Description	Units
`z_hp`	Redshift	—
`log_mstar`	Stellar mass	log(M☉)
`tage_mw`	Mass-weighted age	Gyr
`log_z_mw`	Metallicity log₁₀(Z_MW)	log(Z)
`log_ssfr`	Specific SFR log₁₀(SFR/M★)	log(yr⁻¹)

Modalities

Column	Shape	Description
`image_pixels_raw`	(102400,)	LS image (4, 160, 160), des-g/r/i/z, flat float32, nanomaggies
`spectrum_flux_raw`	(7781,)	DESI-BGS spectrum, float32
`spectrum_ivar`	(7781,)	Inverse variance, float32
`spectrum_mask`	(7781,)	Bad pixel mask, bool
`ls_flux_g/r/z/w1/w2`	scalar	Legacy Survey photometry, float32
`desi_flux_g/r/z/w1/w2`	scalar	DESI photometry, float32

Wavelength grid in wavelength_grid.json, image channel layout in image_shape.json.

Quick Start

python

from datasets import load_dataset
import numpy as np

BASE = "/mnt/si0009256k6u/ckdata/aiready/provabgs/hf_dataset"
ds = load_dataset("parquet", data_dir=BASE, streaming=True)

for sample in ds["train"].with_format(type="numpy").take(10):
    img  = sample["image_pixels_raw"].reshape(4, 160, 160)  # float32
    spec = sample["spectrum_flux_raw"]                       # (7781,) float32
    z    = sample["z_hp"]
    logM = sample["log_mstar"]
    age  = sample["tage_mw"]
    ssf  = sample["log_ssfr"]

Notes

—row_group_size=100 for efficient streaming
—List columns stored as float32 (not float64)
—Normalization deferred to training pipeline
—log_ssfr = log10(AVG_SFR) - log_mstar
—log_z_mw = log10(Z_MW)