Datasetpublic

leharris3/ccrfcd-mrms-hrrr-env-2021-2025

1H gauge accumulation + MRMS/HRRR zarr dataset for the Desert Southwest 50+ MRMS+HRRR variables; 220+ gauges; 400k samples NOTE: work in-progress. This is a dataset for training and evaluating synthetic quantitative precipiation estimation (QPE) models. Given some input context (e.g., radar fields, envionrmental parameters), predict how much rain fell at a rain gauge site over some period of time. Concretely, we've gather and QC'd data from 220 tipping bucket gauges through… See the full description on the dataset page: https://huggingface.co/datasets/leharris3/ccrfcd-mrms-hrrr-env-2021-2025.

sourceHugging Facecc-by-4.0updated 6mo agoView on Hugging Face

3likes402downloads

Dataset Card

1H gauge accumulation + MRMS/HRRR zarr dataset for the Desert Southwest

50+ MRMS+HRRR variables; 220+ gauges; 400k samples

NOTE: work in-progress.

![Hugging Face Dataset](https://huggingface.co/datasets/leharris3/ccrfcd-mrms-hrrr-env-2021-2025)

This is a dataset for training and evaluating synthetic quantitative precipiation estimation (QPE) models. Given some input context (e.g., radar fields, envionrmental parameters), predict how much rain fell at a rain gauge site over some period of time. Concretely, we've gather and QC'd data from 220 tipping bucket gauges through Nevada for the years 2021-2026. These readings are then used as ground-truth for ~400k samples, temporally and spatially aligned to 13 MRMS radar feilds and 39 HRRR envionrmental parameters. This dataset can therefore support training diverse QPE models (point -> point, local-spatial context -> point, etc.)

Region of Interest

—Lattitude: 34.751857˚N, 37.103662˚N
—Longitude: 113.792819˚W, 116.146925˚W

File Structure

data/
├── ccrfcd_rain_gauge_metadata.csv               # gauge site metadata (lat, lon, name, id)
├── gauge_feature_cache.csv                      # cached MRMS product values at gauge locations
├── gauge_feature_cache_meta.json
├── norm_stats_train.json                        # per-channel mean/std for normalization
├── 2021-01-01_2025-07-25_gt.csv                 # ground-truth accumulation samples
├── 2021-01-01_2026-03-23_gt.csv
├── 2021-01-01_2025-07-25_hrrr_env_final_v8.csv  # point-sample dataset (v8)
├── 2021-01-01_2025-07-25_hrrr_env_final_v9.csv  # point-sample dataset (v9)
│
├── ccrfcd_gauges_raw_data/                      # raw per-gauge CSVs from CCRFCD
│   └── <YYYYMMDD-YYYYMMDD>-<id>_<name>_RAIN.csv
│
├── dem/                                         # digital elevation model on HRRR grid
│   ├── dem_hrrr.npy                             # elevation (235, 236) float32, meters
│   ├── dzdx_hrrr.npy                            # east-west slope
│   ├── dzdy_hrrr.npy                            # north-south slope
│   └── srtm_tiles/                              # raw SRTM source tiles
│
├── hrrr_zarr/<product>/<YYYYMMDD>.zarr/         # raw HRRR fields (22 products)
│   ├── data/       (N, 235, 236) float32
│   ├── timestamps/ (N,) float64 unix seconds
│   └── zarr.json
│
├── hrrr_zarr_derived/<product>/<YYYYMMDD>.zarr/ # derived HRRR fields (18 products)
│   ├── data/       (N, 235, 236) float32
│   ├── timestamps/ (N,) float64 unix seconds
│   └── zarr.json
│
└── mrms_zarr/<product>/<YYYYMMDD>.zarr/         # MRMS radar fields (13 products)
    ├── data/       (N, 235, 236) float32
    ├── timestamps/ (N,) float64 unix seconds
    └── zarr.json

Samples

Variable List

Name	Variable	Units	Point	Spatial	Temp. Resolution	Source	Notes
Gauge ID	`gauge_idx`	`int`	✅	–––	–––	CCRFCD	unique gauge id assigned by CCRFCD
Start Time	`start_datetime_utc`	`datetime`	–––	–––	2-min	MRMS	intervals for which MRMS 1H radar QPE are available; UTC
End Time	`end_datetime_utc`	`datetime`	–––	–––	2-min	MRMS	start time + 01:00:00
Gauge 1H Accumulation	`gauge_acc_in`	in	✅	–––	2-min	CCRFCD	rain gauge 1H accumulation; aligned to nearest MRMS 2-min interval
MRMS Radar 1H QPE	`mrms_q3evap_qpe`	in	✅	❌	2-min	MRMS	MRMS operational 1H radar-only QPE
Latitude	`lat`	degrees	✅	–––	–––	CCRFCD	latitude of rain gauge
Longitude	`lon`	degrees	✅	–––	–––	CCRFCD	longitude of rain gauge
10m U-Wind	`10m_above_ground_UGRD`	m/s	❌	✅	1-hour	HRRR Zarr
850mb U-Wind	`850mb_UGRD`	m/s	✅	✅	1-hour	HRRR Zarr
700mb U-Wind	`700mb_UGRD`	m/s	✅	✅	1-hour	HRRR Zarr
10m V-Wind	`10m_above_ground_VGRD`	m/s	❌	✅	1-hour	HRRR Zarr
850mb V-Wind	`850mb_VGRD`	m/s	✅	✅	1-hour	HRRR Zarr
700mb V-Wind	`700mb_VGRD`	m/s	✅	✅	1-hour	HRRR Zarr
850-700mb Mean Wind	`850_700mb_mean_wind`	m/s	✅	✅	1-hour	HRRR Zarr*	magnitude of mean u-v vector
Surface Pressure	`surface_PRES`	Pa	✅	✅	1-hour	HRRR Zarr
850mb Geopot. Height	`850mb_HGT`	gpm	✅	✅	1-hour	HRRR Zarr
700mb Geopot. Height	`700mb_HGT`	gpm	✅	✅	1-hour	HRRR Zarr
Freezing Level	`highest_tropospheric_`<br>`freezing_level_HGT`	m	✅	✅	1-hour	HRRR Zarr
LCL Height	`level_of_adiabatic_`<br>`condensation_`<br>`from_sfc_HGT`	gpm	✅	✅	1-hour	HRRR Zarr
Warm Cloud Depth	`convective_warm`<br>`_cloud_depth`	m	✅	✅	1-hour	HRRR Zarr*
2m Temperature	`2m_above_ground_TMP`	K	✅	✅	1-hour	HRRR Zarr
925mb Temperature	`925mb_TMP`	K	✅	✅	1-hour	HRRR Zarr
850mb Temperature	`850mb_TMP`	K	✅	✅	1-hour	HRRR Zarr
700mb Temperature	`700mb_TMP`	K	✅	✅	1-hour	HRRR Zarr
500mb Temperature	`500mb_TMP`	K	✅	✅	1-hour	HRRR Zarr
2m Dewpoint	`2m_above_ground_DPT`	K	✅	✅	1-hour	HRRR Zarr
925mb Dewpoint	`925mb_DPT`	K	✅	✅	1-hour	HRRR Zarr
850mb Dewpoint	`850mb_DPT`	K	✅	✅	1-hour	HRRR Zarr
700mb Dewpoint	`700mb_DPT`	K	✅	✅	1-hour	HRRR Zarr
500mb Dewpoint	`500mb_DPT`	K	✅	✅	1-hour	HRRR Zarr
925-700mb DPT Diff	`925mb_700mb_DPT`	K	✅	❌	1-hour	HRRR Zarr*
925-700mb DPT Lapse	`925mb_700mb_DPT_lapse`	K	❌	✅	1-hour	HRRR Zarr*	DPT(925) - DPT(700); layer difference
925-700mb Mean DPT	`925mb_700mb_mean_DPT`	K	❌	✅	1-hour	HRRR Zarr*
850-700mb DPT Diff	`850mb_700mb_DPT`	K	✅	❌	1-hour	HRRR Zarr*
850-700mb DPT Lapse	`850mb_700mb_DPT_lapse`	K	❌	✅	1-hour	HRRR Zarr*	DPT(850) - DPT(700); layer difference
850-700mb Mean DPT	`850mb_700mb_mean_DPT`	K	❌	✅	1-hour	HRRR Zarr*
850-500mb DPT Diff	`850mb_500mb_DPT`	K	✅	❌	1-hour	HRRR Zarr*
850-500mb DPT Lapse	`850mb_500mb_DPT_lapse`	K	❌	✅	1-hour	HRRR Zarr*	DPT(850) - DPT(500); layer difference
850-500mb Mean DPT	`850mb_500mb_mean_DPT`	K	❌	✅	1-hour	HRRR Zarr*
Sfc Equiv. Pot. Temp	`surface_theta_e`	K	✅	✅	1-hour	HRRR Zarr*
Low-Level Mixing Ratio	`lowest_100mb_mean_`<br>`mixing_ratio`	kg/kg	✅	✅	1-hour	HRRR Zarr*
LCL-LFC Rel. Humidity	`LCL_LFC_RH`	%	✅	✅	1-hour	HRRR Zarr*	relative humidity between LCL and LFC
0-3km Rel. Humidity	`0-3km_RH`	%	✅	✅	1-hour	HRRR Zarr*	0-3 km (mean) relative humidity
0-5km Rel. Humidity	`0-5km_RH`	%	✅	✅	1-hour	HRRR Zarr*	0-5 km (mean) relative humidity
Downdraft CAPE	`DCAPE`	J/kg	✅	❌	1-hour	HRRR Zarr*	calculated w/ sparse pressure levels: {src, 925, 850, 700, 500}; clip$\text{DCAPE} \in [0, 2000]$ j/kg
Low-Level Lapse Rate	`low_level_lapse_rate`	K/km	✅	✅	1-hour	HRRR Zarr*
3hr Lapse Rate Change	`3hr_lapse_rate_change`	K/km	✅	❌	1-hour	HRRR Zarr*
Sfc-850mb PWAT	`sfc_850_pw`	mm	✅	✅	1-hour	HRRR Zarr*	pwat, surface to 850mb
Sfc-700mb PWAT	`sfc_700_pw`	mm	✅	✅	1-hour	HRRR Zarr*	pwat, surface to 700mb
Total Column PWAT	`entire_atmosphere_`<br>`single_layer_PWAT`	kg/m^2	✅	✅	1-hour	HRRR Zarr
Moisture Flux Conv.	`moisture_flux_`<br>`convergence`	s⁻¹	❌	✅	1-hour	HRRR Zarr*
Orographic Vert. Vel.	`w_oro`	m/s	❌	✅	1-hour	HRRR Zarr*
Bright Band Bottom	`BrightBandBottom`<br>`Height_00.00`	m	✅	✅	2-min	NOAA MRMS
Bright Band Top	`BrightBand`<br>`TopHeight_00.00`	m	✅	✅	2-min	NOAA MRMS
1hr Max Composite Refl.	`CREF_1HR_MAX_00.50`	dBZ	✅	✅	2-min	NOAA MRMS
Radar QPE Quality Idx	`RadarAccumulation`<br>`QualityIndex_01H_00.00`	non-dim	✅	✅	2-min	NOAA MRMS
Refl. at -5C	`Reflectivity_-5C_00.50`	dBZ	✅	✅	2-min	NOAA MRMS
Refl. at -10C	`Reflectivity_-10C_00.50`	dBZ	✅	✅	2-min	NOAA MRMS
Refl. at -15C	`Reflectivity_-15C_00.50`	dBZ	✅	✅	2-min	NOAA MRMS
Refl. at -20C	`Reflectivity_-20C_00.50`	dBZ	✅	✅	2-min	NOAA MRMS
Refl. at 0C	`Reflectivity_0C_00.50`	dBZ	✅	✅	2-min	NOAA MRMS
Lowest Alt. Refl.	`ReflectivityAt`<br>`LowestAltitude_00.50`	dBZ	✅	✅	2-min	NOAA MRMS
Seamless Hybrid Scan Refl.	`SeamlessHSR_00.00`	dBZ	✅	✅	2-min	NOAA MRMS
Seamless HSR Height	`SeamlessHSRHeight_00.00`	km	✅	✅	2-min	NOAA MRMS
Vert. Integrated Liquid	`VIL_00.50`	kg/m^2	✅	✅	2-min	NOAA MRMS
\*Fields derived from Zarr HRRR

Alignment

Temporal

Spatial

Miscillaneous Notes

All MRMS/HRRR fields are stored in Zarray (zarr) format and are condense via Blosc compression**.

python

CODEC = BloscCodec(cname="zstd", clevel=3, shuffle="bitshuffle")