CoolFace
Datasetpublic

ibm-nasa-geospatial/hls_merra2_gppFlux

Dataset Summary: This dataset consists of Harmonized Landsat and Sentinel-2 multispectral reflectance imagery and MERRA-2 observations centered around eddy covariance flux towers and the corresponding Gross Primary Productivity (GPP) data at the towers. Its purpose is to serve as a finetuning dataset for geospatial foundation models for the task of regressing GPP flux observations from HLS and MERRA-2 data. Dataset Structure: The dataset consists of: (1) HLS… See the full description on the dataset page: https://huggingface.co/datasets/ibm-nasa-geospatial/hls_merra2_gppFlux.

sourceHugging Facecc-by-4.0updated 2y agoView on Hugging Face
1likes344downloads
Dataset Card

Dataset Summary:

This dataset consists of Harmonized Landsat and Sentinel-2 multispectral reflectance imagery and MERRA-2 observations centered around eddy covariance flux towers and the corresponding Gross Primary Productivity (GPP) data at the towers. Its purpose is to serve as a finetuning dataset for geospatial foundation models for the task of regressing GPP flux observations from HLS and MERRA-2 data.

Dataset Structure:

The dataset consists of: <br/> (1) HLS 6-band Tiff files of dimension 50x50x6, with the center of the chip colocated with flux tower locations, <br /> (2) 10-dimensional vector of MERRA-2 variables for each chip (1x1x10) recording temperature, soil moisture, heat flux, radiation, precipitation at the flux towers, <br /> (3) Daily GPP data derived from the eddy covariance measurements using the night-time partitioning approach at 37 flux tower sites distributed globally spanning 2018 to 2021. There are a total of 975 instances. MERRA-2 data and GPP flux observations are recorded as csv files, with a row corresponding to each HLS chip.

HLS Band Order:

1, Blue, B02 <br /> 2, Green, B03 <br /> 3, Red, B04 <br /> 4, NIR, B8A <br /> 5, SW 1, B11 <br /> 6, SW 2, B12

MERRA-2 observations:

  1. 1.[M2T1NXSLV] T2MIN,
  2. 2.[M2T1NXSLV] T2MAX,
  3. 3.[M2T1NXSLV] T2MEAN,
  4. 4.[M2T1NXSLV] TSMDEWMEAN,
  5. 5.[M2T1NXLND] GWETROOT,
  6. 6.[M2T1NXLND] LHLAND,
  7. 7.[M2T1NXLND] SHLAND,
  8. 8.[M2T1NXLND] SWLAND,
  9. 9.[M2T1NXLND] PARDFLAND,
  10. 10.[M2T1NXLND] PRECTOTLAND

Data Splits:

The dataset consists of 975 chips and we split the dataset based on years to create train test splits. Given the relatively small size of the dataset, we use a leave-one-year-out-cross-validation approach to train and evaluate. The number of observations vary across years. In this repo, we have used three years for training and one year as test.

Dataset Creation:

The flux observation sites guide our HLS and MERRA-2 data preparation process. HLS data is preselected with a 25% maximum cloud threshold and 75% minimum spatial threshold. We note the lat/long of the flux stations and extract 50x50 size HLS chips around each station and retain scenes with less than 2% snow cover and less than 5% cloud cover. HLS reflectance is transformed by the scaling factor. We record the daily hourly mean of each abovementioned MERRA-2 data at each of the flux location sites. We then apply a quality check on the daily GPP data and retain those with at least 60% of high quality hourly and half-hourly data for temporal aggregation. We further removed data with large negative GPP values (GPP <= -0.1gCm<sup>-2</sup>s<sup>-1</sup>) indicative of poor data quality. After matching the corresponding input HLS and MERRA-2 data for each GPP record, we removed instances that had large negative Enhanced Vegetation Index (EVI) values (EVI <= -0.1), indicative of cloud/snow impacts. EVI values were computed as the spatial average of the HLS chips.

Source data:

  1. 1.HLS Imagery is from V2.0 of HLS. A full description and access to HLS may be found at https://hls.gsfc.nasa.gov/ <br/>
  2. 2.MERRA-2 data is from V5.12.4. Full description and to the MERRA-2 datasets can be found at https://gmao.gsfc.nasa.gov/reanalysis/MERRA-2/data_access/ <br/>
  3. 3.Eddy covariance data were obtained from the AmeriFlux FLUXNET (https://ameriflux.lbl.gov/data/data-policy/) and ICOS Warm Winter 2020 datasets (https://www.icos-cp.eu/data-products/2G60-ZHAK). (List of eddy covariance flux sites used in the study and the respective references can be found below).
Site IDIGBPCitation
BE-LcrDBF(Warm Winter 2020 Team, 2022)
CA-DB2WET(Knox, 2022)
DK-GdsENF(Warm Winter 2020 Team, 2022)
FI-KenENF(Warm Winter 2020 Team, 2022)
FI-QvdCRO(Warm Winter 2020 Team, 2022)
FR-TouGRA(Warm Winter 2020 Team, 2022)
GL-DskOSH(Warm Winter 2020 Team, 2022)
IE-CraWET(Warm Winter 2020 Team, 2022)
IT-BFtDBF(Warm Winter 2020 Team, 2022)
PE-QFRWET(Griffis and Roman, 2021)
US-BZoWET(Euskirchen, 2022)
US-CS1CRO(Desai, 2022a)
US-CS2ENF(Desai, 2022b)
US-CS3CRO(Desai, 2022c)
US-CS4CRO(Desai, 2022d)
US-DFCCRO(Duff and Desai, 2023)
US-DS3CRO(Desai, 2022c)
US-HB1WET(Forsythe et al., 2021)
US-HB2ENF(Forsythe et al., 2023a)
US-HB3ENF(Forsythe et al., 2023b)
US-Mo2GRA(Schreiner-McGraw, 2023)
US-RGACRO(Schuppenhauer et al., 2023b)
US-RGBCRO(Schuppenhauer et al., 2023a)
US-RGoCRO(Schuppenhauer et al., 2023c)
US-SnfGRA(Kusak et al., 2022)
US-Tw5WET(Valach et al., 2022)
US-xABENF(NEON (National Ecological Observatory Network), 2023a)
US-xBAWET(NEON (National Ecological Observatory Network), 2023b)
US-xBLDBF(NEON (National Ecological Observatory Network), 2023c)
US-xBNENF(NEON (National Ecological Observatory Network), 2023d)
US-xDJENF(NEON (National Ecological Observatory Network), 2023e)
US-xJROSH(NEON (National Ecological Observatory Network), 2023f)
US-xSCDBF(NEON (National Ecological Observatory Network), 2023g)
US-xSJSAV(NEON (National Ecological Observatory Network), 2023h)
US-xSLCRO(NEON (National Ecological Observatory Network), 2023i)
US-xWDGRA(NEON (National Ecological Observatory Network), 2023j)
US-xYEENF(NEON (National Ecological Observatory Network), 2023k)

Flux Dataset References

  1. 1.Desai, A.: AmeriFlux FLUXNET-1F US-CS1 Central Sands Irrigated Agricultural Field, Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). AmeriFlux; Univ. of Wisconsin, Madison, WI (United States), https://doi.org/10.17190/AMF/1881576, 2022a.
  1. 1.Desai, A.: AmeriFlux FLUXNET-1F US-CS2 Tri county school Pine Forest, Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). AmeriFlux; Univ. of Wisconsin, Madison, WI (United States), https://doi.org/10.17190/AMF/1881577, 2022b.
  1. 1.Desai, A.: AmeriFlux FLUXNET-1F US-CS3 Central Sands Irrigated Agricultural Field, Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). AmeriFlux; Univ. of Wisconsin, Madison, WI (United States), https://doi.org/10.17190/AMF/1881578, 2022c.
  1. 1.Desai, A.: AmeriFlux FLUXNET-1F US-CS4 Central Sands Irrigated Agricultural Field, Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). AmeriFlux; Univ. of Wisconsin, Madison, WI (United States), https://doi.org/10.17190/AMF/1881579, 2022d.
  1. 1.Duff, A. and Desai, A.: AmeriFlux FLUXNET-1F US-DFC US Dairy Forage Research Center, Prairie du Sac, Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; University of Wisconsin Madison; US Dairy Forage Research Center, https://doi.org/10.17190/AMF/2006975, 2023.
  1. 1.Euskirchen, E.: AmeriFlux FLUXNET-1F US-BZo Bonanza Creek Old Thermokarst Bog, Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). AmeriFlux; University of Alaska Fairbanks, Institute of Arctic Biology, https://doi.org/10.17190/AMF/1881571, 2022.
  1. 1.Forsythe, J. D., Kline, M. A., and O’Halloran, T. L.: AmeriFlux FLUXNET-1F US-HB1 North Inlet Crab Haul Creek, Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). AmeriFlux; Clemson Univ., SC (United States), https://doi.org/10.17190/AMF/1832160, 2021.
  1. 1.Forsythe, J. D., Kline, M. A., and O’Halloran, T. L.: AmeriFlux FLUXNET-1F US-HB2 Hobcaw Barony Mature Longleaf Pine, Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; Clemson Univ., SC (United States), https://doi.org/10.17190/AMF/2229377, 2023a.
  1. 1.Forsythe, J. D., Kline, M. A., and O’Halloran, T. L.: AmeriFlux FLUXNET-1F US-HB3 Hobcaw Barony Longleaf Pine Restoration, Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; Clemson Univ., SC (United States), https://doi.org/10.17190/AMF/2229378, 2023b.
  1. 1.Goldstein, A.: FLUXNET2015 US-Blo Blodgett Forest, https://doi.org/10.18140/FLX/1440068, 2016.
  1. 1.Griffis, T. and Roman, T.: AmeriFlux FLUXNET-1F PE-QFR Quistococha Forest Reserve, Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). AmeriFlux; Univ. of Minnesota, Minneapolis, MN (United States); USDA-Forest Service, https://doi.org/10.17190/AMF/1832157, 2021.
  1. 1.Knox, S.: AmeriFlux FLUXNET-1F CA-DB2 Delta Burns Bog 2, Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). AmeriFlux; The University of British Columbia, https://doi.org/10.17190/AMF/1881564, 2022.
  1. 1.Kusak, K., Sanchez, C. R., Szutu, D., and Baldocchi, D.: AmeriFlux FLUXNET-1F US-Snf Sherman Barn, Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). AmeriFlux; Univ. of California, Berkeley, CA (United States), https://doi.org/10.17190/AMF/1854371, 2022.
  1. 1.NEON (National Ecological Observatory Network): AmeriFlux FLUXNET-1F US-xAB NEON Abby Road (ABBY), Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; National Ecological Observatory Network, https://doi.org/10.17190/AMF/2229403, 2023a.
  1. 1.NEON (National Ecological Observatory Network): AmeriFlux FLUXNET-1F US-xBA NEON Barrow Environmental Observatory (BARR), Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; National Ecological Observatory Network, https://doi.org/10.17190/AMF/2229404, 2023b.
  1. 1.NEON (National Ecological Observatory Network): AmeriFlux FLUXNET-1F US-xBL NEON Blandy Experimental Farm (BLAN), Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; National Ecological Observatory Network, https://doi.org/10.17190/AMF/2229405, 2023c.
  1. 1.NEON (National Ecological Observatory Network): AmeriFlux FLUXNET-1F US-xBN NEON Caribou Creek - Poker Flats Watershed (BONA), Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; National Ecological Observatory Network, https://doi.org/10.17190/AMF/2229406, 2023d.
  1. 1.NEON (National Ecological Observatory Network): AmeriFlux FLUXNET-1F US-xDJ NEON Delta Junction (DEJU), Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; National Ecological Observatory Network, https://doi.org/10.17190/AMF/2229407, 2023e.
  1. 1.NEON (National Ecological Observatory Network): AmeriFlux FLUXNET-1F US-xJR NEON Jornada LTER (JORN), Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; National Ecological Observatory Network, https://doi.org/10.17190/AMF/2229408, 2023f.
  1. 1.NEON (National Ecological Observatory Network): AmeriFlux FLUXNET-1F US-xSC NEON Smithsonian Conservation Biology Institute (SCBI), Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; National Ecological Observatory Network, https://doi.org/10.17190/AMF/2229409, 2023g.
  1. 1.NEON (National Ecological Observatory Network): AmeriFlux FLUXNET-1F US-xSJ NEON San Joaquin Experimental Range (SJER), Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; National Ecological Observatory Network, https://doi.org/10.17190/AMF/2229410, 2023h.
  1. 1.NEON (National Ecological Observatory Network): AmeriFlux FLUXNET-1F US-xSL NEON North Sterling, CO (STER), Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; National Ecological Observatory Network, https://doi.org/10.17190/AMF/2229411, 2023i.
  1. 1.NEON (National Ecological Observatory Network): AmeriFlux FLUXNET-1F US-xWD NEON Woodworth (WOOD), Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; National Ecological Observatory Network, https://doi.org/10.17190/AMF/2229412, 2023j.
  1. 1.NEON (National Ecological Observatory Network): AmeriFlux FLUXNET-1F US-xYE NEON Yellowstone Northern Range (Frog Rock) (YELL), Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; National Ecological Observatory Network, https://doi.org/10.17190/AMF/1985459, 2023k.
  1. 1.Schreiner-McGraw, A.: AmeriFlux FLUXNET-1F US-Mo2 LTAR CMRB Tucker Prairie (CMRB TP), Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; USDA Agricultural Research Service, Cropping Systems & Water Quality Research Unit, https://doi.org/10.17190/AMF/2229383, 2023.
  1. 1.Schuppenhauer, M., Biraud, S. C., and Chan, S.: AmeriFlux FLUXNET-1F US-RGB Butte County Rice Farm, Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; Arva Intelligence Corp, https://doi.org/10.17190/AMF/2204874, 2023a.
  1. 1.Schuppenhauer, M. R., Biraud, S. C., and Chan, S.: AmeriFlux FLUXNET-1F US-RGA Arkansas Corn Farm, Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; Arva Intelligence Corp., https://doi.org/10.17190/AMF/2204873, 2023b.
  1. 1.Schuppenhauer, M. R., Biraud, S. C., and Chan, S.: AmeriFlux FLUXNET-1F US-RGo Glenn County Organic Rice Farm, Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States). AmeriFlux; Arva Intelligence Corp., https://doi.org/10.17190/AMF/2204875, 2023c.
  1. 1.Valach, A., Kasak, K., Szutu, D., Verfaillie, J., and Baldocchi, D.: AmeriFlux FLUXNET-1F US-Tw5 East Pond Wetland, Lawrence Berkeley National Lab. (LBNL), Berkeley, CA (United States). AmeriFlux; Univ. of California, Berkeley, CA (United States), https://doi.org/10.17190/AMF/1881595, 2022.
  1. 1.Warm Winter 2020 Team: Warm Winter 2020 ecosystem eddy covariance flux product for 73 stations in FLUXNET-Archive format—release 2022-1 (Version 1.0), https://doi.org/10.18160/2G60-ZHAK, 2022.

Citation