AnonymousData4NeurIPS/MPCFire

1---2license: mit3task_categories:4- image-classification5- time-series-forecasting6language:7- en8tags:9- wildfire10- remote-sensing11- earth-observation12- canada13- benchmark14- hard-negative-mining15size_categories:16- n>1T17---18 19# FireMPC: A Pan-Canadian Wildfire Forecasting Benchmark20 21FireMPC is a pan-Canadian wildfire risk benchmark covering approximately one22billion hectares across all fifteen Canadian terrestrial ecozones at 1 km daily23resolution from 2000 to 2025, integrating 55 drivers across fuel, terrain,24anthropogenic, and meteorological families.25 26This release provides four pre-built sample caches that share the same27underlying data cube but differ in their training and test sample construction28strategies, enabling controlled study of FWI-guided Hard Negative Mining29(FWI-HNM) versus random negative sampling.30 31## Repository Contents32 33```34MPCFire/35├── cache_A/             # Train: random sampling  | Test: FWI-HNM36├── cache_Y/             # Train: random sampling  | Test: random sampling37├── cache_G/             # Train: FWI-HNM          | Test: FWI-HNM38├── cache_H/             # Train: FWI-HNM          | Test: random sampling39└── Entire_Canada_Maps/  # Raw driver rasters, 13 modalities x 26 years (yearly tar archives)40```41 42Each cache directory contains exactly three files:43 44| File | Size | Description |45| --- | --- | --- |46| `windows_<hash>.h5` | ~14 GB | Pre-extracted 10-day input windows + labels for every sample (positives and negatives). One HDF5 file per cache. |47| `samples_variant_<X>.json` | ~4.4 MB | Sample index: train / val / test split assignments, sample identifiers, and metadata. |48| `norm_stats.npz` | ~2 KB | Channel-wise mean and standard deviation used for input normalisation. Skips the fire-mask channel and the categorical land-cover channel. |49 50## Variant Design51 52The four caches form a 2x2 ablation grid that decouples the negative-sampling53strategy used during training from the strategy used during evaluation:54 55|         | Test = FWI-HNM | Test = Random |56| ------- | -------------- | ------------- |57| **Train = Random** | `cache_A` | `cache_Y` |58| **Train = FWI-HNM** | `cache_G` | `cache_H` |59 60* **FWI-HNM (FWI-guided Hard Negative Mining)** scores every non-fire candidate61  with a calibrated six-component CFFDRS composite (FFMC, DMC, DC, ISI, BUI,62  FWI), partitions the pool at the median, and combines hard negatives63  (fire-weather-matched non-ignitions) with representative negatives64  (low-danger baseline) in equal proportions.65* **Random sampling** draws negatives uniformly from the non-fire candidate66  pool (firemask = 5).67 68Comparing rows isolates the effect of the training-pool construction; comparing69columns isolates the effect of the evaluation-pool construction. The diagonal70pair (`cache_A`, `cache_G`) corresponds to the standard production setup; the71off-diagonal pair (`cache_Y`, `cache_H`) is used to verify that any72FWI-HNM advantage reflects genuine boundary hardening rather than train-test73distributional alignment.74 75## Raw Driver Maps (`Entire_Canada_Maps/`)76 77`Entire_Canada_Maps/` provides the underlying raster stack used to build the78sample caches above. It covers the full Canadian landmass for 2000-2025 (2679years) and is organised into 13 modality subfolders that correspond to the80drivers listed in Table 1 of the paper. Each subfolder contains one tar81archive per year; each archive holds that year's daily (or annual / static)82GeoTIFFs.83 84All rasters are stored with **integer scaling (scaledInt)** to reduce volume.85 86### Subfolder Index87 88| Folder | Channels | Source | Cadence | Notes |89| --- | --- | --- | --- | --- |90| `DEMs/` | DEM, Slope, Aspect (sin/cos), Hillshade, TPI, TWI (7 channels) | ASTER GDEM | Static | Elevation and derived terrain indices; broadcast across the daily axis |91| `ERA5/` | temperature_2m, u/v_component_of_wind_10m, snow_cover, total_precipitation_sum, surface_latent_heat_flux_sum, dewpoint_temperature_2m, surface_pressure, volumetric_soil_water_layer_1-4, temperature_2m_max, skin_temperature_max, potential_evaporation_sum, total_evaporation_sum, skin_reservoir_content, surface_net_solar_radiation_sum (18 channels) | ERA5-Land Daily Aggregate (`ECMWF/ERA5_LAND/DAILY_AGGR`) | Daily | Atmospheric reanalysis fields |92| `FWI/` | FFMC, DMC, DC, ISI, BUI, FWI (6 channels) | CFFDRS (ERA5-driven) | Daily | Canadian Forest Fire Weather Index components |93| `MCD09CMG/` | Coarse Resolution Brightness Temperature Bands 20 / 21 / 31 / 32 (4 channels) | MOD/MYD09CMG | Daily | Coarse-resolution composite brightness temperatures |94| `MCD09GA/` | Bands 1, 2, 3, 7 (4 channels) | MOD/MYD09GA | Daily | QA-filtered, gap-filled surface reflectance |95| `MCD11A1/` | LST_Day_1km, LST_Night_1km, Emis_31, Emis_32 (4 channels) | MOD/MYD11A1 | Daily | QA-filtered, gap-filled land surface temperature and emissivity |96| `MCD12Q1/` | Land Cover Class | MCD12Q1 | Annual | Land cover / land use class; broadcast across the daily axis |97| `MCD14A1/` | Active Fire (binary) | MOD/MYD14A1 | Daily | Supervision target only; **not** included as a model input channel |98| `MCD15A3H/` | LAI, FPAR (2 channels) | MCD15A3H | Daily (from 4-day composite, interpolated) | Leaf area index / fraction of absorbed PAR |99| `NDVI_EVI/` | NDVI, EVI (2 channels) | MODIS-derived | Daily | Vegetation activity indices |100| `OSMs/` | Road / Powerline / Building / Water Density (4 channels) | OSM-derived | Static | Infrastructure accessibility; broadcast across the daily axis |101| `VPD/` | Vapor Pressure Deficit | ERA5-derived | Daily | Atmospheric moisture demand |102| `Worldpop/` | Population Density | WorldPop | Annual | Human population density; broadcast across the daily axis |103 104### File Naming105 106```107Entire_Canada_Maps/<modality>/<modality>__<YYYY>.tar108```109 110After extraction, individual files are named `YYYY_MM_DD.tif` for daily111modalities or `<modality>_YYYY.tif` for static / annual modalities.112 113### Download and Extraction114 115```python116from huggingface_hub import hf_hub_download117import tarfile, pathlib118 119tar_path = hf_hub_download(120    repo_id="AnonymousData4NeurIPS/MPCFire",121    repo_type="dataset",122    filename="Entire_Canada_Maps/ERA5/ERA5__2020.tar",123)124out = pathlib.Path("./ERA5_2020")125with tarfile.open(tar_path) as tf:126    tf.extractall(out)127```128 129To pull an entire modality:130 131```python132from huggingface_hub import snapshot_download133snapshot_download(134    repo_id="AnonymousData4NeurIPS/MPCFire",135    repo_type="dataset",136    allow_patterns=["Entire_Canada_Maps/ERA5/*"],137)138```139 140## Splits141 142A temporal hold-out is used throughout:143 144| Split | Years |145| ----- | ----- |146| Train | 2000 - 2019 |147| Validation | 2020 - 2022 |148| Test  | 2023 - 2025 |149 150The three-year test window deliberately covers the record-setting 2023 fire151season alongside the more typical 2024 and 2025 seasons. All splits maintain a152fixed 1:2 positive-to-negative ratio.153 154Sample identifiers and split assignments are stored in155`samples_variant_<X>.json` inside each cache.156 157## File Format158 159`windows_<hash>.h5` is a single HDF5 file with pre-extracted 10-day driver160windows and binary labels. The hash in the filename identifies the windowing161configuration (10-day backward window, 1 km patches) and is shared across all162four caches because the underlying input cube is identical; only the163positive/negative selection differs per variant.164 165`norm_stats.npz` provides per-channel mean and standard deviation arrays. The166fire-label channel and the categorical land-cover channel are excluded from167z-score normalisation.168 169## Reading the Data170 171Quick example using `huggingface_hub`:172 173```python174from huggingface_hub import snapshot_download175 176local_dir = snapshot_download(177    repo_id="AnonymousData4NeurIPS/MPCFire",178    repo_type="dataset",179    allow_patterns=["cache_G/*"],   # download a single variant180)181```182 183Then load the HDF5 file and the sample index:184 185```python186import h5py, json, numpy as np187 188cache = f"{local_dir}/cache_G"189with open(f"{cache}/samples_variant_G.json") as f:190    samples = json.load(f)191norm = np.load(f"{cache}/norm_stats.npz")192h5 = h5py.File(next(p for p in __import__('os').listdir(cache) if p.endswith('.h5')), "r")193```194 195## Citation196 197This dataset accompanies a paper currently under anonymous peer review. A198citation entry will be added on acceptance.199 200## License201 202Released under the MIT License. The dataset is built from publicly available203products (MODIS, ERA5-Land, ASTER, WorldPop, OpenStreetMap); please consult the204licenses of the upstream sources for redistribution of derivative products.