jq5522/aml_indo_fires

language:

• —en license: cc-by-4.0 task_categories:
• —tabular-classification
• —tabular-regression tags:
• —fire
• —wildfire
• —climate
• —geospatial
• —indonesia
• —sumatra
• —kalimantan
• —peatland
• —ERA5
• —FIRMS
• —VIIRS
• —panel-data prettyname: Indonesian Wildfire Panel Dataset (2012–2025) sizecategories:
• —1M<n<10M ---

Indonesian Wildfire Panel Dataset (2012–2025)

A spatiotemporal panel dataset covering fire activity across Sumatra and Kalimantan, Indonesia, at monthly resolution on a 0.25° grid, spanning January 2012 to December 2025. Built for machine learning research into tropical wildfire prediction and its relationship to land cover, peatlands, and climate.

Dataset Summary

Each observation corresponds to one grid cell × one calendar month. Fire detections from NASA FIRMS VIIRS are aggregated to each cell-month, then joined with ERA5 meteorological reanalysis and static land-cover features. Two models are trained on this data — an Elastic Net logistic regression baseline and a LightGBM classifier — and their predictions are stored in outputs/tables/.

Five panel files are provided at increasing levels of processing:

Geographic Scope

• —Grid resolution: 0.25° × 0.25° (aligned to ERA5 grid)
• —Land cells retained: ~1,839 (ocean cells filtered using province assignment and land-cover overlap)
• —Temporal coverage: January 2012 – December 2025 (168 months)

Files

Panel data

Visualisation layers

Model outputs

Schema — panel_core_2012_2025.parquet

Identifiers & Time

Fire Features (NASA FIRMS VIIRS SNPP)

ERA5 Climate Features (monthly reanalysis, nearest grid point)

Static Land-Cover Features

Schema — panel_eda_enriched_2012_2025.parquet

Contains all columns from panel_core_2012_2025.parquet, plus the engineered features below. All lagged variables are computed within each cell_id group to avoid cross-cell leakage.

Derived Climate Variable

Binary & Derived Land-Cover Indicators

Lagged Fire Variables (lags: 1, 2, 3, 12 months)

Pattern: {var}_lag{n} for n in {1, 2, 3, 12}.

Lagged Climate Variables (lags: 1, 2, 3, 12 months)

Pattern: {var}_lag{n} for n in {1, 2, 3, 12}.

Precipitation Anomaly & Drought Proxies

Wind Direction

Climate x Land-Cover Interactions (EDA use)

Spatial Neighbour Variables (Queen contiguity)

Schema — modelling_data_linear_2012_2025.parquet and modelling_data_nonlinear_2012_2025.parquet

Both files share the same base feature set but differ in interaction terms and categorical encoding.

Identifiers and Targets (both files)

Base Features (both files)

Spatial controls: lon_centre, lat_centre, province (raw string in nonlinear; dummies in linear).

Static land-cover: peat_fraction, oil_palm_fraction, wood_fibre_fraction, road_length_km, road_density_km_per_km2.

Lagged fire history (n/h): fire_count_nh_lag{1/2/3/12}, fire_any_nh_lag{1/2/3/12}, frp_sum_nh_lag{1/2/3/12}.

Lagged climate: tp_lag{1/2/3/12}, t2m_c_lag{1/2/3/12}, wind_speed_lag{1/2/3/12}.

Drought proxies: precip_deficit_3m, precip_deficit_3m_dryonly, tp_3m_sum_lag1.

Spatial neighbour lags: neighbor_fire_count_nh_mean_lag1, neighbor_fire_count_nh_sum_lag1, neighbor_fire_any_nh_mean_lag1, neighbor_frp_sum_nh_mean_lag1.

Interaction Terms (linear file only — 14 terms)

Categorical Encoding

Schema — outputs/tables/

Test-set evaluation results and predicted probabilities from the two trained models.

enet_final_test.csv and lgbm_final_test.csv

One row per test year (2024, 2025).

lgbm_final_test.csv additionally contains n_trees_used (number of boosting rounds selected by early stopping).

enet_test_predictions.parquet and lgbm_test_predictions.parquet

One row per cell-month observation in the final test year.

Modelling Design

Target Variable

The primary target is fire_any_nh (binary: >=1 nominal/high-confidence VIIRS detection in a cell-month). At threshold=1 the positive rate is ~33%. 513 cells burn in >50% of months (chronic hotspots). Only 236 cells never burn, meaning the negative class is genuinely informative. Consistent with Sherwood et al. (2021) and Kurniawan et al. (2025).

fire_count_nh is retained as a secondary regression target for Poisson sensitivity checks.

Train / Validation / Test Split

Inner CV (hyperparameter tuning) uses 4 expanding-window folds:

Model Classes

Elastic Net (05A): Logistic regression with elastic net penalty, solved via FISTA on GPU (JAX) or sklearn SAGA as fallback. Hyperparameters C and l1_ratio tuned by inner CV on log-loss. Features include 14 pre-computed interaction terms.

LightGBM (05B): Gradient boosted decision trees with GPU acceleration. Hyperparameters tuned via Optuna or grid search. Province and month passed as native categorical features. Early stopping applied on a held-out fold separate from the scoring fold to prevent leakage. No scaleposweight — class imbalance is left unweighted to preserve probability calibration.

Data Sources

Loading the Data

import pandas as pd
from huggingface_hub import hf_hub_download

REPO_ID = "jq5522/aml_indo_fires"

# Core panel
panel = pd.read_parquet(hf_hub_download(REPO_ID, "panel_core_2012_2025.parquet", repo_type="dataset"))

# Enriched EDA panel
panel_eda = pd.read_parquet(hf_hub_download(REPO_ID, "panel_eda_enriched_2012_2025.parquet", repo_type="dataset"))

# Model-ready: Elastic Net
panel_linear = pd.read_parquet(hf_hub_download(REPO_ID, "modelling_data_linear_2012_2025.parquet", repo_type="dataset"))

# Model-ready: LightGBM
panel_nonlin = pd.read_parquet(hf_hub_download(REPO_ID, "modelling_data_nonlinear_2012_2025.parquet", repo_type="dataset"))
panel_nonlin["province"] = panel_nonlin["province"].astype("category")
panel_nonlin["month"] = panel_nonlin["month"].astype("category")

# Model outputs — test metrics
enet_test = pd.read_csv(hf_hub_download(REPO_ID, "outputs/tables/enet_final_test.csv", repo_type="dataset"))
lgbm_test = pd.read_csv(hf_hub_download(REPO_ID, "outputs/tables/lgbm_final_test.csv", repo_type="dataset"))

# Model outputs — predicted probabilities
enet_preds = pd.read_parquet(hf_hub_download(REPO_ID, "outputs/tables/enet_test_predictions.parquet", repo_type="dataset"))
lgbm_preds = pd.read_parquet(hf_hub_download(REPO_ID, "outputs/tables/lgbm_test_predictions.parquet", repo_type="dataset"))

# GeoParquet visualisation layer (requires geopandas)
import geopandas as gpd

grid = gpd.read_parquet(hf_hub_download(REPO_ID, "visualisation/grid_025deg.parquet", repo_type="dataset"))
grid.plot()

Construction Notes

• —Ocean cell filtering: Cells are retained if they have a non-null province assignment or a non-zero value for any of peat_fraction, oil_palm_fraction, or wood_fibre_fraction. This preserves ~11 coastal cells with roads but no province assignment.
• —Fire confidence: _nh columns restrict to nominal and high VIIRS confidence classes. _lnh columns include low-confidence detections. Only _nh series are retained in the model-ready files.
• —ERA5 matching: Each grid cell centre is snapped to its nearest ERA5 gridpoint using argmin distance. ERA5 and the analysis grid share the same 0.25° resolution.
• —Panel skeleton: A complete cell × month grid is constructed for all 168 months and all land cells. Months with no fire detections are filled with zeros, not dropped.
• —Lag leakage prevention: All lagged and rolling variables are computed within cell_id groups and shifted before rolling.
• —Wind direction components (u10/v10): Retained in panel_eda_enriched but excluded from both model-ready files. Reserved for a post-prediction transboundary haze transport layer.
• —Interaction terms: Present in the linear file only. LightGBM recovers these through recursive partitioning.
• —Categorical encoding: drop_first=True for province and month dummies in the linear file. The non-linear file retains raw values for LightGBM native categorical handling.
• —Early stopping (LightGBM): A three-way chronological split is used per fold — fit set, early-stopping set, and scoring set — keeping the validation year strictly untouched during tree selection.
• —Class weighting: Neither model uses scaleposweight or class weights, preserving probability calibration for log-loss and Brier score evaluation.

Citation

If you use this dataset, please cite the upstream data sources (NASA FIRMS, Copernicus ERA5, GFW) and link to this repository.

@dataset{aml_indo_fires_2025,
author    = {Jiaqi Chen},
title     = {Indonesian Wildfire Panel Dataset (2012--2025)},
year      = {2026},
publisher = {Hugging Face},
url       = {https://huggingface.co/datasets/jq5522/aml_indo_fires}
}

Licence

Released under Creative Commons Attribution 4.0 (CC BY 4.0). Downstream use of ERA5 data is subject to the Copernicus licence.