ai-for-good-lab/ai4g-flood-dataset

Flood Detection Dataset

Introduction

This dataset accompanies the paper Mapping global floods with 10 years of satellite radar data (Nature Communications, 2025) and contains global flood detections derived from Sentinel-1 Synthetic Aperture Radar (SAR) imagery using a deep learning change detection model. The dataset spans October 2014 – September 2024, offering a longitudinal view of flood-prone areas worldwide.

Key features:

• —Cloud-penetrating SAR data for consistent flood detection through cloud cover and at night
• —High-resolution detections (20 m) grouped into 3° × 3° tiles
• —Auxiliary data (soil moisture, elevation, temperature, land cover) included for filtering false positives
• —GeoTIFF products with 80 m and 240 m buffers
• —Recurrence maps showing number of months with flooding detected

For full methodology and validation details, refer to the paper above.

Recommended Usage

For the most accurate results, follow these guidelines:

1. 1.Single Event Analysis If you are analyzing a specific date or flood event, start with the unfiltered parquet data. For maximum flexibility, you can also run the detection model yourself using our public GitHub repository

1. 1.Aggregating Over Extended Time Periods If you are aggregating across several months or more, apply the recommended filtering step described later in this README to reduce false positives.

1. 1.Spatial Buffering In both cases, apply an 80-meter buffer to the coordinates when generating raster outputs. This improves the accuracy of the flood extent.

The single event analysis guidline is the recommended approach for comparing against validation datasets for specific flood events and for short-term rapid response. For any aggregations across longer time periods in the paper, we apply the post-processing filters to reduce false positives.

Quick Start

# Example: Loading and filtering data for Dolo Ado in SE Ethiopia, one of the sites explored in our paper (4.17°N, 42.05°E)
import pandas as pd
import numpy as np
import rasterio
import math
from rasterio.transform import from_origin


# Load parquet data
df = pd.read_parquet('N03/N03E042/N03E042-post-processing.parquet')

# Apply recommended filters (for when aggregating over long time periods, not necessary for single event floods)
filtered_df = df[
    (df.dem_metric_2 < 10) &
    (df.soil_moisture_sca > 1) &
    (df.soil_moisture_zscore > 1) &
    (df.soil_moisture > 20) &
    (df.temp > 0) &
    (df.land_cover != 60) &
    (df.edge_false_positives == 0)
]

# when outputting as a raster, it's highly recommended to include the 80 meter buffer (4 pixels). See example code below
def df_to_geotiff(df, output_file, buffer_size=4, resolution=0.00018):
    """
    Convert a DataFrame of lat/lon points into a GeoTIFF with dilation.

    Args:
        df: DataFrame with columns ['lat', 'lon']
        output_file: Path to output GeoTIFF
        buffer_size: Number of pixels to dilate around each detection (4 → ~80 m)
        resolution: Pixel size in degrees (~0.00018 ≈ 20 m at equator)
    """
    if df.empty:
        raise ValueError("DataFrame is empty. Nothing to rasterize.")

    lats = df['lat'].values
    lons = df['lon'].values

    # Compute raster bounds
    min_lon, max_lon = lons.min(), lons.max()
    min_lat, max_lat = lats.min(), lats.max()

    width = int(math.ceil((max_lon - min_lon) / resolution)) + 1
    height = int(math.ceil((max_lat - min_lat) / resolution)) + 1

    transform = from_origin(min_lon, max_lat, resolution, resolution)

    # Initialize raster
    raster = np.zeros((height, width), dtype=np.uint8)

    # Burn points with dilation
    for lon, lat in zip(lons, lats):
        row = int((max_lat - lat) / resolution)
        col = int((lon - min_lon) / resolution)

        r_start, r_end = max(0, row - buffer_size), min(height, row + buffer_size + 1)
        c_start, c_end = max(0, col - buffer_size), min(width, col + buffer_size + 1)

        raster[r_start:r_end, c_start:c_end] = 1

    # Write GeoTIFF
    meta = {
        'driver': 'GTiff',
        'height': height,
        'width': width,
        'count': 1,
        'dtype': 'uint8',
        'crs': 'EPSG:4326',
        'transform': transform,
        'compress': 'lzw'
    }

    with rasterio.open(output_file, 'w', **meta) as dst:
        dst.write(raster, 1)

# creates geotiff with 20 meter resolution at equator (~0.00018 degrees), with an 4 pixel (80 meter) buffer. 
df_to_geotiff(filtered_df, 'flood_map.tif', buffer_size=4, resolution=0.00018)

# Load corresponding geotiff
with rasterio.open('N03/N03E042/N03E042-80m-buffer.tif') as src:
    flood_data = src.read(1)  # Read first band

Overview

This dataset provides flood detection data from satellite observations. Each geographic area is divided into 3° × 3° tiles (approximately 330km × 330km at the equator).

What's in each tile?

1. 1.Parquet file (post-processing.parquet): Contains detailed observations with timestamps, locations, and environmental metrics
2. 2.80-meter buffer geotiff (80m-buffer.tif): Filtered flood extent with 80m safety buffer
3. 3.240-meter buffer geotiff (240m-buffer.tif): Filtered flood extent with wider 240m safety buffer
4. 4.Flood recurrence geotiff with 80-meter buffer (recurrence-80m-buffer.tif): Number of distinct months with flooding detected.

In the geotiffs:

• —Value 2: Pixels with flooding detected within the buffer distance (80m or 240m). For the recurrence file, it is number of months with flooding minus 1. So 2=1 month of flooding, 3=2 months of flooding, i.e., a pixel value of N indicates flooding was detected in N-1 distinct months.
• —Value 1: Default exclusion layer representing areas with potential false positives (rough terrain or arid regions) or false negatives (urban areas)
• —Value 0: Areas without any flood detection and outside of our exclusion mask

NOTE: For areas with no flooding detected after the exclusion mask is applied, recurrence files may currently be missing. For these areas, please use the exclusion mask from the 80 or 240-meter geotiffs. Updated files will be added shortly!

Finding Your Area of Interest

1. 1.Identify the coordinates of your area
2. 2.Round down to the nearest 3 degrees for both latitude and longitude
3. 3.Use these as the filename. For example:
4. 4.For Dolo Ado (4.17°N, 42.05°E)
5. 5.Round down to (3°N, 42°E)
6. 6.Look for file N03E042 in the N03 folder

Directory Structure

├── N03                                        # Main directory by latitude
│   ├── N03E042                                # Subdirectory for specific tile
│   │   ├── N03E042-post-processing.parquet    # Tabular data
│   │   ├── N03E042-80m-buffer.tif         # Geotiff with 90m buffer
│   │   └── N03E042-240m-buffer.tif            # Geotiff with 240m buffer

Data Description

Parquet File Schema

Land Cover Classes

Common values in the land_cover column:

• —10: Tree cover
• —20: Shrubland
• —30: Grassland
• —40: Cropland
• —50: Urban/built-up
• —60: Bare ground (typically excluded)
• —70: Snow/Ice
• —80: Permanent Water bodies (excluded in this dataset)
• —90: Wetland

Recommended Filtering

To reduce false positives, apply these filters:

recommended_filters = {
    'dem_metric_2': '< 10',     # Exclude steep terrain
    'soil_moisture_sca': '> 1',  # Ensure meaningful soil moisture
    'soil_moisture_zscore': '> 1', # Above normal moisture
    'soil_moisture': '> 20',    # Sufficient moisture present
    'temp': '> 0',             # Above freezing
    'land_cover': '!= 60',     # Exclude bare ground
    'edge_false_positives': '= 0' # Remove edge artifacts
}

Spatial Resolution

All files are processed with 20 meter resolution input data.

Common Issues and Solutions

1. 1.Edge Effects: If you see suspicious linear patterns near tile edges, use the edge_false_positives filter
2. 2.Desert Areas: Consider stricter soil moisture thresholds in arid regions
3. 3.Mountain Regions: You may need to adjust dem_metric_2 threshold based on your needs

Known Limitations

• —Detection quality may be reduced in urban areas and areas with dense vegetation cover
• —While we try to control for false positives, certain soil types can still lead to false positives

Citation

If you use this dataset, please cite our paper: https://www.nature.com/articles/s41467-025-60973-1

The repo also contains the geotiffs required to reproduce results shown in the paper in the 'paperplottifs' folder. These files have their own README.md file in the folder.

Questions or Issues?

Please open an issue on our GitHub repository at https://github.com/microsoft/ai4g-flood or contact us at [amitmisra@microsoft.com]