ChrisRPL/satellite-disruption-triage-aux-v1-1

Satellite Disruption Triage Dataset v1.1 — Auxiliary Dataset

⚠️ Important: This is auxiliary data, not a canonical benchmark

This dataset is a public auxiliary resource for vision-language model (VLM) research on macro-scale civilian disruption triage from satellite imagery. It is explicitly not a canonical benchmark, not expert-labeled core truth, and not a drop-in substitute for a production satellite triage system. It is suitable for auxiliary VLM training, robustness experiments, transfer learning, and structured-output evaluation, but any production use requires domain-specific validation, expert review, and task-specific internal evaluation.

What v1.1 fixes over v1

What this dataset contains

322 examples of paired satellite images (baseline + current) from 17 global civilian disaster events, annotated with structured JSON triage outputs. This is a material improvement over v0 (120 examples) and v1 (218 examples) in split integrity, per-row modality documentation, and label cleanliness.

Modality mix (per-row documented)

Every example contains a modality field with one of the following exact values:

The optical-to-SAR pairs (173 examples, 54%) introduce a cross-modality challenge: radar speckle, different viewing geometries, and surface moisture effects can create change-detection artifacts unrelated to actual physical damage. Models trained on this subset should be evaluated with explicit awareness of this modality gap.

The optical-to-optical pairs (149 examples, 46%) avoid the SAR gap but may still exhibit seasonality, sun-angle, and resolution differences between pre- and post-event acquisitions.

Per-row modality is available in both the flat JSONL and SFT messages formats. No mixed-modality per-example metadata is needed beyond the source-level distinction above.

Target schema

Each example contains a baseline image, a current image, and a structured JSON output with exactly these fields:

Action definitions

• —`discard` — No meaningful macro-scale civilian disruption visible. Buildings and infrastructure appear structurally intact in the current image compared to baseline.
• —`defer` — Minor to moderate civilian infrastructure disruption visible. Damage is present but localized; may warrant later review but does not indicate immediate widespread impact requiring urgent action.
• —`downlink_now` — Significant macro-scale civilian disruption visible (widespread building damage/destruction, flooding, wildfire destruction, etc.). Urgently warrants satellite downlink and humanitarian response coordination.

Category taxonomy

Excluded categories: No explicit hospital/humanitarian site labels, no road-access labels, no military tactical data. The dataset is scoped to civilian macro-scale building disruption only.

What this dataset is NOT

1. 1.Not a canonical benchmark — This is an auxiliary training/evaluation resource. It should not be used as the sole metric for comparing production satellite triage systems.
2. 2.Not expert-labeled core truth — Labels are algorithmically derived from pixel-level damage masks using fixed thresholds. Rationale text is template-generated. No domain experts (e.g., emergency responders, GIS analysts) reviewed the outputs.
3. 3.Not a drop-in substitute for production evaluation — Any operational system requires task-specific internal evaluation on data that matches its actual sensor mix, resolution, and geographic scope.
4. 4.Not a military targeting dataset — No military objectives, strike damage, or tactical analysis labels.
5. 5.Not a real-time monitoring claim — Labels are based on post-event archived satellite imagery and do not imply real-time detection capability.
6. 6.Not a tiny-object detection dataset — Focus is on macro-scale (tile-level or large-cluster) disruption, not individual small-object detection.
7. 7.Not a hospital/humanitarian-site-specific dataset — While we prefer civilian-relevant labels, source data lacks explicit hospital or humanitarian site annotations.

Split policy: event-held-out

The eval split is event-held-out: no source-event family appears in both train and eval. This prevents the model from memorizing event-specific visual patterns (e.g., "all Beirut tiles look like explosions") and produces a stronger generalization signal.

The eval set was selected to ensure representation of all three action classes across held-out events. Eval events include explosions, wildfires, hurricanes, and mixed-damage storms.

Source datasets used

Primary source 1: BRIGHT (XView2 format)

• —HF repo: GabeT29/BRIGHT-XView2Format
• —Parent repo: Kullervo/BRIGHT
• —Paper: Chen et al., "BRIGHT: a globally distributed multimodal building damage assessment dataset with very-high-resolution for all-weather disaster response", Earth System Science Data, 17, 6217–6253, 2025. DOI: 10.5194/essd-17-6217-2025
• —License: CC-BY-NC-4.0 (Maxar Open Data)
• —Coverage in v1.1: 173 examples across 11 disaster events
• —Modality: optical-to-SAR (cross-modality)

BRIGHT provides paired pre-event optical and post-event SAR images with pixel-level change masks:

• —0 = background (no building)
• —1 = intact / unchanged
• —2 = damaged
• —3 = destroyed

Damage computation in v1.1: Building pixels are counted where mask >= 1 (intact+damaged+destroyed). Damaged pixels are mask >= 2. Destroyed pixels are mask == 3. This is more conservative than v1, which counted any positive pixel as damage.

Primary source 2: xView2-derived (ToRChange format)

• —HF repo: EVER-Z/torchange_xView2
• —Original challenge: xView2 Building Damage Assessment Challenge
• —License: CC-BY-NC-4.0 (Maxar Open Data)
• —Coverage in v1.1: 149 examples across 10 disaster events
• —Modality: optical-to-optical (same-modality)

The xView2-derived subset provides paired pre/post optical images with building-footprint and damage masks following the xBD Joint Damage Scale:

• —0 = background
• —1 = no-damage
• —2 = minor-damage
• —3 = major-damage
• —4 = destroyed

Damage is computed from the difference between pre-mask (building footprint) and post-mask (damage state). Change pixels with delta >= 1 indicate affected areas.

Datasets inspected but not used as primary source

Labeling methodology

Labels were derived algorithmically from source pixel-level masks with the following pipeline:

1. 1.Damage statistics: For each 1024×1024 source tile, we computed damage ratios from the mask using source-appropriate legends (see Source Datasets above).
2. 2.Triage mapping:
3. 3.damage_ratio < 0.05 → discard
4. 4.0.05 ≤ damage_ratio < 0.35 AND destruction_ratio < 0.15 → defer
5. 5.Otherwise → downlink_now
6. 6.Category: Inferred from source event name (e.g., "turkey-earthquake" → earthquake_building_damage). For xView2-derived tiles where event names are flattened, we parse the image filename and apply heuristics; ambiguous cases are labeled building_disruption_unknown.
7. 7.Rationale: Template-generated based on action + category, written in operational language.
8. 8.Bounding box: Computed as the axis-aligned bounding box of all damaged/destroyed pixels, normalized to [0, 1] per tile. null for discard examples.
9. 9.Whole-frame bbox filtering (v1.1): Bounding boxes covering > 80% of the tile area are considered weak signals (often indicating mask bleed or imprecise change detection). These bboxes are discarded and the action is downgraded:
10. 10.downlink_now → defer
11. 11.defer → discard
12. 12.Ratio clamping (v1.1): Both damage_ratio and destruction_ratio are clamped to [0.0, 1.0] to prevent floating-point edge cases from exceeding valid bounds.
13. 13.Deduplication: Perceptual hash (16×16 image thumbnail) on post-event tiles to exclude near-identical examples across sources.
14. 14.Event-held-out split (v1.1): Eval events are chosen to ensure all three action classes are represented while ensuring no eval event family appears in train.

File formats

This repository provides two machine-readable formats describing the same underlying examples:

1. Flat JSONL (train_flat.jsonl, eval_flat.jsonl)

For structured evaluation and non-conversational training. Each row is a JSON object with scalar fields and a target_output JSON object.

{
  "example_id": "v1_1_bright_val_images_beirut-explosion_00000016",
  "baseline_image": "images/baseline/v1_1_bright_val_images_beirut-explosion_00000016_baseline.png",
  "current_image": "images/current/v1_1_bright_val_images_beirut-explosion_00000016_current.png",
  "target_output": {
    "action": "downlink_now",
    "category": "civilian_explosion_damage",
    "rationale": "Severe civilian explosion damage visible in Beirut Explosion imagery. Multiple buildings destroyed across a wide blast radius, indicating potential humanitarian emergency requiring urgent satellite downlink.",
    "bbox_norm": [0.0, 0.0928, 0.999, 0.999]
  },
  "source_dataset": "BRIGHT (GabeT29/BRIGHT-XView2Format)",
  "source_event": "beirut-explosion",
  "source_image_name": "val/images/beirut-explosion_00000016",
  "modality": "optical-to-SAR",
  "provenance": "BRIGHT dataset, event=beirut-explosion, tile=val/images/beirut-explosion_00000016",
  "damage_ratio": 0.235,
  "destruction_ratio": 0.235
}

2. SFT-ready messages format (train_sft.jsonl, eval_sft.jsonl)

For vision-language model fine-tuning in standard conversation format. Each row contains messages conforming to the messages field expected by SFT trainers (e.g., TRL, LLaMA-Factory).

{
  "example_id": "v1_1_bright_val_images_beirut-explosion_00000016",
  "images": [
    "images/baseline/v1_1_bright_val_images_beirut-explosion_00000016_baseline.png",
    "images/current/v1_1_bright_val_images_beirut-explosion_00000016_current.png"
  ],
  "messages": [
    {
      "role": "system",
      "content": "You are a satellite imagery analyst performing structured disruption triage..."
    },
    {
      "role": "user",
      "content": "Compare these two satellite images of the same location..."
    },
    {
      "role": "assistant",
      "content": "{\n  \"action\": \"downlink_now\",\n  \"category\": \"civilian_explosion_damage\",\n  ...\n}"
    }
  ],
  "source_dataset": "BRIGHT (GabeT29/BRIGHT-XView2Format)",
  "modality": "optical-to-SAR",
  "provenance": "..."
}

Note on image tokens: The SFT format uses placeholder text tokens (<image_baseline>, <image_current>) in the user message. Actual image injection depends on the training framework. The images array lists the two image paths in [baseline, current] order.

How to load

from datasets import load_dataset

# Flat format
ds = load_dataset("ChrisRPL/satellite-disruption-triage-v1-1", data_files="train_flat.jsonl", split="train")

# SFT format
ds_sft = load_dataset("ChrisRPL/satellite-disruption-triage-v1-1", data_files="train_sft.jsonl", split="train")

# Access images by path
from PIL import Image
example = ds[0]
baseline = Image.open(example["baseline_image"])
current = Image.open(example["current_image"])

Action distribution

The dataset is discard-heavy (~70% discard overall). This reflects the underlying source data: most satellite tiles in disaster-event image collections show no damage or intact buildings. The eval set is more balanced (40% discard, 18% defer, 43% downlink_now) because held-out events were chosen to include high-damage hurricanes and wildfires. The train set's defer class is small because many moderate-damage tiles had weak whole-frame bboxes that were downgraded to discard during v1.1 cleaning.

Source distribution

Main limitations

1. 1.Algorithmic labels with template rationales — no expert review — Triage actions are derived from pixel-level mask statistics using fixed thresholds, not human expert judgment. Rationale text is template-generated rather than written by domain experts. A production system should include human-in-the-loop validation or at least expert spot-checking.

1. 1.Discard-heavy class imbalance — Approximately 70% of examples are discard (no visible damage). This is an honest property of the source data but means the dataset has limited positive-class density for training a high-recall damage detector. The eval set is more balanced (40% discard) but the train set is heavily skewed.

1. 1.Modality gap in the BRIGHT subset (54% of data) — 173 examples are optical-to-SAR cross-modality pairs. Radar speckle, viewing geometry, and surface moisture differences can create false-change signals that a VLM may learn as "damage." The xView2 subset (46%) avoids this but is smaller.

1. 1.Ambiguous event inference for some xView2-derived tiles — The xView2 HF reformat flattens filenames; event type is inferred by parsing socal-fire_00000880 style names. Most are confidently mapped, but a small number receive the fallback label building_disruption_unknown when inference is ambiguous.

1. 1.Weak whole-frame bbox tradeoff — v1.1 discards bounding boxes covering > 80% of the tile and downgrades the associated action. This improves bbox precision but reduces the defer class count (from ~79 in v1 to ~28 in v1.1) because many moderate-damage tiles had imprecise whole-frame change masks.

1. 1.License restrictions — Derived from Maxar Open Data under CC-BY-NC-4.0. Not for commercial use without additional licensing from Maxar.

1. 1.Not exhaustively complete — 322 examples is larger than v0 and v1 but still modest. It is suitable for few-shot fine-tuning, auxiliary training mixes, and evaluation, but not sufficient as a standalone training corpus for a full production VLM.

Version history

• —v0.1.0 (2025-04-23) — Initial release. 120 examples from 11 BRIGHT events. 92 train / 28 eval.
• —v0.1.1 (2025-04-24) — Repaired dataset card. Removed unsupported object feature type. Clarified auxiliary status.
• —v1.0.0 (2025-04-24) — Expanded to 218 examples from BRIGHT (135) and xView2-derived (83). Added per-source modality documentation.
• —v1.1.0 (2025-04-24) — Event-held-out split (6 eval events, 11 train events). Per-row modality field. Ratio clamping to [0,1]. Whole-frame bbox filtering (>0.8 area nullified). Improved label quality. Honest auxiliary framing.

Citation

@dataset{satellite_disruption_triage_v1_1,
  title = {Satellite Disruption Triage Dataset v1.1 (Auxiliary)},
  author = {Hugging Face Agent},
  year = {2025},
  url = {https://huggingface.co/datasets/ChrisRPL/satellite-disruption-triage-v1-1}
}

@article{Chen2025Bright,
  AUTHOR = {Chen, H. and Song, J. and Dietrich, O. and Broni-Bediako, C. and Xuan, W. and Wang, J. and Shao, X. and Wei, Y. and Xia, J. and Lan, C. and Schindler, K. and Yokoya, N.},
  TITLE = {BRIGHT: a globally distributed multimodal building damage assessment dataset with very-high-resolution for all-weather disaster response},
  JOURNAL = {Earth System Science Data},
  VOLUME = {17},
  YEAR = {2025},
  NUMBER = {11},
  PAGES = {6217--6253},
  DOI = {10.5194/essd-17-6217-2025}
}

@inproceedings{Gupta2019xBD,
  title = {xBD: A Dataset for Assessing Building Damage from Satellite Imagery},
  author = {Gupta, Ritwik and Goodman, Bryce and Patel, Neil and Hosfelt, Richard and Sajeev, Sandra and and others},
  booktitle = {NeurIPS 2019 Workshop on Machine Learning for the Developing World},
  year = {2019},
  url = {https://challenge.xviewdataset.org/}
}