ChrisRPL/blackline-atlas-benchmark-seeds

Blackline Atlas Civilian Disruption Benchmark Seeds

Small public benchmark slices for structured civilian disruption triage.

Purpose:

• —compare prompt baselines and small VLMs on the same runnable public slices
• —regression-check structured JSON behavior before and after Blackline adaptation
• —provide a public-facing benchmark artifact without exposing the internal gold set

Included slices:

• —internal_public_seed_v0: Blackline internal public seed (Blackline Atlas), 1 rows
• —xbd_public_seed_v0: xBD public seed (xBD / xView2), 4 rows
• —spacenet8_public_seed_v0: SpaceNet 8 public seed (SpaceNet 8), 4 rows

What this repo is:

• —public benchmark-only seed material
• —canonical blackline_candidate_eval.jsonl rows plus copied image pairs
• —external transfer checks for disaster and flood disruption

What this repo is not:

• —not the internal Blackline training corpus
• —not the frozen Blackline gold eval set
• —not a live conflict feed or operational monitoring service

Layout:

• —<slice_id>/blackline_candidate_eval.jsonl: canonical runnable benchmark rows
• —<slice_id>/images/...: image pairs used by the slice
• —<slice_id>/source_labels/...: public provenance files when available

Intended use:

• —benchmark and demo material
• —cross-model comparison on a shared public slice basket
• —public reproducibility for a narrow part of the evaluation stack

Limitations:

• —intentionally small seed slices
• —external-only coverage, not full civilian lifeline taxonomy coverage
• —mixed-source public artifacts; source terms apply per slice
• —not a replacement for the full internal Blackline evaluation program

Licensing and provenance:

• —each slice keeps its original public provenance files when available
• —source terms and restrictions follow the original datasets and published artifacts referenced per slice