Thamed-Chowdhury/bd-arsa-road-safety-visual-audit

BD-ARSA: Road Safety Visual Audit Dataset

A multi-task vision-language dataset for visual road-safety auditing in Bangladesh, following the LGED (Local Government Engineering Department) audit methodology. Each record pairs a road image with a structured safety audit. All 12 LGED hazard categories were assessed visually in the field by the expert auditors and all 12 appear in the schema and in evaluation. For two of them — skid_resistance (surface friction) and drainage (wet-condition behaviour) — the auditors drew on experience and simple on-site surface checks; obvious deficiencies leave visible signatures (e.g. bleeding/flushing or polished asphalt → inadequate skid resistance; standing water, scour, erosion → poor drainage), so they are visually inferable mainly in obvious cases and tend to score lowest.

This dataset accompanies the EG-ARSA model and code:

• —🤖 Model: Thamed-Chowdhury/eg-arsa-qwen3vl-8b-lora
• —💻 Code & reproduction: https://github.com/Thamed-Chowdhury/EG-ARSA
• —📄 Paper: arXiv:2608.23563 — anyone using this dataset must cite it (see Citation)

⚠️ Street-view images are fetched locally, not redistributed

The street-view tier (20,897 records) depicts Google Street View imagery, which is © Google and governed by the Google Maps Platform Terms of Service. To comply with those terms, this dataset does not redistribute the street-view images. For the street-view tier it ships only the audit annotations and the capture metadata (coordinates, heading, and the Google panorama ID). Each street-view record therefore has a null `image` field.

You reconstruct the street-view images locally, with your own Google Maps Platform API key, using the included `fetch_images.py`:

export GOOGLE_MAPS_API_KEY=your_key      # https://developers.google.com/maps/documentation/streetview
python fetch_images.py --data-dir data --out streetview_images

The script fetches each image from the official Street View Static API by panorama id (falling back to location + heading). Re-fetched images depict the same panorama but, coming from the Static API, are not byte-identical to the originals (different framing/FOV). Your use of the fetched imagery is subject to Google's terms.

The `expert_gold` and `expert_silver` images are included in this dataset — they are sourced from ARI-BUET/LGED field-audit reports and released with LGED's written permission.

CC BY 4.0 covers the annotations and metadata in this repository only, not the Google Street View imagery (see License).

Sources

Total records: 21,947. Labels follow the LGED 12-category hazard taxonomy. Image text is English with some Bengali place/feature names. The certainty of the image→hazard correspondence sets the tier hierarchy: gold > silver > streetview.

Schema (per record)

record_id              : str — globally unique
source                 : "expert_gold" | "expert_silver" | "streetview"
district               : str | null
location_id            : str
image                  : PIL.Image | null
                         #   expert_gold / expert_silver : embedded image (included)
                         #   streetview                  : null — fetch via fetch_images.py

# street-view capture metadata (populated for `streetview` rows; null otherwise)
latitude               : float | null   # requested capture latitude
longitude              : float | null   # requested capture longitude
heading                : float | null   # camera heading (degrees), looks down the road
snapped_latitude       : float | null   # where Google snapped the panorama
snapped_longitude      : float | null
pano_id                : str | null     # Google Street View panorama id (fetch key)
pano_type              : str | null     # "official" street-view panorama
pano_capture_date      : str | null     # "YYYY-MM" capture month Google overlays

location_description   : str | null    # scene description (full-audit records)
road_type              : str | null
land_use               : str | null
critical_safety_issues : list[str]     # short "Category - issue" phrases
hazards                : list[{ hazard_name, hazard_category, observation, severity }]
overall_risk_level     : "Low" | "Medium" | "High"
additional_recommendations : str | null

tasks_available        : subset of { "hazard_generation", "overall_risk", "recommendation" }
split                  : "train" | "validation" | "test"
source_expert_finding  : str | null    # provenance: the expert finding an expert_silver
                                        # image was aligned to (metadata, not a target)

hazard_category is one of the 12 canonical LGED categories: road_markings, shoulder_condition, roadside_severity, pedestrian_facilities, vision_obstruction, traffic_signs, speed_management, intersection, embankment_safety, bus_stoppage, drainage, skid_resistance.

Granularity note

• —expert_gold images are single-hazard crops → hazards has exactly one entry (no observation/severity), and the scene fields are null.
• —expert_silver and streetview images are full road scenes → multi-hazard audits (~5.5 and ~4.2 hazards/record on average) with per-hazard observation + severity and populated scene fields.

source_expert_finding is populated only for expert_silver records.

Multi-task supervision (tasks_available)

When training a multi-task head, mask the loss for tasks not in tasks_available for a given record. (expert_gold records carry only one hazard and no per-hazard observation/severity — account for that in the hazard-generation target.)

Splits

Splits are location-disjoint: a location_id never appears in more than one split. expert_gold is split by location (36 test / 20 train / 16 val of its 72 locations); expert_silver uses held-out road groups; streetview uses segment-disjoint stratified sampling with an explicit Low-class quota in val/test so per-class evaluation is meaningful.

overall_risk_level distribution (all sources; every record carries this label):

Per-source counts per split:

Class imbalance

The risk classes are imbalanced (Low ≈ 1.4 % of train). The EG-ARSA model handles this with train-only logit adjustment (τ=1) on the risk-token logits rather than loss weighting (see the code repo). For convenience, class_weights.json (repo root) also provides simple inverse-frequency weights from the training split, should you prefer a CrossEntropyLoss(weight=...) or WeightedRandomSampler baseline:

{"weights": {"Low": 23.83, "Medium": 0.85, "High": 0.56}}

Fine-tuning prompts

The prompts/ folder ships with the dataset:

• —`prompts/finetune_prompts.py` — leakage-free student prompts for fine-tuning, plus the JSON target serializer. get_finetune_prompt(record) returns the single-hazard instruction for expert_gold records and the full-audit instruction for expert_silver/streetview; render_target(record) serializes the labels into the assistant-side JSON target consistent with tasks_available; build_pair(record) returns the framework-agnostic (instruction, target) pair.
• —`prompts/generation_provenance.md` — the exact teacher prompt used to generate the street-view labels (archived for reproducibility). Do not fine-tune on it: it assumes extra images the student will not have at inference.

Quick start

from datasets import load_dataset

ds = load_dataset("Thamed-Chowdhury/bd-arsa-road-safety-visual-audit")
sample = ds["train"][0]
print(sample["record_id"], sample["overall_risk_level"])
print(sample["hazards"])
sample["image"]            # PIL.Image for expert_gold/expert_silver; None for streetview

Street-view images are `None` until you fetch them. Run python fetch_images.py --data-dir data --out streetview_images (needs a Google Maps Platform API key) to download them locally, keyed on each record's pano_id / coordinates. See the notice above.

# filter by source / class
sv_train  = ds["train"].filter(lambda r: r["source"] == "streetview")
low_only  = ds["train"].filter(lambda r: r["overall_risk_level"] == "Low")

License & citation

The annotations and metadata in this repository are released under CC BY 4.0. This license covers the audit labels, scene descriptions, recommendations, coordinates, and panorama identifiers — i.e. the content this project created.

It does not cover the street-view imagery: those images are © Google, are not redistributed here, and are obtained by the user through the official Street View Static API under the Google Maps Platform Terms of Service (see the fetch notice). The expert_gold / expert_silver images are shared under CC BY 4.0 with LGED's permission.

The street-view audit labels were generated with Google's Gemma model (the 31B teacher). Use of those labels — and of any model trained on them — is additionally subject to the Gemma Terms of Use and the Gemma Prohibited Use Policy.

Anyone who uses the BD-ARSA dataset — in research, in a product, or in any derivative dataset or model — must cite the paper:

Md Thamed Bin Zaman Chowdhury and Moazzem Hossain. EG-ARSA: An Expert-Grounded Open Model for Visual Road Safety Auditing in Low-Resource Settings. arXiv:2608.23563, 2026. <https://arxiv.org/abs/2608.23563>

@article{chowdhury2026egarsa,
  title         = {EG-ARSA: An Expert-Grounded Open Model for Visual Road Safety
                   Auditing in Low-Resource Settings},
  author        = {Chowdhury, Md Thamed Bin Zaman and Hossain, Moazzem},
  journal       = {arXiv preprint arXiv:2608.23563},
  year          = {2026},
  eprint        = {2608.23563},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  doi           = {10.48550/arXiv.2608.23563},
  url           = {https://arxiv.org/abs/2608.23563}
}

Attribution is also a license condition for the BD-ARSA annotations, which are released under CC BY 4.0.

The expert ground truth derives from on-site Road Safety Audits conducted by faculty of the Accident Research Institute (ARI), BUET, commissioned by LGED under the World Bank–financed RTIP-II (Additional Financing; P166295).