Datasetpublic

kshitijrajsharma/pangaea2-vhr

Mirror Notice This repository contains unofficial mirrors of the following datasets, provided solely for hash-based versioning and reproducibility of research results. This is NOT the official source. PureForest: https://huggingface.co/datasets/IGNF/PureForest mpv4ger: https://huggingface.co/datasets/recursix/geo-bench-1.0 xView2: https://xview2.org/dataset SpaceNet 3 Roads: https://spacenet.ai/spacenet-roads-dataset/ (S3: s3://spacenet-dataset/spacenet/SN3_roads/) Legal… See the full description on the dataset page: https://huggingface.co/datasets/kshitijrajsharma/pangaea2-vhr.

sourceHugging Faceotherupdated 5mo agoView on Hugging Face

1likes59downloads

Dataset Card

Mirror Notice

This repository contains unofficial mirrors of the following datasets, provided solely for hash-based versioning and reproducibility of research results. This is NOT the official source.

—PureForest: https://huggingface.co/datasets/IGNF/PureForest
—mpv4ger: https://huggingface.co/datasets/recursix/geo-bench-1.0
—xView2: https://xview2.org/dataset
—SpaceNet 3 Roads: https://spacenet.ai/spacenet-roads-dataset/ (S3: s3://spacenet-dataset/spacenet/SN3_roads/)

Legal Notice:

—These mirrors are for research reproducibility only. Users must review and comply with the license and restrictions at the official sources.
—No redistribution or commercial use is intended or permitted via this repository.
—If you are a copyright holder and wish for this mirror to be removed, please contact the repository owner.

Dataset Overview

Each dataset is published as a separate config in one Hugging Face repo with sharded Parquet files.

Dataset	Task	Samples	Image size	Bands	Source format	Raw size
PureForest	13-class tree species classification (official taxonomy)	135,569	250x250	4 (NIR, R, G, B) uint8, EPSG:2154	GeoTIFF	~25 GB
xView2	Building damage segmentation (0-4)	~10,101 pairs	1024x1024	RGB uint8, WGS84	PNG + JSON labels	~29 GB
mpv4ger	Binary solar panel classification	13,812	320x320	RGB uint8	HDF5	~1 GB
SN3 Roads	Road network extraction	3,708	1300x1300	RGB uint16, WGS84 (original PS-RGB, lossless)	GeoTIFF + GeoJSON	~25 GB

Parquet schemas

PureForest (load_dataset("...", "pureforest"))

Column	Type	Notes
`image`	Image	PNG bytes, 250x250, 4 channels in order NIR, R, G, B
`label`	int8	0-12, official PureForest class index (13 classes)
`class_name`	string	English class name, e.g. `"Deciduous oak"`, `"Maritime pine"`
`species`	string	Underscored Latin species name, e.g. `"Quercus_petraea"` (multiple species may share one `label`)
`crs`	string	`"EPSG:2154"`
`bbox`	list[float]	`[minx, miny, maxx, maxy]` in EPSG:2154

The 13 classes follow the official PureForest dictionary: four oak species map to Deciduous oak (0), two pine variants to Black pine (7), two fir species to Fir (9), etc. The species column preserves the original folder identity for users who need finer-grained labels. A name lookup table is shipped at pureforest/label_map.json.

xView2 (load_dataset("...", "xview2"))

Column	Type	Notes
`image_id`	string	e.g. `"guatemala-volcano_00000000"`
`disaster`	string	Full event name, e.g. `"guatemala-volcano"`
`disaster_type`	string	Category: `"volcano"`, `"hurricane"`, etc.
`pre_image`	Image	PNG bytes, 1024x1024 RGB
`post_image`	Image	PNG bytes, 1024x1024 RGB
`post_mask`	Image	PNG bytes, 1024x1024 grayscale; pixel values: 0=background, 1=no-damage, 2=minor, 3=major, 4=destroyed
`buildings`	string	JSON array of `{wkt, damage, uid}` from post-disaster label
`sensor`	string	e.g. `"WORLDVIEW03_VNIR"`
`gsd`	float32	Ground sample distance in metres
`geotransform`	list[float]	6-element GDAL geotransform
`crs_wkt`	string	WKT CRS string (WGS84)

mpv4ger (load_dataset("...", "mpv4ger"))

Column	Type	Notes
`image`	Image	PNG bytes, 320x320 RGB
`label`	int8	0 = no solar panel, 1 = solar panel
`location_id`	string	Raw filename stem used as identifier (encodes coordinates)

SN3 Roads (load_dataset("kshitijrajsharma/pangaea2-vhr", "sn3_roads"))

Column	Type	Notes
`image_id`	string	e.g. `"img1"`
`aoi`	string	`"AOI_2_Vegas"`, `"AOI_3_Paris"`, `"AOI_4_Shanghai"`, `"AOI_5_Khartoum"`
`image_tif`	binary	Original uint16 RGB GeoTIFF (DEFLATE-compressed, lossless). Decode with `rasterio` -- not HF Image, because PIL silently truncates uint16 RGB to uint8
`road_mask`	Image	uint8 PNG, 1300x1300, binary (1=road, 0=background); `null` for test
`roads_geojson`	string	Raw GeoJSON (LineStrings with road type, lanes, paved); `null` for test
`roads_speed_geojson`	string	Same as above plus `inferred_speed_mph`/`mps`; `null` for test
`crs`	string	`"EPSG:4326"` (also embedded in the GeoTIFF)
`bbox`	list[float]	`[minx, miny, maxx, maxy]` in EPSG:4326

No normalisation is applied. Pixel values are the original SpaceNet uint16 PS-RGB. Per-AOI and global statistics (mean, std, min, max, p1, p2, p98, p99 per band) computed from the train split are published at data/sn3_roads/stats.json for use at training time.

Decode example:

python

import io, rasterio, numpy as np, json
from datasets import load_dataset

ds = load_dataset("kshitijrajsharma/pangaea2-vhr", "sn3_roads", split="train")
sample = ds[0]

with rasterio.open(io.BytesIO(sample["image_tif"])) as src:
    array = src.read()  # (3, 1300, 1300) uint16, band order R, G, B
    transform = src.transform  # full geotransform preserved

# Normalise with published per-AOI stats
stats = json.load(open("stats.json"))["per_aoi"][sample["aoi"]]
mean = np.array(stats["mean"]).reshape(3, 1, 1)
std  = np.array(stats["std"]).reshape(3, 1, 1)
normalised = (array.astype(np.float32) - mean) / std

Road mask is built by buffering each LineString by ~2 m (ROAD_BUFFER_DEG = 2/111000°) and rasterising, following the official SpaceNet 3 evaluation convention. Test split has no labels (standard SpaceNet challenge format).

Loading

python

from datasets import load_dataset

pureforest = load_dataset("kshitijrajsharma/pangaea2-vhr", "pureforest", split="train")
xview2     = load_dataset("kshitijrajsharma/pangaea2-vhr", "xview2",     split="train")
mpv4ger    = load_dataset("kshitijrajsharma/pangaea2-vhr", "mpv4ger",    split="train")
sn3_roads  = load_dataset("kshitijrajsharma/pangaea2-vhr", "sn3_roads",  split="train")

Conversion

Convert raw source files to Parquet shards:

bash

just setup
just convert-pureforest
just convert-xview2
just convert-mpv4ger
just convert-sn3roads