troni21/esa_philab_embed2heights

THIS IS A COPY OF THE ORIGINAL DATASET TO MAKE IT MORE ACCESSIBLE

SOURCE: https://www.eotdl.com/datasets/embed2heights?ref=philabchallenges-cms.earthpulse.es

embed2heights Challenge - Reaching New Heights with GeoFM Embeddings

Overview

The objective of the embed2heights challenge is to develop a multi-task method that uses Geospatial Foundation Model embeddings to map land cover and estimate heights at scale. Participants are asked to combine multiple embedding sources to segment buildings, vegetation, and water, and to predict building and vegetation heights.

Participants receive an AI-ready package with pre-computed embeddings from four GFMs: AlphaEarth, TESSERA, TerraMind, and THOR, plus reference labels derived from IGN airborne LiDAR products. This enables direct experimentation on feature fusion and multi-task modeling without running large-scale embedding generation.

Dataset

The training dataset includes 2,024 patches of size 256x256 at 10 m resolution, sampled over major French cities and selected rural areas in France. Labels are derived from IGN products based on airborne LiDAR.

Labels are not discrete categories. For each pixel, they represent the percentage contribution of each class within a 10x10 m cell. Label data is provided as 4-band TIFF files:

• —Band 1: percentage of building
• —Band 2: percentage of vegetation
• —Band 3: percentage of water
• —Band 4: relative height above ground (nDSM)

The source data is generated at 1 m spatial resolution and includes four classes: Background, Buildings, Trees/HighVegetation, and Unclassified. The Unclassified class captures mixed/overlap cases (for example, a tree attached to a house).

The test set (around 1,000 patches) is generated with similar data, but from different regions and years.

Training Data Folder Structure

Inside the data/train/ directory, the following subdirectories are provided:

• —alphaearth_emb: AlphaEarth pixel-level embeddings, shape (256, 256, 64), total size 33.93 GB.
• —labels: reference label tensors (not a model output), pixel-level, shape (256, 256, 4), total size 2.12 GB.
• —terramind_s1_emb: TerraMind (S1) patch-level embeddings, shape (16, 16, 768), total size 1.60 GB.
• —terramind_s2_emb: TerraMind (S2) patch-level embeddings, shape (16, 16, 768), total size 1.60 GB.
• —tessera_emb: Tessera pixel-level embeddings, shape (256, 256, 128), total size 67.82 GB.
• —thor_s1_emb: THOR (S1) patch-level embeddings, shape (16, 16, 768), total size 1.96 GB.
• —thor_s2_emb: THOR (S2) patch-level embeddings, shape (16, 16, 768), total size 1.95 GB.

All subdirectories currently contain the same number of files: 2,024.

In addition, the data/ directory also includes a catalog.parquet manifest file (~2.1 MB), with 14,169 rows and 10 columns (type, stac_version, stac_extensions, datetime, id, bbox, geometry, assets, links, repository). The catalog stores one entry per object, with 2,024 entries for each data subdirectory. The assets field provides per-file metadata such as checksum, source href, file size, and timestamp.

Baseline

A reference baseline implementation is available on GitHub:

• —<https://github.com/VMarsocci/emb2heights-baselines>

Evaluation

The team score is computed with a weighted multi-metric evaluation combining segmentation and height accuracy: mIoU_buildings (25%), mIoU_trees (15%), mIoU_water (15%), RMSE_buildings (25%), and RMSE_vegetation (20%). The final leaderboard score is the weighted mean of these five metrics.

Submission Requirements

Each submission must include predictions for all 946 test patches.

Each evaluated submission receives:

• —a public score, computed on a subset of the test set;
• —a private score, computed on the full test set and used for final ranking.

The exact patch-level composition of the public evaluation subset is not disclosed to participants.

At the end of the challenge, private scores are revealed and the final leaderboard is computed using the private score.