musmb/CPAZMaL

CPAZMAL DATASET

Machine learning-ready cryospheric SAR dataset combining ascending et descending PAZ acquisitions over Mont-Blanc massif glaciers (2020–2023). The dataset contains dual-polarization (HH+HV but not for the all time serie cf Acquisition Timeline) time-series recorded with timestamps and organized by geographic groups to support GroupKFold evaluation. It consists of 8 main classes, with an average of 8 groups per class. An additional class, STUDY, can be used for evaluation and segmentation tasks (ground truth to come) with 2–4 glacier surface subclasses. Each of the 70 groups corresponds to an irregular polygon of variable size. Preprocessing on the original data included radiometric calibration, incidence angle and NEBN noise corrections, geometric transformation via IGN LiDAR HD (1m). A dataloader is provided, allowing selection of temporal windows with options for orbit type, percentage/type of geometric distortions, and extraction of all windows of the requested size from all 70 groups.

Example: Request a time-series window of size 16×16 with up to 10% layover in ascending orbit images between January 2020 and December 2020 → produces a dataset of size (3508, 16, 16, 21, 2) with 51 unique groups et 21 timestamps and the following classes repartition: | | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | | ----------- | --- | --- | --- | --- | --- | --- | --- | ---- | | Samples | 788 | 394 | 292 | 3 | 711 | 9 | 4 | 1307 | | Description | ACC | PLA | ROC | HAG | ABL | ICA | LAC | FOR |

1. Structure

CPAZMaL/
├── DATASET/
│   ├── CPAZMaL.hdf5          # Optimized HDF5 dataset for ML
│   └── CPAZMaL.txt   # Dataset summary
└── script/
    ├── load_dataset.py              # MLDatasetLoader class
    └── scenarios.py                 # 4 pre-configured ML scenarios

2. Source Code

The complete data processing pipeline is available on GitHub at the following address: https://github.com/Matthieu-Gallet/CPAZMaL_dataset.

This repository also includes example scenario templates for loading HDF5 data files.

3. Dataset Description

Study Areas

The dataset covers 70 labeled areas across 9 land surface classes in the Mont-Blanc massif. The complete detail of each groups is given in the table_groups.pdffile. A summary is provided with the following table:

Classes: ABL (Ablation zone), ACC (Accumulation zone), FOR (Forefield), HAG (Hummocky ablation glacier), ICA (Ice/snow avalanche cone), LAC (Lake), PLA (Plateau glacier), ROC (Rockfall), STUDY (Study area)

Acquisition Timeline

<p align="left"> <img src="https://cdn-uploads.huggingface.co/production/uploads/68ee4ed89a6f942b016af50f/lHqcrTZ7P3DvjVqQhGeAF.png" alt="Enhanced Acquisition Timeline" width="50%"> </p>

Data Characteristics

• —Polarization: Dual-pol (HH + HV)
• —Temporal coverage: 2008-2020
• —Resolution: ~3m (ground range)
• —Orbit: Ascending (ASC) and Descending (DSC)
• —Total acquisitions: 216 SAR images across 54 groups
• —Format: HDF5 with optimized chunking for ML

HDF5 Structure

PAZTSX_CRYO_ML.hdf5
├── data/
│   ├── {group_name}/          # e.g., ABL001, ACC002
│   │   ├── ASC/               # Ascending orbit
│   │   │   ├── HH/
│   │   │   │   ├── images     # (H, W, T) - Time series
│   │   │   │   ├── masks      # (H, W, T) - Quality masks
│   │   │   │   ├── timestamps # (T,) - Acquisition dates
│   │   │   │   └── angles_incidence
│   │   │   └── HV/
│   │   └── DSC/               # Descending orbit
│   │       ├── HH/
│   │       └── HV/
├── metadata/
│   ├── classes                # JSON array
│   └── nodata_value
└── index/
    ├── by_class/              # Fast class-based filtering
    └── temporal_ranges/       # Date range index

4. Usage

Quick Start

from load_dataset import MLDatasetLoader

# Initialize loader
loader = MLDatasetLoader('DATASET/CPAZMaL.hdf5')

# Load single group data
data = loader.load_data(
    group_name='ABL001',
    orbit='ASC',
    polarisation=['HH', 'HV'],  # Dual-pol
    start_date='20200101',
    end_date='20201231',
    scale_type='amplitude'       # 'intensity', 'amplitude', 'log10'
)

# Extract windows for ML
windows, masks, positions = loader.extract_windows(
    image=data['images'],
    mask=data['masks'],
    window_size=32,
    stride=32,                   # Non-overlapping
    max_mask_value=1,            # Mask quality threshold
    max_mask_percentage=10.0,    # Max % of bad pixels
    min_valid_percentage=50.0,   # Min % of valid pixels
    skip_optim_offset=True       # Deterministic extraction
)

Pre-configured Scenarios

Four ready-to-use ML scenarios are provided in scenarios.py:

Scenario 1: Temporal Stacking Classification

Multi-date classification with dual-pol temporal stacking. Suitable for random forest, SVM, or CNN-based classifiers.

from scenarios import scenario_1_temporal_stacking_classification

data = scenario_1_temporal_stacking_classification(
    loader=loader,
    window_size=32,              # Patch size
    max_mask_value=1,            # Quality filter
    min_valid_percentage=50.0,   # Data validity threshold
    max_mask_percentage=10.0,
    orbit='ASC',
    start_date='20200101',
    end_date='20201231',
    scale_type='amplitude',      # Transformation: 'intensity', 'amplitude', 'log10'
    skip_optim_offset=True       # Deterministic offset
)

# Returns:
# - X: (N,) array of (window_size, window_size, T, 2) - Variable T between windows
# - y: (N,) class labels (encoded as int)
# - groups: (N,) group identifiers
# - masks, satellites, positions, class_names, group_names

Parameters:

• —window_size: Spatial window dimension (default: 32)
• —scale_type: Data transformation
• —'intensity': Raw backscatter values
• —'amplitude': Square root transformation (√intensity)
• —'log10': Logarithmic scale (dB)
• —min_valid_percentage: Minimum valid pixel ratio (0-100%)
• —max_mask_value: Maximum accepted mask value (0-3)
• —skip_optim_offset: If True, use fixed (0,0) offset for deterministic results

Scenario 2: Temporal Prediction (LSTM)

Time series prediction with train/test temporal split. Designed for LSTM, GRU, or Transformer models.

from scenarios import scenario_2_temporal_prediction_lstm

data = scenario_2_temporal_prediction_lstm(
    loader=loader,
    window_size=32,
    orbit='DSC',
    polarization='HH',           # Single-pol
    train_start='20200101',
    train_end='20201031',
    predict_start='20201101',
    predict_end='20201231',
    scale_type='amplitude'
)

# Returns:
# - X_train: (N) array of (window_size, window_size, T_train)
# - X_predict: (N) array of (window_size, window_size, T_predict)
# - timestamps_train, timestamps_predict

Scenario 3: Cross-Polarization Domain Adaptation

Same-date HH → HV domain adaptation. Tests polarization invariance.

from scenarios import scenario_3_domain_adaptation_pol

data = scenario_3_domain_adaptation_pol(
    loader=loader,
    window_size=32,
    target_date='20200804',      # Single acquisition date
    orbit='DSC'
)

# Returns:
# - X_source: (N, window_size, window_size) - HH polarization
# - X_target: (N, window_size, window_size) - HV polarization
# - y: (N,) labels

Scenario 4: Cross-Geometry Domain Adaptation

PAZ different acquisition geometries domain adaptation. Tests geometry invariance between different orbits and dates.

from scenarios import scenario_4_domain_adaptation_satellite

data = scenario_4_domain_adaptation_satellite(
    loader=loader,
    window_size=32,
    source_orbit='DSC',           # Source orbit
    target_orbit='ASC',           # Target orbit
    source_date='20210127',       # Source acquisition date
    target_date='20210214',       # Target acquisition date
    source_polarization='HH',     # Source polarization
    target_polarization='HH',     # Target polarization
    scale_type='amplitude'
)

# Returns:
# - X_source: (N_source, window_size, window_size, n_pol) - Source geometry
# - X_target: (N_target, window_size, window_size, n_pol) - Target geometry
# - y_source: (N_source,) labels for source (labeled)
# - y_target: (N_target,) labels for target (for analysis only)
# - groups_source, groups_target

Key Features

Deterministic Extraction:

• —skip_optim_offset=True allows to speed up window extraction by skipping offset optimization.
• —skip_optim_offset=False performs offset optimization to maximize the number of valid windows, useful for small datasets.

Scale Transformations:

• —Three options for data representation:
• —'intensity': Raw backscatter
• —'amplitude': Square root (√intensity)
• —'log10': Logarithmic scale (dB)

Quality Filtering: Information of distorted pixels is provided via quality masks (Obtained from ESA processing). 0: No distortion, 1: Layover, 2: Shadow, 3: Both

• —max_mask_value: Rejects pixels with mask > threshold (0-3 scale)
• —max_mask_percentage: Maximum bad pixel ratio per window with mask<=max_mask_value
• —min_valid_percentage: Minimum valid data ratio per window (non-nodata and non-NaN). To avoid No Data and NaN use 100%.

Data Loader Parameters

5. Citation

If you use this dataset, please cite:

6. License

apache 2-0

7. Contact

[contact information]

8. Acknowledgements

The authors would like to thank the Spanish Instituto Nacional de Técnica Aeroespacial (INTA) for providing the PAZ images (Project AO-001-051).