FM4CS/THOR-1.0-base
01.5k
1---2license: apache-2.03datasets:4- FM4CS/THOR-Pretrain5pipeline_tag: image-feature-extraction6library_name: terratorch7tags:8- NR9- ESA10- Foundation Model11- Earth Observation12- Geospatial13- Remote Sensing14- Sentinel-115- Sentinel-216- Sentinel-317- SAR18- Multispectral19- Climate20---21 22[](https://thor-model.notion.site/THOR-Foundation-Model-Showcase-2ee64c7f3cb78087bf77feb6350bdcc6)23[](https://arxiv.org/abs/2601.16011)24[](https://github.com/FM4CS/THOR)25[](https://github.com/FM4CS/thor_terratorch_ext)26[](https://huggingface.co/datasets/FM4CS/THOR-Pretrain)27 28# THOR Base29 30THOR (Transformer based foundation model for Heterogeneous Observation and Resolution) is a compute-adaptive geospatial foundation model developed by Norwegian Computing Center (NR), UiT The Arctic University of Norway and ESA Φ-lab.31 32## Model Description33 34THOR unifies data from Copernicus Sentinel-1, -2, and -3 (OLCI & SLSTR) satellites, processing their native 10 m to 1000 m resolutions in a single model. THOR is pre-trained with a novel randomized patch and input image size strategy, allowing deployment at inference with any patch size for dynamic trade-offs between computational cost and feature resolution without retraining.35 36Key features:37- **Multi-sensor support**: Sentinel-1 (SAR), Sentinel-2 (MSI), Sentinel-3 OLCI & SLSTR38- **Flexible resolution**: 10 m to 1000 m native resolutions39- **Compute-adaptive**: Flexible patch sizes and ground covers (1000 m to +100,000 m)40- **Data-efficient**: State-of-the-art performance in data-limited regimes41- **Model type:** Vision Transformer (FlexiViT)42 43## Usage44 45THOR is designed for fine-tuning on downstream tasks such as land cover classification, crop mapping, flood detection, and more. Its flexible architecture allows users to adapt the model to various geospatial applications while leveraging its multi-sensor capabilities.46 47For downstream applications, we recommend using the [terratorch](https://github.com/terrastackai/terratorch) framework with our [THOR terratorch extension](https://github.com/FM4CS/thor_terratorch_ext).48 49 50### Terratorch backbone loading example51 52```python53# Example usage of THOR ViT backbone with terratorch54 55# Import our custom thor_terratorch_ext module to register THOR backbones56import thor_terratorch_ext # noqa: F40157 58# Load the backbone registry59from terratorch import BACKBONE_REGISTRY60 61# List available THOR backbones62print([b for b in list(BACKBONE_REGISTRY) if "thor" in b])63 64# Build a THOR ViT model with specific bands65model = BACKBONE_REGISTRY.build(66 "thor_v1_base",67 pretrained=True,68 model_bands=["BLUE", "GREEN", "RED", "VV", "VH"],69 input_params=dict( # Optional input parameters to customize70 ground_covers=[71 288072 ], # Ground cover in meters (typically input image size [px] * input image resolution)73 flexivit_patch_size_seqs=[8], # Patch size in pixels74 ),75)76```77 78## Training Details79 80### Training Data81 82THOR is pre-trained on [THOR-Pretrain](https://huggingface.co/datasets/FM4CS/THOR-Pretrain), a large-scale multi-sensor dataset containing paired observations from Sentinel-1, Sentinel-2, and Sentinel-3 satellites, as well as auxiliary land cover and elevation data and meteorological variables.83 84 85### Training Procedure86 87For training configuration, see the config file: [thor-base.yaml](https://github.com/FM4CS/THOR/blob/33842760f061063baf9fe3f748008c84a663fa8b/thor/config/pretrain/final/thor-base.yaml)88 89 90### Compute Infrastructure91The model was trained on the LUMI supercomputer in Finland using 4 nodes, each equipped with 4 AMD MI250X GPUs, totaling 32 GCDs. 92 93## Evaluation94 95### Results96 97THOR demonstrates highly competitive performance on the PANGAEA benchmark, particularly in data-limited regimes. With only 10% training data, THOR-Base achieves the best average rank across all datasets.98 99| Model | HLS Burns | MADOS | PASTIS | Sen1Floods11 | FBP | DynEarthNet | CropMap | SN7 | AI4Farms |100|-------|-----------|-------|--------|--------------|-----|-------------|---------|-----|----------|101| CROMA | 76.44 | 32.44 | 32.80 | *87.22* | 37.39 | 36.08 | 36.77 | 42.15 | 38.48 |102| DOFA | 71.98 | 23.77 | 27.68 | 82.84 | 27.82 | **39.15** | 29.91 | 46.10 | 27.74 |103| Prithvi | 77.73 | 21.24 | 33.56 | 86.28 | 29.98 | 32.28 | 27.71 | 36.78 | 35.04 |104| SpectralGPT | **83.35** | 20.29 | 34.53 | 83.12 | 39.51 | 35.33 | 31.06 | 36.31 | 37.35 |105| Terramind-B | 77.39 | **44.06** | **39.96** | 84.43 | *54.00* | *37.35* | 35.65 | 43.21 | 38.59 |106| UNet Baseline | *79.46* | 24.30 | 29.53 | **88.55** | 52.58 | 35.59 | 13.88 | 46.08 | 34.84 |107| ViT Baseline | 75.92 | 10.18 | 38.44 | 81.85 | **56.53** | 35.39 | 27.76 | 36.01 | **39.20** |108| THOR-B | 76.90 | 40.67 | *38.93* | 86.29 | 42.80 | 35.21 | **42.23** | *55.94* | *38.90* |109| THOR-T | 75.98 | *41.65* | 36.26 | 82.70 | 42.81 | 34.03 | *37.82* | **58.52** | 38.56 |110 111*Results in mIoU on PANGAEA benchmark with 10% training data. **Bold** = best, *italic* = second-best.*112 113## Attribution114 115The development of THOR was funded and supported by European Space Agency (ESA) Φ-lab (FM4CS project, contract no. 4000143489/24/I-DT), and the Research Council of Norway (KnowEarth project no. 337481).116 117## Citation118 119If you use THOR in your research, please cite the [paper](https://arxiv.org/abs/2601.16011):120 121**BibTeX:**122 123```bibtex124@article{forgaard2026thor,125 title={THOR: A Versatile Foundation Model for Earth Observation Climate and Society Applications}, 126 author={Theodor Forgaard and Jarle H. Reksten and Anders U. Waldeland and Valerio Marsocci and Nicolas Longépé and Michael Kampffmeyer and Arnt-Børre Salberg},127 year={2026},128 eprint={2601.16011},129 archivePrefix={arXiv},130 primaryClass={eess.IV},131 url={https://arxiv.org/abs/2601.16011}, 132}133```134 135## Contact136Arnt Salberg - Norwegian Computing Center (NR) 137salberg@nr.no