Modelpublic

BiliSakura/SOFTCON-transformers

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes

Model Card

SoftCon Transformers Models

Hugging Face–compatible checkpoints converted from the official SoftCon pretrained weights. Each subfolder is a standalone model repo layout (config.json, model.safetensors, preprocessor, and remote code) for feature extraction on Earth observation imagery.

Model Description

These models are encoders pretrained with multi-label guided soft contrastive learning on SSL4EO-S12 multispectral and SAR imagery, with DINOv2-style continual pretraining for ViT backbones.

This collection bundles 6 converted checkpoints:

—Architectures: ResNet-50, ViT-S/14, ViT-B/14 (DINOv2-style)
—Input modalities: S2-L1C 13-band (s2c), S1 SAR 2-band (s1)

All folders ship self-contained remote code (modeling_softcon.py, processor, pipeline) and load with trust_remote_code=True. ViT backbones reuse transformers' built-in `Dinov2Model`.

Developed by: zhu-xlab / SoftCon Converted for Hugging Face by: BiliSakura License (weights): Apache-2.0 Original paper: Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretraining

Available checkpoints (6 models)

Folder	Backbone	Modality	Channels	Embedding dim	Legacy file
`softcon-resnet50-s2c`	RN50	S2-L1C MS	13	2048	`B13_rn50_softcon.pth`
`softcon-vit-small-patch14-s2c`	ViT-S/14	S2-L1C MS	13	384	`B13_vits14_softcon_enc.pth`
`softcon-vit-base-patch14-s2c`	ViT-B/14	S2-L1C MS	13	768	`B13_vitb14_softcon_enc.pth`
`softcon-resnet50-s1`	RN50	S1 SAR	2	2048	`B2_rn50_softcon.pth`
`softcon-vit-small-patch14-s1`	ViT-S/14	S1 SAR	2	384	`B2_vits14_softcon_enc.pth`
`softcon-vit-base-patch14-s1`	ViT-B/14	S1 SAR	2	768	`B2_vitb14_softcon_enc.pth`

Usage

Processors default to `do_resize: false`. Pass patches at native (H, W, C); the processor rescales to [0, 1] without changing spatial size.

python

from transformers import pipeline
import numpy as np

REPO = "BiliSakura/SOFTCON-transformers"
SUBFOLDER = "softcon-vit-small-patch14-s2c"

pipe = pipeline(
    task="softcon-feature-extraction",
    model=REPO,
    trust_remote_code=True,
    model_kwargs={"subfolder": SUBFOLDER},
)

# S2-L1C: 13 bands at native resolution (e.g. 512×512)
image = np.random.randint(0, 255, (512, 512, 13), dtype=np.uint8)
features = pipe(image, pool=True, return_tensors=True)
print(features.shape)  # [1, 384]

Dense token features:

python

tokens = pipe(image, pool=False, return_tensors=True)

Opt in to resize (patch-14 models were pretrained on 224×224):

python

features = pipe(image, pool=True, return_tensors=True, image_processor_kwargs={"do_resize": True})

Load components directly:

python

from transformers import AutoModel, AutoImageProcessor

model = AutoModel.from_pretrained(REPO, subfolder=SUBFOLDER, trust_remote_code=True)
processor = AutoImageProcessor.from_pretrained(REPO, subfolder=SUBFOLDER, trust_remote_code=True)

Normalization

By default, the bundled image processor rescales inputs to [0, 1] by dividing by 255. SoftCon recommends mapping each channel to uint8 using per-channel mean/std from SSL4EO-S12 or the target dataset before inference. Enable do_normalize=True with per-channel image_mean and image_std on SoftConImageProcessor when using normalized inputs.

Dependencies

—transformers, torch, torchvision, safetensors
—opencv-python (multispectral resize with more than 4 channels)

Citation

bibtex

@misc{wang2024multilabel,
  title={Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretraining},
  author={Wang, Yi and Albrecht, Conrad M and Zhu, Xiao Xiang},
  journal={arXiv preprint arXiv:2405.20462},
  year={2024}
}