Pre-trained cross-modal SAR-optical encoders: DeCUR, CROMA, CroCoMAE (v1)
Weights of the three cross-modal self-supervised frameworks evaluated in the paper (citation below), pre-trained under the same protocol and used zero-shot for change detection between a pre-event Sentinel-2 image and a post-event Sentinel-1 image. Code: https://github.com/tanodino/sar-optical-ssl-zero-shot-hcd; benchmark: https://doi.org/10.5281/zenodo.24166214.
Contents
decur/ croma/ crocomae/
model.pth {"model": state dict of the whole framework (both encoders, heads, decoders), "step": 148000}
model_info.json framework, architecture, input channels
config.json pre-training configuration
norm_stats.json input normalisation (per-band clipping limits, mean, std of the pre-training corpus)
LICENSE, SHA256SUMS
| Framework | Parameters (whole module) | Aligned vector used for change detection | Encoder-only vector |
|---|---|---|---|
| DeCUR | 317.6 M | projector output, common part (first 7,168 of 8,192 dims), of the CLS token | CLS token |
| CROMA | 60.6 M | contrastive head output (FFN + projection) of the averaged patch tokens | average of the patch tokens |
| CroCoMAE | 44.3 M | average of the patch tokens, shared subspace (first 192 of 384 dims) | (same as aligned) |
The change detection pipeline needs the encoders plus the DeCUR projectors / CROMA head (42.8 M, 45.4 M and 317.6 M parameters for CroCoMAE, CROMA and DeCUR); the decoders and the CROMA fusion encoder are only used during pre-training.
Training
- Encoders: one ViT-Small per modality (384 dims, 12 layers, 6 heads), 8 x 8 patches on 64 x 64 inputs.
SAR input: Sentinel-1 GRD VV, VH in dB; optical input: Sentinel-2 L1C B2, B3, B4, B8. Both clipped and standardised with
norm_stats.json. - Data: 1,609,824 tiles of 64 x 64 pixels from 100,614 co-temporal Sentinel-1 / Sentinel-2 pairs of a subset of SSL4EO-S12 v1.1 (December 2019 to October 2021).
- Protocol (identical for the three frameworks): batch 512, AdamW (betas 0.9 / 0.95, weight decay 0.05), peak learning rate 1.5e-4, linear warm-up over 5 % of a 150,000-step schedule then cosine decay to 1e-6, gradient clipping 1.0, mixed precision, seed 0, one GPU. The released weights are those of step 148,000 (the checkpoint used in the paper). Each framework keeps its own losses, projectors, temperatures and masking (see the code).
Usage
With the code repository (strict loading, the framework rebuilt exactly as in pre-training):
import json
from model_loading import load_model
info = json.load(open("croma/model_info.json"))
model, step = load_model("croma", info) # step == 148000
extract_features.py --run <this folder>/croma ... computes the zero-shot change maps. CROMA and CroCoMAE were pre-trained
with 16 visible tokens out of 64 and are applied in the same regime (see the code).
Licence and attribution
CC BY 4.0 (LICENSE). Trained on SSL4EO-S12 v1.1 (CC BY 4.0; Blumenstiel et al., arXiv:2503.00168, and Wang et al.,
SSL4EO-S12), which contains modified Copernicus Sentinel data. Frameworks: DeCUR (Wang et al., ECCV 2024), CROMA (Fuller
et al., NeurIPS 2023), CroCoMAE (Recchia et al., arXiv:2611.14124).
How to cite
TODO: paper reference.