lcrucial1f/MPS-CLIP-Models

MPS-CLIP checkpoints

Checkpoints for MPS-CLIP: Multi-Perspective Subimage CLIP with Keyword Guidance for Remote Sensing Image-Text Retrieval (ICME 2026).

• Paper: arXiv:2601.18190
• Code: github.com/Lcrucial1f/MPS-CLIP
• Data and keyword-guided subimages: lcrucial1f/MPS-CLIP_Data

These weights need the code at commit f668644 or later. That code loads them with strict=True and reproduces the numbers below.

Files

File	Dataset	Image→Text R@1 / R@5 / R@10	Text→Image R@1 / R@5 / R@10	mR
MPS-CLIP_ViT-B-32_RSICD_rmean35.24.pth	RSICD	18.12 / 36.69 / 49.31	13.63 / 38.90 / 54.78	35.24
MPS-CLIP_ViT-B-32_RSITMD_rmean48.86.pth	RSITMD	28.10 / 52.65 / 63.27	22.43 / 54.82 / 71.86	48.86

Scores are on the standard test splits: 1,093 images and 5,465 captions for RSICD, and 452 images and 2,260 captions for RSITMD. Both models use keyword-guided subimages at training and test time.

Each file is a torch.save dict with two entries:

• model: the full state dict, with 755 tensors.
• config: the settings the model was trained with. The hyperparameters are the same as in configs/Retrieval_{rsicd,rsitmd}_vit.yaml of the code repository; use the data paths from those files.

The files contain only tensors and plain Python types, so they load with torch.load(..., weights_only=True).

Usage

import torch
from huggingface_hub import hf_hub_download

path = hf_hub_download("lcrucial1f/MPS-CLIP-Models", "MPS-CLIP_ViT-B-32_RSITMD_rmean48.86.pth")
checkpoint = torch.load(path, map_location="cpu", weights_only=True)

# Run from the root of the MPS-CLIP repository:
from models.model_retrieval import MPSCLIP
model = MPSCLIP(config={**checkpoint["config"], "if_evaluation": True})
model.load_state_dict(checkpoint["model"], strict=True)

To evaluate with the repository's launcher, download the files into ./checkpoints/MPS-CLIP (as in the code README) and run:

python run.py --task itr_rsitmd_vit --dist gpu0 --evaluate \
  --checkpoint ./checkpoints/MPS-CLIP/MPS-CLIP_ViT-B-32_RSITMD_rmean48.86.pth \
  --output_dir ./checkpoints/MPS-CLIP/test

The global attention in the text tower attends across the captions of a batch, so a caption's embedding depends on the other captions encoded with it. To reproduce the numbers exactly, keep batch_size_test_text: 128 and the caption order of the annotation files, as the provided configs do.

Training

• Backbone. OpenCLIP ViT-B/32 initialized from LAION-2B (laion2b_s34b_b79k) and kept frozen.
• Trained parameters. G2A adapters, the MPR head, the gates, the temperature and the loss weights: 8.24M of 159.5M parameters.
• RSICD. rsicd_train1.json. 2 GPUs × 64 images per step, 30 epochs. The released checkpoint is from epoch 11.
• RSITMD. rsitmd_train1.json. 2 GPUs × 128 images per step with gradient checkpointing, 35 epochs. The released checkpoint is from epoch 21.
• Optimizer. AdamW, lr 4e-4 for all trained parameters, betas (0.9, 0.98), weight decay 0.04 (none on biases and norms). Linear schedule with 10% warmup.
• Checkpoint selection. The test mR is computed after every epoch, and the epoch with the highest test mR is released.

Acknowledgements

The training code and the adapter design build on HarMA (Apache-2.0). The backbone weights are from OpenCLIP / LAION.

Citation

@article{li2026mpsclip,
  title={Multi-Perspective Subimage CLIP with Keyword Guidance for Remote Sensing Image-Text Retrieval},
  author={Li, Yifan and Wang, Shiying and Huang, Jianqiang},
  journal={arXiv preprint arXiv:2601.18190},
  year={2026}
}