license: apache-2.0
pipeline_tag: image-feature-extraction
GAIR: Location-Aware Self-Supervised Contrastive Pre-Training with Geo-Aligned Implicit Representations
Official implementation of GAIR, a location-aware self-supervised pre-training framework for learning geo-aligned representations from remote sensing imagery, street-view imagery, and geographic coordinates.
Overview
GAIR solves the representation problem of detailed localized image representations at arbitrary positions by using an implicit neural representation (INR) module extending ViT with Neural Implicit Local Interpolation (NILI). It integrates overhead remote sensing (RS) data, street view (SV) imagery, and their geolocation metadata for contrastive learning. The model produces continuous RS image representations covering arbitrary locations.
Sample Usage
To use this model, ensure you have the dependencies installed and the gair module from the official repository in your path.
import torch
from huggingface_hub import hf_hub_download
from gair import GAIRModel, load_rs_image, load_sv_image, preprocess_rs_array, preprocess_sv_array
# Download the checkpoint
checkpoint = hf_hub_download(repo_id="PingL/GAIR", filename="checkpoint.pth")
device = "cuda:0" if torch.cuda.is_available() else "cpu"
# Load the model
model = GAIRModel.from_checkpoint(checkpoint, device=device, query_mode="nili")
# Prepare inputs
rs = preprocess_rs_array(load_rs_image("path/to/rs_crop.tif")).to(device)
sv = preprocess_sv_array(load_sv_image("path/to/street_view.jpg")).to(device)
coords = torch.tensor([[116.397, 39.908]], device=device) # [lon, lat]
bbox = torch.tensor([[116.387, 39.918, 116.407, 39.898]], device=device) # bbox: [lon_min, lat_max, lon_max, lat_min]
with torch.inference_mode():
# Extract global embeddings
rs_embedding = model.encode_rs(rs, normalize=True)
sv_embedding = model.encode_sv(sv, normalize=True)
loc_embedding = model.encode_location(coords, normalize=True)
# Query localized RS embeddings using the NILI module
rs_global, localized_rs_embedding = model.query_localized_rs(
rs,
coords,
bbox,
mode="nili",
normalize=True,
)
Input Conventions
- RS crop bbox order:
[lon_min, lat_max, lon_max, lat_min] - Coordinate order:
[lon, lat] - RS inputs: Should be 10-band Sentinel-2 image (B2, B3, B4, B5, B6, B7, B8, B8A, B11, B12).
Citation
The citation entry will be added after the paper is officially published. For now, please refer to:
https://huggingface.co/papers/2503.16683