---
license: apache-2.0
pipeline_tag: image-feature-extraction
---

# GAIR: Location-Aware Self-Supervised Contrastive Pre-Training with Geo-Aligned Implicit Representations

Official implementation of GAIR, a location-aware self-supervised pre-training framework for learning geo-aligned representations from remote sensing imagery, street-view imagery, and geographic coordinates.

[**Paper**](https://huggingface.co/papers/2503.16683) | [**GitHub**](https://github.com/zpl99/GAIR)

## Overview
GAIR solves the representation problem of detailed localized image representations at arbitrary positions by using an implicit neural representation (INR) module extending ViT with Neural Implicit Local Interpolation (NILI). It integrates overhead remote sensing (RS) data, street view (SV) imagery, and their geolocation metadata for contrastive learning. The model produces continuous RS image representations covering arbitrary locations.

## Sample Usage

To use this model, ensure you have the dependencies installed and the `gair` module from the official repository in your path.

```python
import torch
from huggingface_hub import hf_hub_download
from gair import GAIRModel, load_rs_image, load_sv_image, preprocess_rs_array, preprocess_sv_array

# Download the checkpoint
checkpoint = hf_hub_download(repo_id="PingL/GAIR", filename="checkpoint.pth")
device = "cuda:0" if torch.cuda.is_available() else "cpu"

# Load the model
model = GAIRModel.from_checkpoint(checkpoint, device=device, query_mode="nili")

# Prepare inputs
rs = preprocess_rs_array(load_rs_image("path/to/rs_crop.tif")).to(device)
sv = preprocess_sv_array(load_sv_image("path/to/street_view.jpg")).to(device)
coords = torch.tensor([[116.397, 39.908]], device=device)  # [lon, lat]
bbox = torch.tensor([[116.387, 39.918, 116.407, 39.898]], device=device)  # bbox: [lon_min, lat_max, lon_max, lat_min]

with torch.inference_mode():
    # Extract global embeddings
    rs_embedding = model.encode_rs(rs, normalize=True)
    sv_embedding = model.encode_sv(sv, normalize=True)
    loc_embedding = model.encode_location(coords, normalize=True)
    
    # Query localized RS embeddings using the NILI module
    rs_global, localized_rs_embedding = model.query_localized_rs(
        rs,
        coords,
        bbox,
        mode="nili",
        normalize=True,
    )
```

### Input Conventions
- **RS crop bbox order**: `[lon_min, lat_max, lon_max, lat_min]`
- **Coordinate order**: `[lon, lat]`
- **RS inputs**: Should be 10-band Sentinel-2 image (B2, B3, B4, B5, B6, B7, B8, B8A, B11, B12).

## Citation

The citation entry will be added after the paper is officially published. For now, please refer to:

```text
https://huggingface.co/papers/2503.16683
```