Add model card, metadata and sample usage

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +64 -3
README.md CHANGED
@@ -1,3 +1,64 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ pipeline_tag: image-feature-extraction
4
+ ---
5
+
6
+ # GAIR: Location-Aware Self-Supervised Contrastive Pre-Training with Geo-Aligned Implicit Representations
7
+
8
+ Official implementation of GAIR, a location-aware self-supervised pre-training framework for learning geo-aligned representations from remote sensing imagery, street-view imagery, and geographic coordinates.
9
+
10
+ [**Paper**](https://huggingface.co/papers/2503.16683) | [**GitHub**](https://github.com/zpl99/GAIR)
11
+
12
+ ## Overview
13
+ GAIR solves the representation problem of detailed localized image representations at arbitrary positions by using an implicit neural representation (INR) module extending ViT with Neural Implicit Local Interpolation (NILI). It integrates overhead remote sensing (RS) data, street view (SV) imagery, and their geolocation metadata for contrastive learning. The model produces continuous RS image representations covering arbitrary locations.
14
+
15
+ ## Sample Usage
16
+
17
+ To use this model, ensure you have the dependencies installed and the `gair` module from the official repository in your path.
18
+
19
+ ```python
20
+ import torch
21
+ from huggingface_hub import hf_hub_download
22
+ from gair import GAIRModel, load_rs_image, load_sv_image, preprocess_rs_array, preprocess_sv_array
23
+
24
+ # Download the checkpoint
25
+ checkpoint = hf_hub_download(repo_id="PingL/GAIR", filename="checkpoint.pth")
26
+ device = "cuda:0" if torch.cuda.is_available() else "cpu"
27
+
28
+ # Load the model
29
+ model = GAIRModel.from_checkpoint(checkpoint, device=device, query_mode="nili")
30
+
31
+ # Prepare inputs
32
+ rs = preprocess_rs_array(load_rs_image("path/to/rs_crop.tif")).to(device)
33
+ sv = preprocess_sv_array(load_sv_image("path/to/street_view.jpg")).to(device)
34
+ coords = torch.tensor([[116.397, 39.908]], device=device) # [lon, lat]
35
+ bbox = torch.tensor([[116.387, 39.918, 116.407, 39.898]], device=device) # bbox: [lon_min, lat_max, lon_max, lat_min]
36
+
37
+ with torch.inference_mode():
38
+ # Extract global embeddings
39
+ rs_embedding = model.encode_rs(rs, normalize=True)
40
+ sv_embedding = model.encode_sv(sv, normalize=True)
41
+ loc_embedding = model.encode_location(coords, normalize=True)
42
+
43
+ # Query localized RS embeddings using the NILI module
44
+ rs_global, localized_rs_embedding = model.query_localized_rs(
45
+ rs,
46
+ coords,
47
+ bbox,
48
+ mode="nili",
49
+ normalize=True,
50
+ )
51
+ ```
52
+
53
+ ### Input Conventions
54
+ - **RS crop bbox order**: `[lon_min, lat_max, lon_max, lat_min]`
55
+ - **Coordinate order**: `[lon, lat]`
56
+ - **RS inputs**: Should be 10-band Sentinel-2 image (B2, B3, B4, B5, B6, B7, B8, B8A, B11, B12).
57
+
58
+ ## Citation
59
+
60
+ The citation entry will be added after the paper is officially published. For now, please refer to:
61
+
62
+ ```text
63
+ https://huggingface.co/papers/2503.16683
64
+ ```