GeoCLIP Location Encoder

This is the location encoder from GeoCLIP, available as a standalone model. It takes latitude and longitude in degrees and returns a 512-dimensional vector you can use as geographic features in your own models. It runs on a CPU and does not need the image backbone.

Demo · Paper · GitHub · Python package

Download weights · Browse all model files

Get started

pip install geoclip

import torch
from geoclip import LocationEncoder

encoder = LocationEncoder().eval()
coordinates = torch.tensor([
    [40.7128, -74.0060],  # New York City
    [34.0522, -118.2437],  # Los Angeles
], dtype=torch.float32)
with torch.inference_mode():
    embeddings = encoder(coordinates)
print(embeddings.shape)  # torch.Size([2, 512])

Pass coordinates as an (N, 2) tensor in [latitude, longitude] order, in degrees. You get one 512-dimensional vector per location: an (N, 512) tensor. These vectors are unnormalized; L2-normalize them before computing cosine similarity.

The location embeddings share a space with GeoCLIP's projected image features. To predict a photo's location, use the full GeoCLIP model.

Load from Hugging Face

The from_pretrained helpers are available in geoclip 1.2.2 and newer.

python -m pip install --upgrade "geoclip>=1.2.2"

import torch
from geoclip import LocationEncoder

encoder = LocationEncoder.from_pretrained("VicenteVivanco/geoclip-location-encoder")
coordinates = torch.tensor([[40.7128, -74.0060]], dtype=torch.float32)
with torch.inference_mode():
    embeddings = encoder(coordinates)
print(embeddings.shape)  # torch.Size([1, 512])

This loads just the location encoder, with no image-backbone download. Downloads are cached. You can pass revision to select a Hub commit or a local release folder in place of the model ID.

Manual Hub loading with geoclip 1.2.1

The quick start uses the weights included in the Python package. To load the weights from this page instead, use the example below.

pip install geoclip huggingface_hub safetensors

import json
import torch
from geoclip import LocationEncoder
from huggingface_hub import snapshot_download
from safetensors.torch import load_file

folder = snapshot_download("VicenteVivanco/geoclip-location-encoder")
config = json.load(open(f"{folder}/config.json"))
encoder = LocationEncoder(sigma=config["sigma"], from_pretrained=False)
encoder.load_state_dict(load_file(f"{folder}/model.safetensors"))
encoder.eval()
coordinates = torch.tensor([[40.7128, -74.0060]], dtype=torch.float32)
with torch.inference_mode():
    embeddings = encoder(coordinates)
print(embeddings.shape)  # torch.Size([1, 512])

Using GPS features in your own models

We trained the location encoder as part of GeoCLIP on roughly 4.7 million geotagged photos from MP-16. It learns geographic features by matching coordinates to photographs in a shared embedding space.

In the paper, we combined these embeddings with image features to improve classification on the Geo-Tagged NUS-Wide dataset, as shown above. You can use the encoder in a similar way when your model has coordinates available and could benefit from geographic context.

The encoder uses an Equal Earth projection, random Fourier features at three scales, and trained neural layers. Its usefulness depends on your task and on the geographic coverage of the training photos. Cosine similarity compares the learned embeddings; it does not measure distance between places.

What's included in the download

model.safetensors: the complete location encoder, including the frozen Fourier matrices and trained weights.
config.json: the Fourier scales and coordinate conventions.
provenance.json: the source commit, package version, and checkpoint hashes.

The exported tensors match the original location checkpoint exactly. The source checkpoint matches the one included in geoclip==1.2.1.

Citation and license

If GeoCLIP is useful in your work, please cite the paper:

@inproceedings{geoclip,
  title={GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization},
  author={Vivanco Cepeda, Vicente and Nayak, Gaurav Kumar and Shah, Mubarak},
  booktitle={Advances in Neural Information Processing Systems},
  year={2023}
}

Released under the MIT license. The location encoder includes code from Joshua M. Long's Random Fourier Features PyTorch.