README.md

metadata

license: other
license_name: dinov3-license
license_link: https://ai.meta.com/resources/models-and-libraries/dinov3-license
base_model: facebook/dinov3-vitl16-pretrain-lvd1689m
base_model_relation: finetune
library_name: pytorch
tags:
  - geolocation
  - image-regression
  - flow-matching
  - dinov3
  - street-view
  - jakarta
  - computer-vision
language:
  - id
datasets:
  - nadh0708/JKTSV-primary

Img2LocJakarta 2.0 — DINOv3 + flow matching street-view geolocation

Predict a distribution of where a street-view image was taken in Jakarta. A fine-tuned DINOv3 ViT-L/16 backbone encodes the image, then a conditional flow-matching (CFM) head propagates Gaussian-noise particles onto the geographic coordinates.

Input	RGB street-level image (any size, resized to 256×256)
Output	num_particles predicted coordinates (100 particles → 100 lat/lon points), as a list or GeoJSON
Scope	Jakarta street imagery only [primary road]
Backbone	DINOv3 ViT-L/16, fine-tuned end to end
Head	Velocity MLP (4 AdaLN residual blocks, hidden 512), Heun sampler

Quick Start

pip install torch torchvision pillow numpy huggingface_hub safetensors

git is needed on first run: the DINOv3 architecture source is cloned from facebookresearch/dinov3 into the torch hub cache (no extra weights are downloaded).

from huggingface_hub import hf_hub_download

# Download the inference code and the example image
for fname in ["inference.py", "modeling_geotag.py", "examples/kokas.png"]:
    hf_hub_download("nadh0708/Img2LocJakarta2.0", fname, local_dir=".")

from inference import GeoTagPredictor

predictor = GeoTagPredictor("nadh0708/Img2LocJakarta2.0")
coords = predictor.predict("examples/kokas.png", num_particles=100)
print(len(coords), coords[0])
# 100 {'lat': -6.2257, 'lon': 106.8309}      <- 100 predicted coordinates

# Batched: one list of coordinates per image
clouds = predictor.predict(["a.jpg", "b.jpg"], num_particles=100)

# Optional: mean point + uncertainty of one cloud
print(predictor.summarize(coords))
# {'lat': -6.2246, 'lon': 106.8335, 'n_particles': 100, 'sigma_m': 484.3,
#  'p50_radius_m': 372.1, 'p90_radius_m': 734.0}

# GeoJSON, one Point feature per coordinate
geojson = predictor.to_geojson(["examples/kokas.png"], num_particles=100)

The printed values are from a CUDA GPU with the default seed=0. On CPU the random draws differ, so the cloud is slightly different (mean -6.2247, 106.8339, sigma_m 496.2).

From a local folder

predictor = GeoTagPredictor(".")          # folder containing model.safetensors

CLI (GeoJSON output)

python inference.py nadh0708/Img2LocJakarta2.0 examples/kokas.png --particles 100 -o out.geojson
python inference.py nadh0708/Img2LocJakarta2.0 a.jpg b.jpg --particles 512 --seed 1
python inference.py nadh0708/Img2LocJakarta2.0 a.jpg --truth 106.83 -6.22   # adds error_m
python inference.py nadh0708/Img2LocJakarta2.0 a.jpg --table                # text summary

--particles N gives N coordinates per image, so N Point features. Open out.geojson in QGIS or geojson.io to see them on a map.

Low-level API

import torch
from PIL import Image
from torchvision.transforms import v2
from modeling_geotag import DinoV3CFMRegressor

model = DinoV3CFMRegressor.from_pretrained("nadh0708/Img2LocJakarta2.0")

tf = v2.Compose([
    v2.ToImage(),
    v2.Resize((256, 256), antialias=True),
    v2.ToDtype(torch.float32, scale=True),
    v2.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
])
x = tf(Image.open("examples/kokas.png").convert("RGB")).unsqueeze(0)
cloud = model.sample_lonlat(x, num_particles=256, steps=8)   # (1, 256, 2) [lon, lat]

Output

predict(image, num_particles=N) returns N predicted coordinates (the predictive distribution) for the image, each {"lat": ..., "lon": ...} in WGS-84 degrees.

GeoJSON (to_geojson / CLI) is a FeatureCollection with one Point per coordinate (properties: image, particle, plus error_m if a true location is given) and a top-level summary list with one entry per image:

summary field	Meaning
lat, lon	mean of the coordinates
n_particles	number of coordinates
sigma_m	RMS spread around the mean, metres
p50_radius_m, p90_radius_m	radius around the mean containing 50 % / 90 % of the coordinates
error_m	distance of the mean to --truth, when given

num_particles is the number of samples drawn from the predicted coordinate distribution; more particles give a smoother estimate. seed makes the cloud reproducible.

Performance

Evaluated on the full test split of JKTSV-primary: 137,173 images (8 headings per location, 0°–315° in 45° steps), with this checkpoint (stage D, epoch 80). Setup: 32 particles, Heun integrator. Error is the great-circle (haversine) distance between the true location and the mean of the particle cloud.

Metric	Value
Mean error	1.05 km
Median error	0.40 km
75th percentile	0.82 km
90th percentile	2.34 km
RMSE	2.36 km

Within	100 m	250 m	500 m	1 km	2 km	5 km
% of test images	6.9	30.4	59.3	79.4	88.7	95.2

How the error shrinks while the flow runs

Mean particle error over the test set:

Flow time t	0.0	0.2	0.4	0.5	0.6	0.8	1.0
Mean error (km)	11.7	8.9	3.9	2.4	1.7	1.4	1.4

Note

• Individual particles are noisier than their mean, so the cloud mean is the better single point estimate.
• Numbers were measured with 32 particles; the default in predict is 256.

Architecture

DinoV3CFMRegressor
 ├─ encoder_net    : DINOv3 ViT-L/16 (fine-tuned), CLS token -> (B, 1024)
 ├─ velocity_head  : v(y_t, t, cond) -> R^2
 │                    [cond ‖ y_t] -> 512 -> 4 × AdaLN residual block (time-conditioned) -> 2
 └─ target_mean/var: standardiser of the local (x, y) metre targets (stored in the weights)

sampling: y_0 ~ N(0, I)  --Heun, 8 steps-->  y_1  --de-standardise-->  (x, y) metres
          --inverse flat-earth projection-->  (lon, lat)

The meter offsets are relative to the Jakarta origin (lon 106.828320, lat -6.227468). The checkpoint bundles the fine-tuned backbone and the head, so no separate LVD-1689M weight download is required.

Training

Final checkpoint: stage D (full-backbone fine-tuning), epoch 80. Best validation RMSE logged during training: 2.32 km. Trained on nadh0708/JKTSV-primary (Google Street View along Jakarta motorway and primary roads). The backbone starts from Meta's DINOv3 ViT-L/16 (LVD-1689M).

Limitations

• Geographically restricted to Jakarta.
• Uncertainty is not calibrated against a likelihood.
• Trained on street-view cutouts; other camera settings may degrade accuracy.

Files

File	Purpose
modeling_geotag.py	Model classes, Heun sampler, from_pretrained loader, coordinate conversion
inference.py	GeoTagPredictor high-level API + GeoJSON CLI
model.safetensors	Full state dict (fine-tuned backbone + head + standardiser, ~1.2 GB)
requirements.txt	Runtime dependencies
examples/kokas.png	Example street-view image used in the Quick Start
LICENSE	Apache License 2.0 (the code)
LICENSE-DINOv3.md	DINOv3 License (the weights)

Acknowledgment

Thanks to Meta for this onderful model & Claude for helping me write the inference code for this repo.

Citation

The backbone is DINOv3:

@misc{simeoni2025dinov3,
  title={{DINOv3}},
  author={Sim{\'e}oni, Oriane and Vo, Huy V. and Seitzer, Maximilian and Baldassarre, Federico and Oquab, Maxime and Jose, Cijo and Khalidov, Vasil and Szafraniec, Marc and Yi, Seungeun and Ramamonjisoa, Micha{\"e}l and Massa, Francisco and Haziza, Daniel and Wehrstedt, Luca and Wang, Jianyuan and Darcet, Timoth{\'e}e and Moutakanni, Th{\'e}o and Sentana, Leonel and Roberts, Claire and Vedaldi, Andrea and Tolan, Jamie and Brandt, John and Couprie, Camille and Mairal, Julien and J{\'e}gou, Herv{\'e} and Labatut, Patrick and Bojanowski, Piotr},
  year={2025},
  eprint={2508.10104},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2508.10104},
}

License

• Code (inference.py, modeling_geotag.py): Apache License 2.0, see LICENSE.
• Weights (model.safetensors): the backbone is fine-tuned from Meta's DINOv3, so the weights are a derivative of the DINO Materials and are distributed under the DINOv3 License, see LICENSE-DINOv3.md.
• The DINOv3 architecture source is fetched from facebookresearch/dinov3 at runtime and remains under its own license.