litecalib_vitb

A released model of LiteCalib: Fast, Model-Agnostic Single-View Camera Calibration (Viktor Kocur, Lukáš Bujňák, Lukáš Gajdošech, Torsten Sattler, Zuzana Kukelova).

LiteCalib with a DINOv2 ViT-B/14 backbone: the camera's field of view at a 5 x 5 grid of image points, to which any radially symmetric camera model is fitted. 106 M parameters.

Code, documentation and training: github.com/kocurvik/litecalib.

pip install litecalib
import litecalib

model = litecalib.load("litecalib_vitb")
calib = model.calibrate("photo.jpg", "radial:1")
print(calib.K, calib.distortion)

Accuracy and runtime

Averaged over the ten benchmarks of the paper (Stanford2D3D, TartanAir, MegaDepth, LaMAR, MegaDepth-radial, ScanNet++, Mono, Cathedral, InFlux and InFlux++), each image fitted with its benchmark's camera model:

median angular error angular error mAA@5° median vFoV error vFoV error mAA@5°
1.50° 66.6 % 3.20° 43.7 %

The angular error is AnyCalib's: the angle between the predicted and the true ray, over the pixels of the image; mAA@5° is the area under its recall curve up to 5°.

Runtime per image, batch 1 (network + camera fit):

device network fit total
NVIDIA A100, PyTorch fp16 6.3 ms 0.3 ms (CPU) 6.6 ms
Jetson Orin Nano 8 GB (15 W), TensorRT fp16 22.9 ms 4.2 ms (CPU) 27.1 ms

The fit is the mean over the camera models; the Jetson's network is a TensorRT engine of the exported model, which this package does not build.

Training data

Trained from DINOv2's weights for 24 epochs (warmup-stable-decay schedule), each epoch 144,000 views: 72,000 drawn from the panorama views and 72,000 from MegaScenes. All the released models were trained on the same data.

source images views (train / val) cameras licences
OpenPano, AnyCalib's extended version 3,651 / 202 panoramas 51,719 / 2,872 pinhole, Brown-Conrady (radial:1), EUCM, about a third each per source (below)
CommonsPano (by and by-sa subsets) 9,336 / 555 panoramas 148,626 / 8,851 as OpenPano public domain, CC0, CC BY (by); CC BY-SA (by-sa)
MegaScenes 14,382 / 160 scenes 1,211,520 / 10,779 radial:1 from COLMAP's SIMPLE_RADIAL per photo, from Wikimedia Commons (74 % CC BY-SA, 17 % CC BY, 8 % public domain / CC0)
  • OpenPano: introduced by GeoCalib, which describes how its panoramas were collected and split (GeoCalib: OpenPano Dataset); extended by AnyCalib with panoramas that need not be aligned with gravity (AnyCalib: Extended OpenPano Dataset). It consists of tonemapped panoramas from The Laval Photometric Indoor HDR Dataset, PolyHaven, HDRMaps, AmbientCG and BlenderKit. AnyCalib's splits.
  • CommonsPano (LiteCalib/CommonsPano): 360° panoramas from Wikimedia Commons, split by uploader, with a per-file attribution list. 465 duplicates were removed: 444 of OpenPano panoramas (found by image content and by source name) and 21 within Commons. Three training panoramas yield no valid view, so the views come from 9,333 of them.
  • Panorama views: 640 x 640 perspective and distorted views rendered from each panorama with AnyCalib's camera sampling (its OP_g recipe), principal point at the center, square pixels.
  • MegaScenes (megascenes.github.io): Wikimedia Commons photos with COLMAP reconstructions. Each photo is center-cropped to 1:1, 4:3 or 3:2 at a short side of 600 px, its principal point at the center. Left out: reconstructions with fewer than 20 photos, failed self-calibrations, and the scenes of the MegaDepth evaluation landmarks.
  • Training: AnyCalib's resolution (about 102,400 pixels, aspect ratio 0.5 to 2) and its photometric augmentations.

More in the repository.

License

Apache License 2.0. We thank the authors of The Laval Photometric Indoor HDR Dataset for allowing us to release these weights under this license.

Downloads last month
29
Safetensors
Model size
0.1B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support