litecalib_vitl
A released model of LiteCalib: Fast, Model-Agnostic Single-View Camera Calibration (Viktor Kocur, Lukáš Bujňák, Lukáš Gajdošech, Torsten Sattler, Zuzana Kukelova).
LiteCalib with a DINOv2 ViT-L/14 backbone: the camera's field of view at a 5 x 5 grid of image points, to which any radially symmetric camera model is fitted. 338 M parameters.
Code, documentation and training: github.com/kocurvik/litecalib.
pip install litecalib
import litecalib
model = litecalib.load("litecalib_vitl")
calib = model.calibrate("photo.jpg", "radial:1")
print(calib.K, calib.distortion)
Accuracy and runtime
Averaged over the ten benchmarks of the paper (Stanford2D3D, TartanAir, MegaDepth, LaMAR, MegaDepth-radial, ScanNet++, Mono, Cathedral, InFlux and InFlux++), each image fitted with its benchmark's camera model:
| median angular error | angular error mAA@5° | median vFoV error | vFoV error mAA@5° |
|---|---|---|---|
| 1.15° | 74.0 % | 2.40° | 52.3 % |
The angular error is AnyCalib's: the angle between the predicted and the true ray, over the pixels of the image; mAA@5° is the area under its recall curve up to 5°.
Runtime per image, batch 1 (network + camera fit):
| device | network | fit | total |
|---|---|---|---|
| NVIDIA A100, PyTorch fp16 | 10.1 ms | 0.3 ms (CPU) | 10.5 ms |
| Jetson Orin Nano 8 GB (15 W), TensorRT fp16 | 69.8 ms | 9.5 ms (CPU) | 79.3 ms |
The fit is the mean over the camera models; the Jetson's network is a TensorRT engine of the exported model, which this package does not build.
Training data
Trained from DINOv2's weights for 24 epochs (warmup-stable-decay schedule), each epoch 144,000 views: 72,000 drawn from the panorama views and 72,000 from MegaScenes. All the released models were trained on the same data.
| source | images | views (train / val) | cameras | licences |
|---|---|---|---|---|
| OpenPano, AnyCalib's extended version | 3,651 / 202 panoramas | 51,719 / 2,872 | pinhole, Brown-Conrady (radial:1), EUCM, about a third each |
per source (below) |
CommonsPano (by and by-sa subsets) |
9,336 / 555 panoramas | 148,626 / 8,851 | as OpenPano | public domain, CC0, CC BY (by); CC BY-SA (by-sa) |
| MegaScenes | 14,382 / 160 scenes | 1,211,520 / 10,779 | radial:1 from COLMAP's SIMPLE_RADIAL |
per photo, from Wikimedia Commons (74 % CC BY-SA, 17 % CC BY, 8 % public domain / CC0) |
- OpenPano: introduced by GeoCalib, which describes how its panoramas were collected and split (GeoCalib: OpenPano Dataset); extended by AnyCalib with panoramas that need not be aligned with gravity (AnyCalib: Extended OpenPano Dataset). It consists of tonemapped panoramas from The Laval Photometric Indoor HDR Dataset, PolyHaven, HDRMaps, AmbientCG and BlenderKit. AnyCalib's splits.
- CommonsPano (LiteCalib/CommonsPano): 360° panoramas from Wikimedia Commons, split by uploader, with a per-file attribution list. 465 duplicates were removed: 444 of OpenPano panoramas (found by image content and by source name) and 21 within Commons. Three training panoramas yield no valid view, so the views come from 9,333 of them.
- Panorama views: 640 x 640 perspective and distorted views rendered from each panorama with AnyCalib's camera
sampling (its
OP_grecipe), principal point at the center, square pixels. - MegaScenes (megascenes.github.io): Wikimedia Commons photos with COLMAP reconstructions. Each photo is center-cropped to 1:1, 4:3 or 3:2 at a short side of 600 px, its principal point at the center. Left out: reconstructions with fewer than 20 photos, failed self-calibrations, and the scenes of the MegaDepth evaluation landmarks.
- Training: AnyCalib's resolution (about 102,400 pixels, aspect ratio 0.5 to 2) and its photometric augmentations.
More in the repository.
License
Apache License 2.0. We thank the authors of The Laval Photometric Indoor HDR Dataset for allowing us to release these weights under this license.
- Downloads last month
- 29