|
Download README.md from LiteCalib/litecalib-vitb: direct link, hf CLI and curl.
- Browser
- Download file 4.76 kB
-
https://huggingface.co/LiteCalib/litecalib-vitb/resolve/main/README.md
- Command line
-
hf download hf://LiteCalib/litecalib-vitb/README.md
-
curl -L -o README.md https://huggingface.co/LiteCalib/litecalib-vitb/resolve/main/README.md
4.76 kB
| license: apache-2.0 | |
| library_name: litecalib | |
| tags: | |
| - camera-calibration | |
| - single-view-calibration | |
| - computer-vision | |
| # litecalib_vitb | |
| A released model of **LiteCalib: Fast, Model-Agnostic Single-View Camera Calibration** (Viktor Kocur, Lukáš Bujňák, Lukáš Gajdošech, Torsten Sattler, Zuzana Kukelova). | |
| LiteCalib with a DINOv2 ViT-B/14 backbone: the camera's field of view at a 5 x 5 grid of image points, to which any radially symmetric camera model is fitted. 106 M parameters. | |
| Code, documentation and training: [github.com/kocurvik/litecalib](https://github.com/kocurvik/litecalib). | |
| ```bash | |
| pip install litecalib | |
| ``` | |
| ```python | |
| import litecalib | |
| model = litecalib.load("litecalib_vitb") | |
| calib = model.calibrate("photo.jpg", "radial:1") | |
| print(calib.K, calib.distortion) | |
| ``` | |
| ## Accuracy and runtime | |
| Averaged over the ten benchmarks of the paper (Stanford2D3D, TartanAir, MegaDepth, LaMAR, MegaDepth-radial, ScanNet++, Mono, Cathedral, InFlux and InFlux++), each image fitted with its benchmark's camera model: | |
| | median angular error | angular error mAA@5° | median vFoV error | vFoV error mAA@5° | | |
| |---|---|---|---| | |
| | 1.50° | 66.6 % | 3.20° | 43.7 % | | |
| The angular error is AnyCalib's: the angle between the predicted and the true ray, over the pixels of the image; | |
| mAA@5° is the area under its recall curve up to 5°. | |
| Runtime per image, batch 1 (network + camera fit): | |
| | device | network | fit | total | | |
| |---|---|---|---| | |
| | NVIDIA A100, PyTorch fp16 | 6.3 ms | 0.3 ms (CPU) | 6.6 ms | | |
| | Jetson Orin Nano 8 GB (15 W), TensorRT fp16 | 22.9 ms | 4.2 ms (CPU) | 27.1 ms | | |
| The fit is the mean over the camera models; the Jetson's network is a TensorRT engine of the exported model, | |
| which this package does not build. | |
| ## Training data | |
| Trained from DINOv2's weights for 24 epochs (warmup-stable-decay schedule), each epoch 144,000 views: 72,000 drawn | |
| from the panorama views and 72,000 from MegaScenes. All the released models were trained on the same data. | |
| | source | images | views (train / val) | cameras | licences | | |
| |---|---|---|---|---| | |
| | OpenPano, AnyCalib's extended version | 3,651 / 202 panoramas | 51,719 / 2,872 | pinhole, Brown-Conrady (`radial:1`), EUCM, about a third each | per source (below) | | |
| | CommonsPano (`by` and `by-sa` subsets) | 9,336 / 555 panoramas | 148,626 / 8,851 | as OpenPano | public domain, CC0, CC BY (`by`); CC BY-SA (`by-sa`) | | |
| | MegaScenes | 14,382 / 160 scenes | 1,211,520 / 10,779 | `radial:1` from COLMAP's `SIMPLE_RADIAL` | per photo, from Wikimedia Commons (74 % CC BY-SA, 17 % CC BY, 8 % public domain / CC0) | | |
| - **OpenPano**: introduced by GeoCalib, which describes how its panoramas were collected and split | |
| ([GeoCalib: OpenPano Dataset](https://github.com/cvg/GeoCalib?tab=readme-ov-file#openpano-dataset)); extended by | |
| AnyCalib with panoramas that need not be aligned with gravity | |
| ([AnyCalib: Extended OpenPano Dataset](https://github.com/javrtg/AnyCalib?tab=readme-ov-file#extended-openpano-dataset)). | |
| It consists of tonemapped panoramas from [The Laval Photometric Indoor HDR Dataset](http://hdrdb.com/indoor-hdr-photometric/), | |
| [PolyHaven](https://polyhaven.com/hdris), [HDRMaps](https://hdrmaps.com/freebies/free-hdris/), | |
| [AmbientCG](https://ambientcg.com/list?type=hdri&sort=popular) and | |
| [BlenderKit](https://www.blenderkit.com/asset-gallery?query=category_subtree:hdr). AnyCalib's splits. | |
| - **CommonsPano** ([LiteCalib/CommonsPano](https://huggingface.co/datasets/LiteCalib/CommonsPano)): 360° | |
| panoramas from Wikimedia Commons, split by uploader, with a per-file attribution list. 465 duplicates were | |
| removed: 444 of OpenPano panoramas (found by image content and by source name) and 21 within Commons. Three | |
| training panoramas yield no valid view, so the views come from 9,333 of them. | |
| - **Panorama views**: 640 x 640 perspective and distorted views rendered from each panorama with AnyCalib's camera | |
| sampling (its `OP_g` recipe), principal point at the center, square pixels. | |
| - **MegaScenes** ([megascenes.github.io](https://megascenes.github.io)): Wikimedia Commons photos with COLMAP | |
| reconstructions. Each photo is center-cropped to 1:1, 4:3 or 3:2 at a short side of 600 px, its principal point | |
| at the center. Left out: reconstructions with fewer than 20 photos, failed self-calibrations, and the scenes of the | |
| MegaDepth evaluation landmarks. | |
| - **Training**: AnyCalib's resolution (about 102,400 pixels, aspect ratio 0.5 to 2) and its photometric | |
| augmentations. | |
| More in the [repository](https://github.com/kocurvik/litecalib#datasets). | |
| ## License | |
| Apache License 2.0. We thank the authors of [The Laval Photometric Indoor HDR Dataset](http://hdrdb.com/indoor-hdr-photometric/) for allowing us to | |
| release these weights under this license. | |