| --- |
| license: other |
| license_name: fair-noncommercial-research-license |
| license_link: LICENSE |
| library_name: any3r |
| pipeline_tag: image-to-3d |
| tags: |
| - 3d-reconstruction |
| - camera-pose-estimation |
| - depth-estimation |
| - fisheye |
| - panorama |
| - omnidirectional |
| - feed-forward |
| --- |
| |
| # Any3R: Feedforward 3D Reconstruction from Any Camera Type |
|
|
| <p align="left"><a href="https://arxiv.org/abs/2610.36150"><img src="https://img.shields.io/badge/arXiv-2610.36150-b31b1b?logo=arxiv&logoColor=white" alt="arXiv"></a> <a href="https://github.com/Jingong-Chen/Any3R"><img src="https://img.shields.io/badge/GitHub-Code-181717?logo=github&logoColor=white" alt="GitHub"></a> <a href="https://jingong-chen.github.io/Any3R/"><img src="https://img.shields.io/badge/Project-Page-4b6bfb?logo=googlechrome&logoColor=white" alt="Project Page"></a></p> |
|
|
| **Paper:** [arXiv:2610.36150](https://arxiv.org/abs/2610.36150) · **Code:** [github.com/Jingong-Chen/Any3R](https://github.com/Jingong-Chen/Any3R) · **Project page:** [jingong-chen.github.io/Any3R](https://jingong-chen.github.io/Any3R/) |
|
|
| Any3R reconstructs a scene from images taken with any central camera, in a single forward pass: pinhole and distorted lenses, fisheyes beyond 180°, 360° panoramas, and any mix of them in one scene. Each view is described by its ray field, the direction of every pixel, and the model predicts per-view depth along each ray, camera poses and a fused point cloud. |
|
|
| ## Usage |
|
|
| ```bash |
| git clone https://github.com/Jingong-Chen/Any3R.git && cd Any3R && pip install -e . |
| ``` |
|
|
| ```python |
| from any3r import Any3R, cameras |
| from any3r.io import prepare_views |
| |
| model = Any3R.from_pretrained("Jingong/Any3R").to("cuda").eval() |
| cams = [cameras.from_fov("equidistant", 1024, 1024, fov_deg=190), cameras.ERP()] |
| views = prepare_views(["fisheye.jpg", "panorama.jpg"], cams) |
| pred = model.infer(views.images, views.rays) # depth, depth_conf, camera_poses, world_points |
| ``` |
|
|
| See the [repository](https://github.com/Jingong-Chen/Any3R) for camera conventions, COLMAP / OpenCV calibrations, demos and evaluation. |
|
|
| ## Model |
|
|
| - 1.15 B parameters (float32 safetensors, 4.6 GB); VGGT-Omega trunk with ray-conditioned angular rotary coordinates and a ray position embedding. |
| - Fine-tuned from VGGT-Omega on Matterport3D, Stanford2D3DS, Structured3D, PanoCity and DL3DV, with panoramic samples resampled through eight central camera models (30° to 360° field of view). |
| - Inference in bfloat16 autocast (`Any3R.infer`); 32 views at 512 × 512 take 1.1 s on an RTX 5090. |
|
|
| ## Acknowledgments |
|
|
| Any3R builds on the code and models of [VGGT-Omega](https://github.com/facebookresearch/vggt-omega), [VGGT](https://github.com/facebookresearch/vggt) and [PanoVGGT](https://github.com/YijingGuo-June/PanoVGGT), whose authors also released the [PanoCity](https://huggingface.co/datasets/YijingGuo/PanoCity) dataset. We train on [Matterport3D](https://niessner.github.io/Matterport/) (with the panoramas stitched by [360MonoDepth](https://github.com/manurare/360monodepth)), [Stanford2D3DS](https://github.com/alexsax/2D-3D-Semantics), [Structured3D](https://structured3d-dataset.org/), PanoCity and [DL3DV](https://dl3dv-10k.github.io/DL3DV-10K/), and evaluate zero-shot on [ScanNet++](https://kaldir.vc.in.tum.de/scannetpp/) and [FIORD](https://zenodo.org/records/15181976). We thank the authors of [π³](https://github.com/yyfz/Pi3), [MapAnything](https://github.com/facebookresearch/map-anything), [Depth Anything 3](https://github.com/ByteDance-Seed/Depth-Anything-3), [OmniVGGT](https://github.com/Livioni/OmniVGGT-official), [Wid3R](https://github.com/jdk9405/Wid3R) and [Argus](https://github.com/realsee-developer/Argus) for releasing the code we compare against. The evaluation uses [Open3D](https://www.open3d.org/). |
|
|
| ## License |
|
|
| The weights are released under the FAIR Noncommercial Research License (see `LICENSE`), inherited from VGGT-Omega: noncommercial research use only. The code on [GitHub](https://github.com/Jingong-Chen/Any3R) is under the MIT License, except the VGGT-Omega-derived trunk (`any3r/_vggt_omega/` and `any3r/aggregator.py`), which is under the same FAIR license as the weights. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{chen2026any3r, |
| title = {Any3R: Feedforward 3D Reconstruction from Any Camera Type}, |
| author = {Chen, Jingong and Zhang, Qingwen and Deng, Youming and Pack, Chulwoo and |
| Wysocki, Olaf and Jutzi, Boris and Won, Kwanghee and Gao, Kyle}, |
| journal = {arXiv preprint arXiv:2610.36150}, |
| year = {2026} |
| } |
| ``` |
|
|
