license: other
license_name: fair-noncommercial-research-license
license_link: LICENSE
library_name: any3r
pipeline_tag: image-to-3d
tags:
- 3d-reconstruction
- camera-pose-estimation
- depth-estimation
- fisheye
- panorama
- omnidirectional
- feed-forward
Any3R: Feedforward 3D Reconstruction from Any Camera Type
Paper: arXiv:2610.36150 · Code: github.com/Jingong-Chen/Any3R · Project page: jingong-chen.github.io/Any3R
Any3R reconstructs a scene from images taken with any central camera, in a single forward pass: pinhole and distorted lenses, fisheyes beyond 180°, 360° panoramas, and any mix of them in one scene. Each view is described by its ray field, the direction of every pixel, and the model predicts per-view depth along each ray, camera poses and a fused point cloud.
Usage
git clone https://github.com/Jingong-Chen/Any3R.git && cd Any3R && pip install -e .
from any3r import Any3R, cameras
from any3r.io import prepare_views
model = Any3R.from_pretrained("Jingong/Any3R").to("cuda").eval()
cams = [cameras.from_fov("equidistant", 1024, 1024, fov_deg=190), cameras.ERP()]
views = prepare_views(["fisheye.jpg", "panorama.jpg"], cams)
pred = model.infer(views.images, views.rays) # depth, depth_conf, camera_poses, world_points
See the repository for camera conventions, COLMAP / OpenCV calibrations, demos and evaluation.
Model
- 1.15 B parameters (float32 safetensors, 4.6 GB); VGGT-Omega trunk with ray-conditioned angular rotary coordinates and a ray position embedding.
- Fine-tuned from VGGT-Omega on Matterport3D, Stanford2D3DS, Structured3D, PanoCity and DL3DV, with panoramic samples resampled through eight central camera models (30° to 360° field of view).
- Inference in bfloat16 autocast (
Any3R.infer); 32 views at 512 × 512 take 1.1 s on an RTX 5090.
Acknowledgments
Any3R builds on the code and models of VGGT-Omega, VGGT and PanoVGGT, whose authors also released the PanoCity dataset. We train on Matterport3D (with the panoramas stitched by 360MonoDepth), Stanford2D3DS, Structured3D, PanoCity and DL3DV, and evaluate zero-shot on ScanNet++ and FIORD. We thank the authors of π³, MapAnything, Depth Anything 3, OmniVGGT, Wid3R and Argus for releasing the code we compare against. The evaluation uses Open3D.
License
The weights are released under the FAIR Noncommercial Research License (see LICENSE), inherited from VGGT-Omega: noncommercial research use only. The code on GitHub is under the MIT License, except the VGGT-Omega-derived trunk (any3r/_vggt_omega/ and any3r/aggregator.py), which is under the same FAIR license as the weights.
Citation
@article{chen2026any3r,
title = {Any3R: Feedforward 3D Reconstruction from Any Camera Type},
author = {Chen, Jingong and Zhang, Qingwen and Deng, Youming and Pack, Chulwoo and
Wysocki, Olaf and Jutzi, Boris and Won, Kwanghee and Gao, Kyle},
journal = {arXiv preprint arXiv:2610.36150},
year = {2026}
}