---
license: other
license_name: fair-noncommercial-research-license
license_link: LICENSE
library_name: any3r
pipeline_tag: image-to-3d
tags:
- 3d-reconstruction
- camera-pose-estimation
- depth-estimation
- fisheye
- panorama
- omnidirectional
- feed-forward
---

# Any3R: Feedforward 3D Reconstruction from Any Camera Type

<p align="left"><a href="https://arxiv.org/abs/2610.36150"><img src="https://img.shields.io/badge/arXiv-2610.36150-b31b1b?logo=arxiv&logoColor=white" alt="arXiv"></a> <a href="https://github.com/Jingong-Chen/Any3R"><img src="https://img.shields.io/badge/GitHub-Code-181717?logo=github&logoColor=white" alt="GitHub"></a> <a href="https://jingong-chen.github.io/Any3R/"><img src="https://img.shields.io/badge/Project-Page-4b6bfb?logo=googlechrome&logoColor=white" alt="Project Page"></a></p>

**Paper:** [arXiv:2610.36150](https://arxiv.org/abs/2610.36150) · **Code:** [github.com/Jingong-Chen/Any3R](https://github.com/Jingong-Chen/Any3R) · **Project page:** [jingong-chen.github.io/Any3R](https://jingong-chen.github.io/Any3R/)

Any3R reconstructs a scene from images taken with any central camera, in a single forward pass: pinhole and distorted lenses, fisheyes beyond 180°, 360° panoramas, and any mix of them in one scene. Each view is described by its ray field, the direction of every pixel, and the model predicts per-view depth along each ray, camera poses and a fused point cloud.

## Usage

```bash
git clone https://github.com/Jingong-Chen/Any3R.git && cd Any3R && pip install -e .
```

```python
from any3r import Any3R, cameras
from any3r.io import prepare_views

model = Any3R.from_pretrained("Jingong/Any3R").to("cuda").eval()
cams = [cameras.from_fov("equidistant", 1024, 1024, fov_deg=190), cameras.ERP()]
views = prepare_views(["fisheye.jpg", "panorama.jpg"], cams)
pred = model.infer(views.images, views.rays)    # depth, depth_conf, camera_poses, world_points
```

See the [repository](https://github.com/Jingong-Chen/Any3R) for camera conventions, COLMAP / OpenCV calibrations, demos and evaluation.

## Model

- 1.15 B parameters (float32 safetensors, 4.6 GB); VGGT-Omega trunk with ray-conditioned angular rotary coordinates and a ray position embedding.
- Fine-tuned from VGGT-Omega on Matterport3D, Stanford2D3DS, Structured3D, PanoCity and DL3DV, with panoramic samples resampled through eight central camera models (30° to 360° field of view).
- Inference in bfloat16 autocast (`Any3R.infer`); 32 views at 512 × 512 take 1.1 s on an RTX 5090.

## Acknowledgments

Any3R builds on the code and models of [VGGT-Omega](https://github.com/facebookresearch/vggt-omega), [VGGT](https://github.com/facebookresearch/vggt) and [PanoVGGT](https://github.com/YijingGuo-June/PanoVGGT), whose authors also released the [PanoCity](https://huggingface.co/datasets/YijingGuo/PanoCity) dataset. We train on [Matterport3D](https://niessner.github.io/Matterport/) (with the panoramas stitched by [360MonoDepth](https://github.com/manurare/360monodepth)), [Stanford2D3DS](https://github.com/alexsax/2D-3D-Semantics), [Structured3D](https://structured3d-dataset.org/), PanoCity and [DL3DV](https://dl3dv-10k.github.io/DL3DV-10K/), and evaluate zero-shot on [ScanNet++](https://kaldir.vc.in.tum.de/scannetpp/) and [FIORD](https://zenodo.org/records/15181976). We thank the authors of [π³](https://github.com/yyfz/Pi3), [MapAnything](https://github.com/facebookresearch/map-anything), [Depth Anything 3](https://github.com/ByteDance-Seed/Depth-Anything-3), [OmniVGGT](https://github.com/Livioni/OmniVGGT-official), [Wid3R](https://github.com/jdk9405/Wid3R) and [Argus](https://github.com/realsee-developer/Argus) for releasing the code we compare against. The evaluation uses [Open3D](https://www.open3d.org/).

## License

The weights are released under the FAIR Noncommercial Research License (see `LICENSE`), inherited from VGGT-Omega: noncommercial research use only. The code on [GitHub](https://github.com/Jingong-Chen/Any3R) is under the MIT License, except the VGGT-Omega-derived trunk (`any3r/_vggt_omega/` and `any3r/aggregator.py`), which is under the same FAIR license as the weights.

## Citation

```bibtex
@article{chen2026any3r,
  title   = {Any3R: Feedforward 3D Reconstruction from Any Camera Type},
  author  = {Chen, Jingong and Zhang, Qingwen and Deng, Youming and Pack, Chulwoo and
             Wysocki, Olaf and Jutzi, Boris and Won, Kwanghee and Gao, Kyle},
  journal = {arXiv preprint arXiv:2610.36150},
  year    = {2026}
}
```