File size: 3,586 Bytes
622659d
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
---
license: other
license_name: apple-amlr
license_link: https://github.com/apple-aiml-research/ml-depth-pro-2/blob/main/LICENSE_MODEL
library_name: depth-pro-2
tags:
  - focal-length-estimation
  - field-of-view
  - camera-calibration
  - depth-estimation
---

<h1 align="center">Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth</h1>

<p align="center">
  <a href="https://github.com/apple-aiml-research/ml-depth-pro-2"><img src="https://img.shields.io/badge/GitHub-181717?style=flat-square&amp;logo=github&amp;logoColor=white" alt="GitHub"/></a>
  <a href="https://arxiv.org/abs/2611.26603"><img src="https://img.shields.io/badge/Technical%20Report-B31B1B?style=flat-square&amp;logo=arxiv&amp;logoColor=white" alt="Technical Report"/></a>
  <a href="https://apple-aiml-research.github.io/ml-depth-pro-2/"><img src="https://img.shields.io/badge/Project%20Page-666666?style=flat-square&amp;logo=data%3Aimage%2Fsvg%2Bxml%3Bbase64%2CPHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0ibm9uZSIgc3Ryb2tlPSJ3aGl0ZSIgc3Ryb2tlLXdpZHRoPSIyIj48Y2lyY2xlIGN4PSIxMiIgY3k9IjEyIiByPSIxMCIvPjxwYXRoIGQ9Ik0yIDEyaDIwTTEyIDJhMTUgMTUgMCAwIDEgMCAyME0xMiAyYTE1IDE1IDAgMCAwIDAgMjAiLz48L3N2Zz4%3D" alt="Project Page"/></a>
  <a href="https://huggingface.co/apple/DepthPro2"><img src="https://img.shields.io/badge/All%20Sizes-FFD21E?style=flat-square&amp;logo=huggingface&amp;logoColor=black" alt="All Sizes"/></a>
</p>
<h3 align="center">Focal length estimator · 23.5M parameters</h3>

## How to use

```bash
pip install "git+https://github.com/apple-aiml-research/ml-depth-pro-2"
```

Load it with any size, for photos and videos without a focal length:

```python
import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from depth_pro_2 import DepthPro2
from depth_pro_2.network.fov import FOVEstimator

weights = hf_hub_download("apple/DepthPro2-focal", "model.safetensors")
model = DepthPro2.from_pretrained("apple/DepthPro2-H", precision=torch.bfloat16)
model.fov = FOVEstimator(torch.bfloat16)
model.fov.load_state_dict(load_file(weights))
model = model.to("cuda").eval()
```

On its own, it estimates the focal length of a photo, or one for the frames of a clip:

```python
from functools import partial

import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from depth_pro_2 import load_rgb
from depth_pro_2.model import to_normalized_tensor
from depth_pro_2.network.fov import FOVEstimator

weights = hf_hub_download("apple/DepthPro2-focal", "model.safetensors")
fov = FOVEstimator()
fov.load_state_dict(load_file(weights))
fov = fov.to("cuda").eval()
transform = partial(to_normalized_tensor, device=torch.device("cuda"))

image, _ = load_rgb("photo.jpg")
with torch.no_grad():
    f_px = fov.focal_length_px(transform(image)[None])  # pixels of the photo; frames (T, 3, H, W)
```

## License

The weights are released under the [Apple Machine Learning Research Model license](https://github.com/apple-aiml-research/ml-depth-pro-2/blob/main/LICENSE_MODEL), for
research use only: non-commercial scientific research and academic development.

## Citation

```bibtex
@article{DepthPro2:2026:arxiv,
  title   = {Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth},
  author  = {Peiyun Hu and Marko Mihajlovic and Lyndon Duong and Lahav Lipson and Wei Dong and Ama\"{e}l Delaunoy and Stephan R. Richter and Vladlen Koltun},
  journal = {arXiv preprint arXiv:2611.26603},
  year    = {2026},
  url     = {https://arxiv.org/abs/2611.26603},
}
```