DepthPro2-S / README.md
|
Raw History Blame Contribute Delete
4.17 kB
metadata
license: other
license_name: apple-amlr
license_link: https://github.com/apple-aiml-research/ml-depth-pro-2/blob/main/LICENSE_MODEL
pipeline_tag: depth-estimation
library_name: depth-pro-2
tags:
  - depth-estimation
  - metric-depth
  - monocular-depth
  - video-depth

Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth

GitHub Technical Report Project Page All Sizes

Model size S · 24M parameters

B200 VGA 720p 1080p 4K
Throughput (fps) 347.0 143.9 44.8 3.6
Peak memory (GiB) 1.9 3.8 8.0 31.9
Benchmark details
  • GPU: NVIDIA B200, in bfloat16 and PyTorch eager mode (no compilation)
  • Input: a 32-frame video clip, processed in one call at its native resolution
  • Resolutions: VGA is 640 × 480; 720p, 1080p and 4K are 16:9
  • Throughput: frames per second, averaged over the clip
  • Memory: the peak GPU memory PyTorch reserves
  • Included: the focal length estimator, a separate 23.5M-parameter model that loads with the depth model

How to use

pip install "git+https://github.com/apple-aiml-research/ml-depth-pro-2"
from functools import partial

import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from depth_pro_2 import DepthPro2, load_rgb
from depth_pro_2.model import to_normalized_tensor
from depth_pro_2.network.fov import FOVEstimator

model = DepthPro2.from_pretrained("apple/DepthPro2-S", precision=torch.bfloat16)
# The focal length estimator all sizes share, for photos and videos without a focal length.
weights = hf_hub_download("apple/DepthPro2-focal", "model.safetensors")
model.fov = FOVEstimator(torch.bfloat16)
model.fov.load_state_dict(load_file(weights))
model = model.to("cuda").eval()
transform = partial(to_normalized_tensor, device=torch.device("cuda"))

image, f_px = load_rgb("photo.jpg")  # f_px: EXIF focal length, or None
depth = model.infer(transform(image), f_px=f_px)["depth"]  # meters, (H, W)

Videos, the other sizes and the command line are on the Depth Pro 2 page.

License

The weights are released under the Apple Machine Learning Research Model license, for research use only: non-commercial scientific research and academic development.

Citation

@article{DepthPro2:2026:arxiv,
  title   = {Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth},
  author  = {Peiyun Hu and Marko Mihajlovic and Lyndon Duong and Lahav Lipson and Wei Dong and Ama\"{e}l Delaunoy and Stephan R. Richter and Vladlen Koltun},
  journal = {arXiv preprint arXiv:2611.26603},
  year    = {2026},
  url     = {https://arxiv.org/abs/2611.26603},
}