Instructions to use christianengel/DepthPro2-B-ONNX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- TensorRT
How to use christianengel/DepthPro2-B-ONNX with TensorRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Depth Pro 2 B ONNX, focal-free
Unofficial ONNX export of apple/DepthPro2-B for calibrated cameras. Pass the focal length f_px; the 23.5M-parameter focal-length network is gone. Output is metric depth in meters. Every graph passes a parity gate against PyTorch. Try it in the browser. Not affiliated with or endorsed by Apple.
Research use only. These weights are a Model Derivative under the Apple Machine Learning Research Model License: non-commercial research and academic use only. Apple Machine Learning Research Model is licensed under the Apple Machine Learning Research Model License Agreement.
Quick start
pip install onnxruntime-gpu huggingface_hub pillow numpy # onnxruntime for CPU
import json
import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
repo = "christianengel/DepthPro2-B-ONNX"
config = json.load(open(hf_hub_download(repo, "config.json")))
session = ort.InferenceSession(
hf_hub_download(repo, "onnx/model_fp16.onnx"),
providers=["CUDAExecutionProvider", "CPUExecutionProvider"],
)
# pixel_values: RGB in [0, 1], shape (1, 3, 480, 640). f_px: fx of that 640x480 image in pixels.
depth = session.run(None, {"pixel_values": pixels, "f_px": np.array([fx], np.float32)})[0]
example.py resizes and crops any photo and scales f_px with it. From ROS 2, f_px is CameraInfo.k[0] of the rectified image.
Files
Static shapes, opset 18. AbsRel and delta1 are against PyTorch FP32 on a 17-frame fixture.
| File | Input | Frames | Precision | MB | AbsRel | delta1 |
|---|---|---|---|---|---|---|
onnx/model.onnx |
480x640 | 1 frame | FP32 | 387 | 0.0002 | 1.0000 |
onnx/model_fp16.onnx |
480x640 | 1 frame | FP16 | 199 | 0.0007 | 1.0000 |
onnx/model_int8.onnx |
480x640 | 1 frame | INT8 | 143 | 0.0032 | 1.0000 |
onnx/model_clip4.onnx |
480x640 | 4 frames attending to each other | FP32 | 389 | 0.0002 | 1.0000 |
onnx/model_clip4_fp16.onnx |
480x640 | 4 frames attending to each other | FP16 | 201 | 0.0006 | 1.0000 |
onnx/model_stream4.onnx |
480x640 | 1 frame + 3-frame cache as state | FP32 | 387 | 0.0001 | 1.0000 |
onnx/model_stream4_fp16.onnx |
480x640 | 1 frame + 3-frame cache as state | FP16 | 199 | 0.0005 | 1.0000 |
onnx/384x672/model.onnx |
384x672 | 1 frame | FP32 | 387 | 0.0003 | 1.0000 |
onnx/384x672/model_fp16.onnx |
384x672 | 1 frame | FP16 | 199 | 0.0007 | 1.0000 |
onnx/384x672/model_int8.onnx |
384x672 | 1 frame | INT8 | 143 | 0.0026 | 1.0000 |
onnx/384x672/model_stream4.onnx |
384x672 | 1 frame + 3-frame cache as state | FP32 | 387 | 0.0002 | 1.0000 |
onnx/384x672/model_stream4_fp16.onnx |
384x672 | 1 frame + 3-frame cache as state | FP16 | 199 | 0.0005 | 1.0000 |
- Single frame (
model*.onnx): each frame on its own. - Stream (
model_stream4*.onnx): one frame per call, attending to the previous three; Depth Pro 2's unbuffered video mode. The cache goes in aspast_*and comes out aspresent_*; zeros for the first frame. - Clip (
model_clip4*.onnx): four frames jointly, asDepthPro2.inferon a(4, 3, H, W)clip.
I/O:
pixel_values: float32 RGB in [0, 1], NCHW, not normalized.f_px: float32(1,), horizontal focal length in pixels ofpixel_values.depth_meters: float32(T, H, W), z-depth in meters;infwhere infinitely far (sky).
Speed
NVIDIA GeForce RTX 3090, ONNX Runtime 1.23.2, 480x640, one frame at a time, host array to host array. Peak VRAM is the whole process (NVML).
| Model | Runtime | Latency (ms/frame) | FPS | Peak VRAM (MB) | Disk (MB) | AbsRel vs PyTorch FP32 |
|---|---|---|---|---|---|---|
| DepthPro2-B | PyTorch BF16 (Apple default) | 11.5 | 86.6 | 1198 | - | 0.0022 |
| DepthPro2-B | PyTorch FP32 | 24.9 | 40.1 | 1112 | - | 0.0000 |
| DepthPro2-B | ORT CUDA FP32 | 22.4 | 44.6 | 1314 | 387 | 0.0002 |
| DepthPro2-B | ORT CUDA FP16 | 14.9 | 67.1 | 1322 | 199 | 0.0007 |
| DepthPro2-B | ORT CUDA INT8 | 23.1 | 43.2 | 1308 | 143 | 0.0030 |
| DepthPro2-B | TensorRT FP16 engine | 7.1 | 140.5 | 700 | 204 | 0.0009 |
| DepthPro2-B | TensorRT FP16 engine, stream window 4 | 38.9 | 25.7 | 830 | 204 | 0.0009 |
| DepthPro2-B | ORT CUDA FP16 stream, window 4 | 21.7 | 46.0 | 2472 | 199 | 0.0005 |
| DepthPro2-B | ORT CUDA FP16 clip of 4 | 19.2 | 52.1 | 5430 | 201 | 0.0006 |
TensorRT rows are engines from build_tensorrt.sh on that GPU. The stream engine copies its state through host memory each frame; ONNX Runtime keeps it on the GPU.
TensorRT and Jetson
On the device: ./build_tensorrt.sh onnx/model_fp16.onnx (script, trtexec --stronglyTyped). Do not use --fp16 or ONNX Runtime's TensorRT provider: both may run block 0's attention (logits up to 2e6) or B's LayerNorms in FP16 and return NaN.
ROS 2
Node: subscribes to Image and CameraInfo, takes f_px from k[0], publishes 32FC1 depth in meters, matching CameraInfo, optional PointCloud2.
Changes from the original
- Focal-length network (apple/DepthPro2-focal) removed;
f_pxis an input. - ImageNet normalization and
depth = f_px / (W * inverse_depth)are in the graph. - No padding: sides must be multiples of 32.
- FP16: FP16 weights; block 0 attention, RoPE tables, depth head and final division stay FP32.
- INT8: weight-only 8-bit MatMul weights (symmetric, blocks of 32). Dynamic activation INT8 collapses this backbone (delta1 < 0.5).
- Stream graphs follow the unbuffered step of Apple's Core ML exporter, with the cache as explicit tensors.
Source: apple/DepthPro2-B@281dc98, 96.5M parameters. Export code: https://github.com/protortyp/depthpro2-onnx (Apache-2.0).
Citation
@article{DepthPro2:2026:arxiv,
title = {Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth},
author = {Peiyun Hu and Marko Mihajlovic and Lyndon Duong and Lahav Lipson and Wei Dong and Ama\"{e}l Delaunoy and Stephan R. Richter and Vladlen Koltun},
journal = {arXiv preprint arXiv:2611.26603},
year = {2026},
}
- Downloads last month
- -
Model tree for christianengel/DepthPro2-B-ONNX
Base model
apple/DepthPro2-B