Prompt-Depth-Anything-Large — transformers format (original code)

Unofficial transformers port of Prompt Depth Anything (depth-anything/prompt-depth-anything-vitl, model.ckpt) that runs the original code. Loads with trust_remote_code. model.safetensors holds the checkpoint's tensors under their original names (without the Lightning pipeline. prefix, as the original loader), every tensor checked equal. The model benchmarked in the paper (README).

Not an official release. Prompt Depth Anything was created by Haotong Lin, Sida Peng, Jingxiao Chen, Songyou Peng, Jiaming Sun, Minghuan Liu, Hujun Bao, Jiashi Feng, Xiaowei Zhou and Bingyi Kang — https://github.com/DepthAnything/PromptDA, paper "Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation" (CVPR 2025).

What it is

Metric depth from an RGB image plus a low-resolution depth prompt (e.g. iPhone LiDAR, 192 × 256, meters). PromptDepthAnythingForDepthEstimation holds the official PromptDA's submodules and buffers (pretrained: DINOv2 ViT-L/14, depth_head: the prompt-conditioned DPT head, _mean / _std) and computes depth with the official PromptDA.forward, returning the standard DepthEstimatorOutput (predicted_depth: (B, H, W), meters); the official predict is kept. DINOv2 is built from the repo's torchhub copy with the hub's arguments (the original calls torch.hub.load(..., source='local')). Code vendored from DepthAnything/PromptDA @ 9161547c into promptda_dinov2.py / promptda_model.py; edits: imports, and Block → NestedTensorBlock where the original imported NestedTensorBlock as Block. PromptDepthAnythingProcessor (processing_prompt_depth_anything.py, processor_config.json) wraps PromptDepthAnythingImageProcessor, the original load_image / load_depth.

Verified against the official code (same process, V100): preprocessing (image and prompt depth), predicted_depth (also as a tuple), predict, the hidden states and fp16 autocast outputs are bit-identical.

Usage

import torch
from transformers import AutoModelForDepthEstimation, AutoProcessor

model = AutoModelForDepthEstimation.from_pretrained("Prompt-Depth-Anything-Large", trust_remote_code=True).eval().cuda()
processor = AutoProcessor.from_pretrained("Prompt-Depth-Anything-Large", trust_remote_code=True)
# image: path (read with imageio, as the original), PIL image or RGB uint8 array; prompt_depth: .png (mm) / .npz path or meters
inputs = processor(images="image.jpg", prompt_depth="arkit_depth.png", return_tensors="pt").to("cuda")
with torch.no_grad():
    outputs = model(**inputs)                 # outputs.predicted_depth: (1, H, W), meters, at the processed image size

• AutoModel loads the same class. output_hidden_states=True adds the DINOv2 features the head reads; return_dict=False returns a tuple.
• pixel_values are RGB in [0, 1]: the model normalises them itself, as the original.
• The depth is at the processed image size (longer side at most 1008, multiples of 14), as the original (no resizing back).

Runtime packages beyond torch / transformers: opencv-python, imageio (as the original).

Licensing

• Code and weights: Apache-2.0 — LICENSE (DepthAnything/PromptDA), as published on depth-anything/prompt-depth-anything-vitl.
• DINOv2 code (the repo's torchhub copy): Apache-2.0 — LICENSE.dinov2.
• Attribution, sources and the changes made here: NOTICE.

Citation

If you use this model, please cite the original work, as its authors ask:

@inproceedings{lin2024promptda,
  title={Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation},
  author={Lin, Haotong and Peng, Sida and Chen, Jingxiao and Peng, Songyou and Sun, Jiaming and Liu, Minghuan and Bao, Hujun and Feng, Jiashi and Zhou, Xiaowei and Kang, Bingyi},
  journal={arXiv},
  year={2024}
}