README.md

---
license: apache-2.0
library_name: transformers
base_model: depth-anything/prompt-depth-anything-vitl
tags:
- prompt-depth-anything
- depth-estimation
- metric-depth
pipeline_tag: depth-estimation
---

# Prompt-Depth-Anything-Large — transformers format (original code)

Unofficial transformers port of **Prompt Depth Anything** (`depth-anything/prompt-depth-anything-vitl`, `model.ckpt`) that runs the
**original** code. Loads with `trust_remote_code`. `model.safetensors` holds the checkpoint's tensors under their
original names (without the Lightning `pipeline.` prefix, as the original loader), every tensor checked equal. The model benchmarked in the paper (README).

> **Not an official release.** Prompt Depth Anything was created by Haotong Lin, Sida Peng, Jingxiao Chen, Songyou
> Peng, Jiaming Sun, Minghuan Liu, Hujun Bao, Jiashi Feng, Xiaowei Zhou and Bingyi Kang —
> https://github.com/DepthAnything/PromptDA, paper *"Prompting Depth Anything for 4K Resolution Accurate Metric Depth
> Estimation"* (CVPR 2025).

## What it is

Metric depth from an RGB image plus a low-resolution depth prompt (e.g. iPhone LiDAR, 192 × 256, meters).
`PromptDepthAnythingForDepthEstimation` holds the official `PromptDA`'s submodules and buffers (`pretrained`: DINOv2
ViT-L/14, `depth_head`: the prompt-conditioned DPT head, `_mean` / `_std`) and computes depth with the official
`PromptDA.forward`, returning the standard `DepthEstimatorOutput` (`predicted_depth`: (B, H, W), meters); the official
`predict` is kept. DINOv2 is built from the repo's torchhub copy with the hub's arguments (the original calls
`torch.hub.load(..., source='local')`). Code vendored from DepthAnything/PromptDA @ 9161547c into `promptda_dinov2.py` /
`promptda_model.py`; edits: imports, and `Block` → `NestedTensorBlock` where the original imported
`NestedTensorBlock as Block`. `PromptDepthAnythingProcessor` (`processing_prompt_depth_anything.py`,
`processor_config.json`) wraps `PromptDepthAnythingImageProcessor`, the original `load_image` / `load_depth`.

Verified against the official code (same process, V100): preprocessing (image and prompt depth), `predicted_depth`
(also as a tuple), `predict`, the hidden states and fp16 autocast outputs are **bit-identical**.

## Usage

```python
import torch
from transformers import AutoModelForDepthEstimation, AutoProcessor

model = AutoModelForDepthEstimation.from_pretrained("Prompt-Depth-Anything-Large", trust_remote_code=True).eval().cuda()
processor = AutoProcessor.from_pretrained("Prompt-Depth-Anything-Large", trust_remote_code=True)
# image: path (read with imageio, as the original), PIL image or RGB uint8 array; prompt_depth: .png (mm) / .npz path or meters
inputs = processor(images="image.jpg", prompt_depth="arkit_depth.png", return_tensors="pt").to("cuda")
with torch.no_grad():
    outputs = model(**inputs)                 # outputs.predicted_depth: (1, H, W), meters, at the processed image size
```

- `AutoModel` loads the same class. `output_hidden_states=True` adds the DINOv2 features the head reads; `return_dict=False` returns a tuple.
- `pixel_values` are RGB in [0, 1]: the model normalises them itself, as the original.
- The depth is at the processed image size (longer side at most 1008, multiples of 14), as the original (no resizing back).

Runtime packages beyond torch / transformers: `opencv-python`, `imageio` (as the original).

## Licensing

- Code and weights: **Apache-2.0** — [`LICENSE`](LICENSE) (DepthAnything/PromptDA), as published on `depth-anything/prompt-depth-anything-vitl`.
- DINOv2 code (the repo's torchhub copy): **Apache-2.0** — [`LICENSE.dinov2`](LICENSE.dinov2).
- Attribution, sources and the changes made here: [`NOTICE`](NOTICE).

## Citation

If you use this model, please cite the original work, as its authors ask:

```bibtex
@inproceedings{lin2024promptda,
  title={Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation},
  author={Lin, Haotong and Peng, Sida and Chen, Jingxiao and Peng, Songyou and Sun, Jiaming and Liu, Minghuan and Bao, Hujun and Feng, Jiashi and Zhou, Xiaowei and Kang, Bingyi},
  journal={arXiv},
  year={2024}
}
```