| --- |
| license: apache-2.0 |
| library_name: transformers |
| base_model: depth-anything/prompt-depth-anything-vits |
| tags: |
| - prompt-depth-anything |
| - depth-estimation |
| - metric-depth |
| pipeline_tag: depth-estimation |
| --- |
| |
| # Prompt-Depth-Anything-Small — transformers format (original code) |
|
|
| Unofficial transformers port of **Prompt Depth Anything** (`depth-anything/prompt-depth-anything-vits`, `model.ckpt`) that runs the |
| **original** code. Loads with `trust_remote_code`. `model.safetensors` holds the checkpoint's tensors under their |
| original names (without the Lightning `pipeline.` prefix, as the original loader), every tensor checked equal. |
|
|
| > **Not an official release.** Prompt Depth Anything was created by Haotong Lin, Sida Peng, Jingxiao Chen, Songyou |
| > Peng, Jiaming Sun, Minghuan Liu, Hujun Bao, Jiashi Feng, Xiaowei Zhou and Bingyi Kang — |
| > https://github.com/DepthAnything/PromptDA, paper *"Prompting Depth Anything for 4K Resolution Accurate Metric Depth |
| > Estimation"* (CVPR 2025). |
|
|
| ## What it is |
|
|
| Metric depth from an RGB image plus a low-resolution depth prompt (e.g. iPhone LiDAR, 192 × 256, meters). |
| `PromptDepthAnythingForDepthEstimation` holds the official `PromptDA`'s submodules and buffers (`pretrained`: DINOv2 |
| ViT-S/14, `depth_head`: the prompt-conditioned DPT head, `_mean` / `_std`) and computes depth with the official |
| `PromptDA.forward`, returning the standard `DepthEstimatorOutput` (`predicted_depth`: (B, H, W), meters); the official |
| `predict` is kept. DINOv2 is built from the repo's torchhub copy with the hub's arguments (the original calls |
| `torch.hub.load(..., source='local')`). Code vendored from DepthAnything/PromptDA @ 9161547c into `promptda_dinov2.py` / |
| `promptda_model.py`; edits: imports, and `Block` → `NestedTensorBlock` where the original imported |
| `NestedTensorBlock as Block`. `PromptDepthAnythingProcessor` (`processing_prompt_depth_anything.py`, |
| `processor_config.json`) wraps `PromptDepthAnythingImageProcessor`, the original `load_image` / `load_depth`. |
|
|
| Verified against the official code (same process, V100): preprocessing (image and prompt depth), `predicted_depth` |
| (also as a tuple), `predict`, the hidden states and fp16 autocast outputs are **bit-identical**. |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from transformers import AutoModelForDepthEstimation, AutoProcessor |
| |
| model = AutoModelForDepthEstimation.from_pretrained("Prompt-Depth-Anything-Small", trust_remote_code=True).eval().cuda() |
| processor = AutoProcessor.from_pretrained("Prompt-Depth-Anything-Small", trust_remote_code=True) |
| # image: path (read with imageio, as the original), PIL image or RGB uint8 array; prompt_depth: .png (mm) / .npz path or meters |
| inputs = processor(images="image.jpg", prompt_depth="arkit_depth.png", return_tensors="pt").to("cuda") |
| with torch.no_grad(): |
| outputs = model(**inputs) # outputs.predicted_depth: (1, H, W), meters, at the processed image size |
| ``` |
|
|
| - `AutoModel` loads the same class. `output_hidden_states=True` adds the DINOv2 features the head reads; `return_dict=False` returns a tuple. |
| - `pixel_values` are RGB in [0, 1]: the model normalises them itself, as the original. |
| - The depth is at the processed image size (longer side at most 1008, multiples of 14), as the original (no resizing back). |
|
|
| Runtime packages beyond torch / transformers: `opencv-python`, `imageio` (as the original). |
|
|
| ## Licensing |
|
|
| - Code and weights: **Apache-2.0** — [`LICENSE`](LICENSE) (DepthAnything/PromptDA), as published on `depth-anything/prompt-depth-anything-vits`. |
| - DINOv2 code (the repo's torchhub copy): **Apache-2.0** — [`LICENSE.dinov2`](LICENSE.dinov2). |
| - Attribution, sources and the changes made here: [`NOTICE`](NOTICE). |
|
|
| ## Citation |
|
|
| If you use this model, please cite the original work, as its authors ask: |
|
|
| ```bibtex |
| @inproceedings{lin2024promptda, |
| title={Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation}, |
| author={Lin, Haotong and Peng, Sida and Chen, Jingxiao and Peng, Songyou and Sun, Jiaming and Liu, Minghuan and Bao, Hujun and Feng, Jiashi and Zhou, Xiaowei and Kang, Bingyi}, |
| journal={arXiv}, |
| year={2024} |
| } |
| ``` |
|
|
