README.md

---
license: cc-by-nc-4.0
library_name: transformers
base_model: depth-anything/Video-Depth-Anything-Base
tags:
- video-depth-anything
- depth-estimation
- video
pipeline_tag: depth-estimation
---

# Video-Depth-Anything-Base — transformers format (original code)

Unofficial transformers port of **Video Depth Anything** Base (`depth-anything/Video-Depth-Anything-Base`, `video_depth_anything_vitb.pth`) that runs
the **original** code. Loads with `trust_remote_code`. `model.safetensors` holds the checkpoint's tensors under their
original names, converted from `.pth` (every tensor checked equal).

> **Not an official release.** Video Depth Anything was created by Sili Chen, Hengkai Guo, Shengnan Zhu, Feihu Zhang,
> Zilong Huang, Jiashi Feng and Bingyi Kang (ByteDance) — https://github.com/DepthAnything/Video-Depth-Anything, paper
> *"Video Depth Anything: Consistent Depth Estimation for Super-Long Videos"* (arXiv 2501.12375).

## What it is

`VideoDepthAnythingForDepthEstimation` holds the official `VideoDepthAnything`'s submodules (`pretrained`: DINOv2
ViT-B/14, `head`: the temporal DPT head with motion modules) and computes depth with that class's own `forward`, returning
the standard `DepthEstimatorOutput` (`predicted_depth`: (B, T, H, W), up to 32 frames per clip). The official
whole-video inference `infer_video_depth` (32-frame windows with key-frame overlap and scale / shift alignment) and
the experimental streaming inference `infer_video_depth_one` run unchanged on the model. Code vendored
from DepthAnything/Video-Depth-Anything @ 4f5ae231 into `vda_*.py`; edits: imports, and `Block` → `NestedTensorBlock`
where the original imported `NestedTensorBlock as Block`. `VideoDepthAnythingProcessor`
(`processing_video_depth_anything.py`, `processor_config.json`) wraps `VideoDepthAnythingImageProcessor`, the per-frame
transform of `infer_video_depth`.

Verified against the official code (same process, V100): preprocessing, `predicted_depth` (also as a tuple), the
hidden states, the post-processing, `infer_video_depth` on a video longer than one window and streaming inference, in
fp32 and with the original's fp16 autocast, are **bit-identical** (the repo's 2 example videos; 70 frames for
`infer_video_depth`, 40 streamed frames).

## Usage

```python
import torch
from transformers import AutoModelForDepthEstimation, AutoProcessor

model = AutoModelForDepthEstimation.from_pretrained("Video-Depth-Anything-Base", trust_remote_code=True).eval().cuda()
processor = AutoProcessor.from_pretrained("Video-Depth-Anything-Base", trust_remote_code=True)
# frames: RGB uint8 (T, H, W, 3), T <= 32
inputs = processor(videos=frames, return_tensors="pt").to("cuda")
with torch.no_grad():
    outputs = model(**inputs)                       # outputs.predicted_depth: (1, T, H', W') relative inverse depth
depth = processor.post_process_depth_estimation(outputs, target_sizes=[frames.shape[1:3]])[0]["predicted_depth"]

# a whole video of any length, exactly as the original run.py (fp16 autocast unless fp32=True):
depths, fps = model.infer_video_depth(frames, target_fps=30, input_size=518, device="cuda")
# streaming, one frame at a time (experimental upstream): model.infer_video_depth_one(frame); model.reset_streaming()
```

- `AutoModel` loads the same class. `output_hidden_states=True` adds the DINOv2 features the head reads; `return_dict=False` returns a tuple.
- `post_process_depth_estimation` resizes as the original (bilinear, `align_corners=True`).

Runtime packages beyond torch / transformers: `opencv-python`, `torchvision`, `einops`, `easydict`, `tqdm` (as the
original; `xformers` is optional, as in the original).

## Licensing

- Weights (`depth-anything/Video-Depth-Anything-Base`): **CC-BY-NC-4.0** (non-commercial) — [`LICENSE`](LICENSE).
- Code (vendored files and the port): **Apache-2.0** — [`LICENSE.code`](LICENSE.code).
- Attribution, sources and the changes made here: [`NOTICE`](NOTICE).

## Citation

If you use this model, please cite the original work, as its authors ask:

```bibtex
@article{video_depth_anything,
  title={Video Depth Anything: Consistent Depth Estimation for Super-Long Videos},
  author={Chen, Sili and Guo, Hengkai and Zhu, Shengnan and Zhang, Feihu and Huang, Zilong and Feng, Jiashi and Kang, Bingyi},
  journal={arXiv:2501.12375},
  year={2025}
}
```