| --- |
| license: cc-by-nc-4.0 |
| library_name: transformers |
| base_model: depth-anything/Video-Depth-Anything-Large |
| tags: |
| - video-depth-anything |
| - depth-estimation |
| - video |
| pipeline_tag: depth-estimation |
| --- |
| |
| # Video-Depth-Anything-Large — transformers format (original code) |
|
|
| Unofficial transformers port of **Video Depth Anything** Large (`depth-anything/Video-Depth-Anything-Large`, `video_depth_anything_vitl.pth`) that runs |
| the **original** code. Loads with `trust_remote_code`. `model.safetensors` holds the checkpoint's tensors under their |
| original names, converted from `.pth` (every tensor checked equal). |
|
|
| > **Not an official release.** Video Depth Anything was created by Sili Chen, Hengkai Guo, Shengnan Zhu, Feihu Zhang, |
| > Zilong Huang, Jiashi Feng and Bingyi Kang (ByteDance) — https://github.com/DepthAnything/Video-Depth-Anything, paper |
| > *"Video Depth Anything: Consistent Depth Estimation for Super-Long Videos"* (arXiv 2501.12375). |
|
|
| ## What it is |
|
|
| `VideoDepthAnythingForDepthEstimation` holds the official `VideoDepthAnything`'s submodules (`pretrained`: DINOv2 |
| ViT-L/14, `head`: the temporal DPT head with motion modules) and computes depth with that class's own `forward`, returning |
| the standard `DepthEstimatorOutput` (`predicted_depth`: (B, T, H, W), up to 32 frames per clip). The official |
| whole-video inference `infer_video_depth` (32-frame windows with key-frame overlap and scale / shift alignment) and |
| the experimental streaming inference `infer_video_depth_one` run unchanged on the model. Code vendored |
| from DepthAnything/Video-Depth-Anything @ 4f5ae231 into `vda_*.py`; edits: imports, and `Block` → `NestedTensorBlock` |
| where the original imported `NestedTensorBlock as Block`. `VideoDepthAnythingProcessor` |
| (`processing_video_depth_anything.py`, `processor_config.json`) wraps `VideoDepthAnythingImageProcessor`, the per-frame |
| transform of `infer_video_depth`. |
|
|
| Verified against the official code (same process, V100): preprocessing, `predicted_depth` (also as a tuple), the |
| hidden states, the post-processing, `infer_video_depth` on a video longer than one window and streaming inference, in |
| fp32 and with the original's fp16 autocast, are **bit-identical** (the repo's 2 example videos; 70 frames for |
| `infer_video_depth`, 40 streamed frames). Large was compared at input size 392: at 518 these 16:9 videos need more than the V100's 32 GB (the original's README reports 26.7 GB for 518 x 518). |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from transformers import AutoModelForDepthEstimation, AutoProcessor |
| |
| model = AutoModelForDepthEstimation.from_pretrained("Video-Depth-Anything-Large", trust_remote_code=True).eval().cuda() |
| processor = AutoProcessor.from_pretrained("Video-Depth-Anything-Large", trust_remote_code=True) |
| # frames: RGB uint8 (T, H, W, 3), T <= 32 |
| inputs = processor(videos=frames, return_tensors="pt").to("cuda") |
| with torch.no_grad(): |
| outputs = model(**inputs) # outputs.predicted_depth: (1, T, H', W') relative inverse depth |
| depth = processor.post_process_depth_estimation(outputs, target_sizes=[frames.shape[1:3]])[0]["predicted_depth"] |
| |
| # a whole video of any length, exactly as the original run.py (fp16 autocast unless fp32=True): |
| depths, fps = model.infer_video_depth(frames, target_fps=30, input_size=518, device="cuda") |
| # streaming, one frame at a time (experimental upstream): model.infer_video_depth_one(frame); model.reset_streaming() |
| ``` |
|
|
| - `AutoModel` loads the same class. `output_hidden_states=True` adds the DINOv2 features the head reads; `return_dict=False` returns a tuple. |
| - `post_process_depth_estimation` resizes as the original (bilinear, `align_corners=True`). |
|
|
| Runtime packages beyond torch / transformers: `opencv-python`, `torchvision`, `einops`, `easydict`, `tqdm` (as the |
| original; `xformers` is optional, as in the original). |
|
|
| ## Licensing |
|
|
| - Weights (`depth-anything/Video-Depth-Anything-Large`): **CC-BY-NC-4.0** (non-commercial) — [`LICENSE`](LICENSE). |
| - Code (vendored files and the port): **Apache-2.0** — [`LICENSE.code`](LICENSE.code). |
| - Attribution, sources and the changes made here: [`NOTICE`](NOTICE). |
|
|
| ## Citation |
|
|
| If you use this model, please cite the original work, as its authors ask: |
|
|
| ```bibtex |
| @article{video_depth_anything, |
| title={Video Depth Anything: Consistent Depth Estimation for Super-Long Videos}, |
| author={Chen, Sili and Guo, Hengkai and Zhu, Shengnan and Zhang, Feihu and Huang, Zilong and Feng, Jiashi and Kang, Bingyi}, |
| journal={arXiv:2501.12375}, |
| year={2025} |
| } |
| ``` |
|
|
