Metric-Video-Depth-Anything-Base — transformers format (original code)

Unofficial transformers port of Video Depth Anything Metric Base (depth-anything/Metric-Video-Depth-Anything-Base, metric_video_depth_anything_vitb.pth) that runs the original code. Loads with trust_remote_code. model.safetensors holds the checkpoint's tensors under their original names, converted from .pth (every tensor checked equal).

Not an official release. Video Depth Anything was created by Sili Chen, Hengkai Guo, Shengnan Zhu, Feihu Zhang, Zilong Huang, Jiashi Feng and Bingyi Kang (ByteDance) — https://github.com/DepthAnything/Video-Depth-Anything, paper "Video Depth Anything: Consistent Depth Estimation for Super-Long Videos" (arXiv 2501.12375).

What it is

VideoDepthAnythingForDepthEstimation holds the official VideoDepthAnything's submodules (pretrained: DINOv2 ViT-B/14, head: the temporal DPT head with motion modules) and computes depth with that class's own forward, returning the standard DepthEstimatorOutput (predicted_depth: (B, T, H, W), up to 32 frames per clip). The official whole-video inference infer_video_depth (32-frame windows with key-frame overlap and scale / shift alignment) and the experimental streaming inference infer_video_depth_one run unchanged on the model. The metric checkpoint (config.metric=True) skips the scale / shift fitting between windows, as the original. Code vendored from DepthAnything/Video-Depth-Anything @ 4f5ae231 into vda_*.py; edits: imports, and Block → NestedTensorBlock where the original imported NestedTensorBlock as Block. VideoDepthAnythingProcessor (processing_video_depth_anything.py, processor_config.json) wraps VideoDepthAnythingImageProcessor, the per-frame transform of infer_video_depth.

Verified against the official code (same process, V100): preprocessing, predicted_depth (also as a tuple), the hidden states, the post-processing, infer_video_depth on a video longer than one window and streaming inference, in fp32 and with the original's fp16 autocast, are bit-identical (the repo's 2 example videos; 70 frames for infer_video_depth, 40 streamed frames).

Usage

import torch
from transformers import AutoModelForDepthEstimation, AutoProcessor

model = AutoModelForDepthEstimation.from_pretrained("Metric-Video-Depth-Anything-Base", trust_remote_code=True).eval().cuda()
processor = AutoProcessor.from_pretrained("Metric-Video-Depth-Anything-Base", trust_remote_code=True)
# frames: RGB uint8 (T, H, W, 3), T <= 32
inputs = processor(videos=frames, return_tensors="pt").to("cuda")
with torch.no_grad():
    outputs = model(**inputs)                       # outputs.predicted_depth: (1, T, H', W') metric depth in meters
depth = processor.post_process_depth_estimation(outputs, target_sizes=[frames.shape[1:3]])[0]["predicted_depth"]

# a whole video of any length, exactly as the original run.py (fp16 autocast unless fp32=True):
depths, fps = model.infer_video_depth(frames, target_fps=30, input_size=518, device="cuda")
# streaming, one frame at a time (experimental upstream): model.infer_video_depth_one(frame); model.reset_streaming()

• AutoModel loads the same class. output_hidden_states=True adds the DINOv2 features the head reads; return_dict=False returns a tuple.
• post_process_depth_estimation resizes as the original (bilinear, align_corners=True).

Runtime packages beyond torch / transformers: opencv-python, torchvision, einops, easydict, tqdm (as the original; xformers is optional, as in the original).

Licensing

• Weights (depth-anything/Metric-Video-Depth-Anything-Base): CC-BY-NC-4.0 (non-commercial) — LICENSE.
• Code (vendored files and the port): Apache-2.0 — LICENSE.code.
• Attribution, sources and the changes made here: NOTICE.

Citation

If you use this model, please cite the original work, as its authors ask:

@article{video_depth_anything,
  title={Video Depth Anything: Consistent Depth Estimation for Super-Long Videos},
  author={Chen, Sili and Guo, Hengkai and Zhu, Shengnan and Zhang, Feihu and Huang, Zilong and Feng, Jiashi and Kang, Bingyi},
  journal={arXiv:2501.12375},
  year={2025}
}