EfficientViT-Depth-5M

A 4.85M-parameter relative-depth model with an EfficientViT-M2 encoder and a Lite-DPT decoder, trained with Depth Anything V2 Small as a teacher.

This is an independent community model, not an official Depth Anything release, and it is not a fine-tune, derivative, or pruned version of Depth Anything V2. No Depth Anything weights, layers, or architecture are inherited. The architecture is deliberately different: a compact CNN encoder with a lightweight DPT-style decoder, aimed at small size and low CPU latency. Depth Anything V2 Small is used only to generate distillation targets.

Float vs INT8 size/quality comparison.

NYUv2 indoor scene; float vs INT8 at 336×448.

Qualitative examples on ADE20K/NYUv2 scenes; per-row color scale.

Highlights

• 4,854,203 parameters, 9.84 MB fp32 ONNX (static 336×448)
• 0.8546 NYU-654 δ1 at 336×448 (inverse-depth protocol, affine alignment)
• 0.7242 teacher-fidelity δ1 on 700 held-out images
• 12.1 ms / 82.5 FPS on CPU with ONNX Runtime, 8 threads, 336×448
• Dynamic-resolution ONNX graph included
• INT8 sibling: EfficientViT-Depth-5M-int8 — 3.48 MB, 0.8319 NYU δ1

Comparison

All quality numbers are at 336×448 on the 654-image NYUv2 Eigen test split. “Teacher fidelity” is agreement with the original fp32 Depth Anything V2 Small teacher on held-out distillation data.

model	params	disk	NYU-654 δ1	teacher fidelity δ1	note
Depth Anything V2 Small	24.79M	99.06 MB	0.9213	1.000	original teacher
Depth Anything V2 Tiny (8-layer)	16.50M	66.04 MB	0.8936	0.8976	pruned DA-V2 transformer
EfficientViT-Depth-5M (float)	4.85M	9.84 MB	0.8546	0.7242	this model
EfficientViT-Depth-5M (INT8)	4.85M	3.48 MB	0.8319	0.6729	INT8 repo
EfficientViT-Depth-5M (4-bit, experimental)	4.85M	2.31 MB	0.6456	0.4260	not shipped; conv 4-bit QDQ collapses

Relative to Depth Anything V2 Small, this model uses 80.4% fewer parameters and is 90.1% smaller in fp32 ONNX, while retaining 92.8% of its NYU-654 δ1. Relative to the 8-layer Tiny, it uses 70.6% fewer parameters and is 85.1% smaller, retaining 95.6% of its NYU-654 δ1. Teacher fidelity is lower than both, so it is a size/latency-oriented model rather than a drop-in replacement for their detail.

Architecture

component	detail
encoder	EfficientViT-M2, ImageNet-pretrained, 3.96M params
decoder	Lite-DPT: 1×1 laterals, stride-32 ASPP, top-down fusion, depthwise-separable residual blocks (2 per level)
decoder channels	80 / 112 / 144 / 176
ASPP hidden	64
output	affine-invariant inverse depth, higher = closer
input	RGB, 336×448, ImageNet normalization
inference params	4,854,203

Deep supervision at stride 8 and 16 is used only during training. The released float checkpoint and ONNX graph are the clean inference model without auxiliary heads.

The network pads the input internally to a multiple of 32, runs the encoder/decoder, and bilinearly resizes the depth map back to the original input size.

Metrics

NYUv2 Eigen-654, 336×448

metric	value
δ1	0.854563
AbsRel	0.126748
RMSE	0.079416

The protocol is inverse-depth space with per-image affine scale and shift alignment. The model predicts inverse depth; the ground-truth metric depth is inverted before alignment.

Resolution sweep (PyTorch, NYU-654)

resolution	δ1	AbsRel	RMSE
224×294	0.632913	0.238141	0.129551
252×336	0.805205	0.153038	0.092275
336×448	0.853374	0.127079	0.079458
448×602	0.812116	0.147902	0.089518
518×686	0.730083	0.193571	0.115353

336×448 is the primary deployment point and the clear sweet spot. 224×294 is substantially weaker; use 252×336 or higher when possible.

Teacher fidelity

Scale-and-shift aligned inverse-depth agreement with the fp32 Depth Anything V2 Small teacher:

eval set	δ1	AbsRel	Pearson	gradient corr
700 held-out images	0.724169	0.313551	0.914005	0.379217
2,200 held-out images	0.681209	0.395252	0.896204	0.344748

The gap to the teacher is concentrated in fine structure and high-frequency detail, not overall layout.

Usage

ONNX Runtime

pip install -r requirements.txt
python example.py --image photo.jpg --output depth.png

The example resizes the RGB image to exactly 336×448, applies ImageNet normalization, runs the ONNX graph, and writes a normalized grayscale preview plus optional raw .npy inverse depth.

Minimal code:

import numpy as np
from PIL import Image
import onnxruntime as ort

mean = np.array([0.485, 0.456, 0.406], dtype=np.float32)
std = np.array([0.229, 0.224, 0.225], dtype=np.float32)

image = Image.open("photo.jpg").convert("RGB").resize((448, 336), Image.Resampling.BICUBIC)
x = np.asarray(image, dtype=np.float32) / 255.0
x = (x - mean) / std
x = np.transpose(x, (2, 0, 1))[None].astype(np.float32)

session = ort.InferenceSession("efficientvit_depth_5m_fp32.onnx", providers=["CPUExecutionProvider"])
depth = session.run(None, {session.get_inputs()[0].name: x})[0][0]
# depth is inverse depth: higher = closer, not metric depth

PyTorch

import torch
from efficientvit_depth_5m import load_compact

model = load_compact("efficientvit_depth_5m_fp32.pt").eval()
x = torch.randn(1, 3, 336, 448)
with torch.no_grad():
    depth = model(pixel_values=x).predicted_depth

efficientvit_depth_5m.py is the full model definition. It requires torch and timm.

Dynamic resolution

efficientvit_depth_5m_fp32_dynamic.onnx accepts dynamic height and width. It was verified at 252×336, 336×448, and 448×602. The network pads internally to a multiple of 32. For best quality, use 336×448.

Output interpretation

The output is affine-invariant inverse depth, not metric depth. To compare with metric depth z:

1. Compute 1 / z.
2. Fit a per-image scale and shift between prediction and 1 / z.
3. Report error after alignment.

For visualization, normalize per image. Do not interpret the raw values as meters.

Training

• Teacher: original fp32 depth-anything/Depth-Anything-V2-Small-hf
• Data: 23,805 training images from COCO, ADE20K, and NYUv2 train; 2,200 held-out validation images; NYUv2 Eigen-654 test kept separate
• Schedule: 224×294 / 252×336 / 336×448 multi-resolution training, then a deep-supervision fine-tune
• Loss: scale-shift-invariant L1 on inverse depth + gradient L1; auxiliary stride-8/16 heads during the deep-supervision stage
• Optimizer: AdamW, cosine schedule, EMA weights
• Encoder initialization: ImageNet-pretrained EfficientViT-M2, then distillation

No NYUv2 test images were used for training or model selection.

Limitations

• Relative inverse depth only; no metric scale.
• 336×448 is the intended resolution. 224×294 is much weaker, and 448/518 degrade without resolution-specific fine-tuning.
• Fine structures, thin objects, clutter, transparent/reflective surfaces, and extreme depth ranges can be inaccurate.
• Teacher fidelity is lower than Depth Anything V2 Small and the 8-layer Tiny; this model trades detail for size and CPU speed.
• No temporal or multi-frame consistency.
• Not an official Depth Anything release and not affiliated with its authors.

Files

file	description
efficientvit_depth_5m_fp32.onnx	static 336×448 float ONNX
efficientvit_depth_5m_fp32_dynamic.onnx	dynamic-resolution float ONNX
efficientvit_depth_5m_fp32.pt	clean PyTorch checkpoint (state_dict, config, params)
model.safetensors	safetensors copy of the 4.85M model parameters (BatchNorm buffers are in the .pt)
efficientvit_depth_5m.py	PyTorch model definition and load_compact
config.json	architecture and I/O metadata
preprocessor_config.json	resize/normalization metadata
metrics.json	full metrics and artifact hashes
example.py	ONNX Runtime example
comparison.png	quality and size comparison
example.png	qualitative float vs INT8 example
examples.png	five-scene qualitative gallery
checksums.txt	SHA256 checksums

License

Apache-2.0. See LICENSE and NOTICE. The teacher, Depth Anything V2 Small, is also Apache-2.0.

Contact

Petros Fanioudakis — @Petrouil