StarNew commited on
Commit
28c3e6e
Β·
verified Β·
1 Parent(s): 87e736d

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +179 -1
README.md CHANGED
@@ -1,3 +1,181 @@
1
  ---
2
- license: agpl-3.0
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: mit
3
+ tags:
4
+ - monocular-depth-estimation
5
+ - self-supervised
6
+ - autonomous-driving
7
+ - yolo
8
+ - pytorch
9
+ - kitti
10
+ - cityscapes
11
+ library_name: pytorch
12
+ pipeline_tag: depth-estimation
13
  ---
14
+
15
+ # FlexDepth: A Flexible Scale-Driven Family for Self-Supervised Monocular Depth Estimation
16
+
17
+ **Towards Robust Driving Perception: A Flexible Scale-Driven Family for Self-Supervised Monocular Depth Estimation**
18
+
19
+ [![ECCV 2026](https://img.shields.io/badge/ECCV-2026-4F70F2?style=flat-square)](https://eccv.ecva.net/)
20
+ [![arXiv](https://img.shields.io/badge/arXiv-2607.00736-b31b1b?style=flat-square)](https://arxiv.org/abs/2607.00736)
21
+ [![Project Page](https://img.shields.io/badge/Project-Page-4F70F2?style=flat-square)](https://startnew.github.io/projects/flexdepth/)
22
+ [![Code](https://img.shields.io/badge/GitHub-Code-181717?style=flat-square)](https://github.com/StarNew/FlexDepth)
23
+
24
+ ## Overview
25
+
26
+ FlexDepth is a family of self-supervised monocular depth estimation models designed for robust driving perception. It introduces a **Scale-Driven Decoder (SDD)** with adaptive component selection, enabling a single architecture to span five model scales β€” from ultra-lightweight (1.5M params, 0.7 GFLOPs) to high-accuracy (32.3M params, 24.6 GFLOPs).
27
+
28
+ ### Key Features
29
+
30
+ - **Five model scales**: Nano (N), Small (S), Medium (M), Large (L), X-Large (X)
31
+ - **Scale-Driven Decoder (SDD)**: Adaptive component selection based on model scale
32
+ - **High-Efficiency Bottleneck (HEB)**: For small models (N, S) β€” maximizes efficiency
33
+ - **High-Performance Bottleneck (HPB)**: For large models (M, L, X) β€” maximizes accuracy
34
+ - **Dynamic upsampling**: Sharper depth boundaries via learned upsampling
35
+ - **Two-stage static-dynamic decoupled training**: Handles dynamic scenes in driving scenarios
36
+ - **YOLO11-based encoder**: Leverages rich visual representations from YOLO segmentation pretraining
37
+
38
+ ## Model Zoo
39
+
40
+ ### KITTI-trained Models
41
+
42
+ | Model | Params | GFLOPs | Abs Rel ↓ | Sq Rel ↓ | RMSE ↓ | RMSE log ↓ | Ξ΄<1.25 ↑ | Ξ΄<1.25Β² ↑ | Ξ΄<1.25Β³ ↑ |
43
+ |-------|--------|--------|-----------|----------|--------|------------|----------|-----------|-----------|
44
+ | Flex-Nano | 1.5M | 0.7 | 0.110 | 0.794 | 4.678 | 0.184 | 0.878 | 0.961 | 0.983 |
45
+ | Flex-Small | 6.1M | 2.8 | 0.104 | 0.713 | 4.458 | 0.179 | 0.890 | 0.964 | 0.983 |
46
+ | Flex-Medium | 12.7M | 10.0 | 0.096 | 0.639 | 4.253 | 0.172 | 0.903 | 0.968 | 0.985 |
47
+ | Flex-Large | 15.2M | 11.5 | 0.095 | 0.642 | 4.199 | 0.171 | 0.906 | 0.968 | 0.984 |
48
+ | Flex-X-Large | 32.3M | 24.6 | **0.093** | **0.605** | **4.114** | **0.167** | **0.910** | **0.969** | **0.985** |
49
+
50
+ ### Cityscapes-trained Models
51
+
52
+ | Model | Params | GFLOPs | Abs Rel ↓ | Sq Rel ↓ | RMSE ↓ | RMSE log ↓ | Ξ΄<1.25 ↑ | Ξ΄<1.25Β² ↑ | Ξ΄<1.25Β³ ↑ |
53
+ |-------|--------|--------|-----------|----------|--------|------------|----------|-----------|-----------|
54
+ | Flex-Nano | 1.5M | 0.6 | 0.107 | 1.261 | 6.133 | 0.164 | 0.893 | 0.971 | 0.989 |
55
+ | Flex-Small | 6.1M | 2.2 | 0.100 | 1.078 | 5.813 | 0.153 | 0.904 | 0.975 | 0.991 |
56
+ | Flex-Medium | 12.7M | 8.0 | 0.089 | 0.885 | 5.358 | 0.143 | 0.917 | 0.979 | 0.993 |
57
+ | Flex-Large | 15.2M | 9.2 | 0.087 | 0.911 | 5.310 | 0.139 | 0.924 | 0.981 | 0.993 |
58
+ | Flex-X-Large | 32.3M | 19.7 | **0.086** | **0.877** | **5.268** | **0.137** | **0.926** | **0.982** | **0.993** |
59
+
60
+ ### Efficiency
61
+
62
+ | Model | FPS (Snapdragon 8 Elite) | FPS (RTX 2080 Ti) |
63
+ |-------|--------------------------|---------------------|
64
+ | Flex-Nano | 37.6 | 180+ |
65
+ | Flex-Small | 18.6 | 120+ |
66
+ | Flex-Medium | 5.8 | 60+ |
67
+ | Flex-Large | 5.2 | 55+ |
68
+ | Flex-X-Large | 3.0 | 40+ |
69
+
70
+ ## Model Files
71
+
72
+ Each model consists of two weight files:
73
+
74
+ ```
75
+ β”œβ”€β”€ kitti/
76
+ β”‚ β”œβ”€β”€ flex_n/
77
+ β”‚ β”‚ β”œβ”€β”€ encoder.pth # YOLO11-based encoder weights
78
+ β”‚ β”‚ └── depth.pth # Scale-Driven Decoder weights
79
+ β”‚ β”œβ”€β”€ flex_s/
80
+ β”‚ β”œβ”€β”€ flex_m/
81
+ β”‚ β”œβ”€β”€ flex_l/
82
+ β”‚ └── flex_x/
83
+ └── cs/
84
+ β”œβ”€β”€ flex_n/
85
+ β”œβ”€β”€ flex_s/
86
+ β”œβ”€β”€ flex_m/
87
+ β”œβ”€β”€ flex_l/
88
+ └── flex_x/
89
+ ```
90
+
91
+ ## Usage
92
+
93
+ ### Installation
94
+
95
+ ```bash
96
+ conda create -n flexdepth python=3.10
97
+ conda activate flexdepth
98
+
99
+ pip install torch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu118
100
+ pip install -r requirements.txt
101
+ ```
102
+
103
+ ### Download Weights
104
+
105
+ ```python
106
+ from huggingface_hub import snapshot_download
107
+
108
+ # Download all models
109
+ snapshot_download(repo_id="StarNew/flexdepth", local_dir="./models")
110
+
111
+ # Or download a specific model
112
+ from huggingface_hub import hf_hub_download
113
+
114
+ hf_hub_download(
115
+ repo_id="StarNew/flexdepth",
116
+ filename="kitti/flex_n/encoder.pth",
117
+ local_dir="./models"
118
+ )
119
+ hf_hub_download(
120
+ repo_id="StarNew/flexdepth",
121
+ filename="kitti/flex_n/depth.pth",
122
+ local_dir="./models"
123
+ )
124
+ ```
125
+
126
+ ### Evaluation
127
+
128
+ ```bash
129
+ # Flex-Nano on KITTI
130
+ python evaluate_depth.py --png --eval_mono --scale 4 \
131
+ --encoder_model_type yolo11n-seg --decoder_model_type flexn \
132
+ --load_weights_folder ./models/kitti/flex_n \
133
+ --data_path <kitti_data_path> --split_path <splits_path>
134
+
135
+ # Flex-X-Large on KITTI
136
+ python evaluate_depth.py --png --eval_mono --scale 4 \
137
+ --encoder_model_type yolo11x-seg --decoder_model_type flexx \
138
+ --load_weights_folder ./models/kitti/flex_x \
139
+ --data_path <kitti_data_path> --split_path <splits_path>
140
+ ```
141
+
142
+ ### ONNX Export
143
+
144
+ ```bash
145
+ python export_onnx.py --encoder_model_type yolo11n-seg --decoder_model_type flexn \
146
+ --load_weights_folder ./models/kitti/flex_n --scales 4 --export_name flex-n
147
+ ```
148
+
149
+ ## Comparison with Depth Anything V2
150
+
151
+ On the KITTI Eigen benchmark with dense ground truth and least-squares alignment:
152
+
153
+ | Method | Type | Params | GFLOPs | Resolution | Abs Rel ↓ | Ξ΄<1.25 ↑ |
154
+ |--------|------|--------|--------|------------|-----------|----------|
155
+ | DA2 (ViT-L) | Zero-Shot | 335M | 1947 | 1722Γ—518 | 0.070 | **0.956** |
156
+ | DA2 (ViT-S) | Zero-Shot | 25M | 137 | 1722Γ—518 | 0.077 | 0.944 |
157
+ | DA2 (ViT-L) | Zero-Shot | 335M | 276 | 644Γ—196 | 0.092 | 0.915 |
158
+ | DA2 (ViT-S) | Zero-Shot | 25M | 19 | 644Γ—196 | 0.110 | 0.881 |
159
+ | **Flex-X-Large** | Self-Supervised | 32M | 25 | 640Γ—192 | **0.063** | 0.952 |
160
+
161
+ FlexDepth achieves comparable or better accuracy than Depth Anything V2 with **~13Γ— fewer parameters** and **~78Γ— fewer GFLOPs** at similar resolution.
162
+
163
+ ## Citation
164
+
165
+ ```bibtex
166
+ @misc{zhu2026robustdrivingperceptionflexible,
167
+ title={Towards Robust Driving Perception: A Flexible Scale-Driven Family for Self-Supervised Monocular Depth Estimation},
168
+ author={Zhaowen Zhu and Li Zhang and Yujie Chen and Tian Zhang and Yingjie Wang and Mingxia Zhan},
169
+ year={2026},
170
+ eprint={2607.00736},
171
+ archivePrefix={arXiv},
172
+ primaryClass={cs.CV},
173
+ url={https://arxiv.org/abs/2607.00736}
174
+ }
175
+ ```
176
+
177
+ ## Acknowledgment
178
+
179
+ This work is supported by the National Natural Science Foundation of China under Grant 62332016.
180
+
181
+ Our code is built upon [Monodepth2](https://github.com/nianticlabs/monodepth2), [Manydepth](https://github.com/nianticlabs/manydepth), and [Ultralytics](https://github.com/ultralytics/ultralytics).