gimmy87 commited on
Commit
f56db95
·
verified ·
1 Parent(s): aeede9c

Sync model repo (text/metadata)

Browse files
LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Fabio Tosi
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
README.md ADDED
@@ -0,0 +1,171 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: executorch
3
+ license: mit
4
+ base_model: fabiotosi92/ZipDepth
5
+ display_name: ZipDepth Base FP16 — ExecuTorch
6
+ tags:
7
+ - depth-estimation
8
+ - monocular-depth-estimation
9
+ - zipdepth
10
+ - executorch
11
+ - xnnpack
12
+ - fp16
13
+ - arm
14
+ - edge-ai
15
+ pipeline_tag: depth-estimation
16
+ ---
17
+
18
+ <!--
19
+ SPDX-FileCopyrightText: Copyright 2026 Arm Limited and/or its affiliates <open-source-office@arm.com>
20
+
21
+ SPDX-License-Identifier: Apache-2.0
22
+ -->
23
+
24
+ # ZipDepth Base FP16 — ExecuTorch
25
+
26
+ This repository packages the unfold-free ZipDepth Base mobile/NPU checkpoint as a static 384×384
27
+ ExecuTorch model. The depth network uses FP16 activations with FP32 serialized convolution weights
28
+ and external tensors.
29
+
30
+ ## ✨ Key Highlights
31
+
32
+ - **2.06× faster on Android™ Vivo X300** — one-thread p50 depth-model latency is **20.431 ms**,
33
+ compared with **41.987 ms** for FP32 under the same inference-only protocol.
34
+ - **Negligible quality change** — across all 654 NYUv2 test images, δ1 changes by only
35
+ **−0.046 percentage points** and AbsRel improves by **0.011 percentage points** relative to the
36
+ FP32 depth model under identical fixed-shape preprocessing and evaluation.
37
+
38
+ ## 📦 Model Details
39
+
40
+ ZipDepth predicts affine-invariant inverse depth from one RGB image. The package accepts arbitrary
41
+ image dimensions up to 4096×4096, crops the largest top-left square, resizes it to the static
42
+ 384×384 depth-model input, and restores the result to the original tensor shape with zero padding.
43
+
44
+ - **Developed by:** Fabio Tosi, Luca Bartolomei, Matteo Poggi, and Stefano Mattoccia
45
+ - **Model type:** Zero-shot monocular relative-depth estimation
46
+ - **License:** MIT
47
+ - **Base model:** [ZipDepth](https://github.com/fabiotosi92/ZipDepth)
48
+ - **Source revision:** [`a302e543`](https://github.com/fabiotosi92/ZipDepth/tree/a302e5437bc58f15c4efd41d3e8222bf24f7d470)
49
+ - **Source checkpoint:** `zipdepth_base_npu.pth`
50
+ - **ExecuTorch revision:** `d13c78971338b35a219e0a27f7045b29c7f4e8c1`
51
+ - **Parameters:** Approximately 6.1 million after inference-time fusion
52
+ - **Artifact size:** 23.47 MiB, 0.27% smaller than the 23.53 MiB FP32 artifact
53
+
54
+ ## 🚀 Get Started with the Model
55
+
56
+ ### 🔓 Compute Flow — Early Access
57
+
58
+ The inference engine for this model package is available through the Compute Flow Early Access
59
+ Program. Contact ai-early-access@arm.com to request access.
60
+
61
+ ## 📊 Quality evaluation
62
+
63
+ Quality was evaluated on all 654 images in the NYUv2 test split. Both models received the same
64
+ top-left square crop resized to 384×384. Their predictions were resized to the ground-truth shape,
65
+ aligned independently in inverse-depth space using a least-squares scale and shift, and evaluated
66
+ over the NYUv2 Eigen crop and 0.001–10 m depth range. This isolates the static depth models; it is
67
+ not an evaluation of the dynamic prepare/restore transforms.
68
+
69
+ | Metric | FP32 | FP16 ExecuTorch | Change |
70
+ | --- | ---: | ---: | ---: |
71
+ | AbsRel ↓ | 15.796% | **15.785%** | 0.011 percentage points lower |
72
+ | δ1 ↑ | 77.055% | 77.009% | 0.046 percentage points lower |
73
+
74
+ The FP16 candidate uses a different official ZipDepth checkpoint because the mobile/NPU checkpoint
75
+ replaces convex unfold upsampling with a deployment-friendly learned blend. The comparison
76
+ therefore measures the complete mobile graph and FP16 conversion, not FP16 rounding in isolation.
77
+ On the distributed 384×384 sample, its output has 0.99645 Pearson correlation with the FP32 output.
78
+
79
+ ## 🎯 Performance evaluation
80
+
81
+ The static depth models were profiled using the distributed `samples/im0.jpg` image, which is
82
+ already 384×384. The dynamic prepare and restore models were therefore bypassed.
83
+
84
+ - **CPU:** One Arm CPU thread.
85
+ - **Runs:** One unmeasured in-process inference to initialize each runtime, followed by 5 warmup
86
+ and 30 measured fresh-process runs with 1 second between processes.
87
+ - **Timing scope:** The setup call is not included.
88
+ - **Runtimes:** LiteRT 2.1.6 FP32 baseline and ExecuTorch `d13c7897` FP16 candidate, both with the
89
+ XNNPACK CPU backend.
90
+ - **Target:** Android™ Vivo X300 smartphone (`V2502A`) on Android™ 16, USB powered at 100% battery.
91
+
92
+ | Metric | Android™ Vivo X300: FP32 | Android™ Vivo X300: FP16 | Uplift |
93
+ | --- | ---: | ---: | ---: |
94
+ | p50 inference latency | 41.987 ms | **20.431 ms** | **2.06× faster** |
95
+ | p90 inference latency | 42.612 ms | **21.342 ms** | **2.00× faster** |
96
+ | p99 inference latency | 43.236 ms | **27.181 ms** | **1.59× faster** |
97
+
98
+ ## 🛠️ Technical Specifications
99
+
100
+ ### Runtime Architecture
101
+
102
+ | Component role | Framework / format |
103
+ | --- | --- |
104
+ | Input square crop and resize | ExecuTorch / PTE |
105
+ | ZipDepth Base inference | ExecuTorch / PTE |
106
+ | Output resize and zero padding | ExecuTorch / PTE |
107
+
108
+ ### Precision
109
+
110
+ - Internal depth-model activations use FP16.
111
+ - Public input and output tensors remain FP32 to preserve the existing brick contract.
112
+ - Dynamic image-shape transforms use FP32.
113
+ - The fixed 24×24-to-12×12 adaptive average pool is represented as an equivalent 2×2,
114
+ stride-2 average pool supported by the deployed ExecuTorch kernel set.
115
+
116
+ ### Input Specification
117
+
118
+ | Input | Data type | Shape | Value range |
119
+ | --- | --- | --- | --- |
120
+ | RGB image | FP32 | `3 × H × W`, with `1 ≤ H,W ≤ 4096` | `[0, 1]` |
121
+
122
+ ### Output Specification
123
+
124
+ | Output | Data type | Shape | Description |
125
+ | --- | --- | --- | --- |
126
+ | Relative depth | FP32 | `H × W` | Affine-invariant inverse depth; padded regions are zero |
127
+
128
+ ### Repository Contents
129
+
130
+ - `zipdepth_base_fp16.pte` — static FP16 ZipDepth Base mobile/NPU depth model.
131
+ - `zipdepth_prepare_image.pte` — dynamic input crop-and-resize transform.
132
+ - `zipdepth_restore_depth.pte` — dynamic output resize-and-padding transform.
133
+ - `zipdepth_manifest.json` — model tensor contracts and runtime configuration.
134
+ - `samples/im0.jpg` — 384×384 upstream sample used for profiling.
135
+ - `metadata.yaml` and `benchmarks/` — model and benchmark metadata.
136
+ - `LICENSE` — upstream MIT license.
137
+ - `SHA256SUMS` — reproducibility checksums.
138
+
139
+ ## ⚠️ Known Limitations
140
+
141
+ - The square-crop package pipeline does not preserve the aspect-ratio preprocessing used for
142
+ ZipDepth's published benchmark results.
143
+ - The output is affine-invariant relative depth, not metric depth.
144
+ - The model has no temporal-consistency mechanism and may flicker when applied independently to
145
+ video frames.
146
+
147
+ ## 🗂️ Model and Asset Origin
148
+
149
+ - [`zipdepth_base_npu.pth`](https://github.com/fabiotosi92/ZipDepth/tree/a302e5437bc58f15c4efd41d3e8222bf24f7d470/checkpoints)
150
+ is the source checkpoint for `zipdepth_base_fp16.pte`.
151
+ - [`samples/im0.jpg`](https://github.com/fabiotosi92/ZipDepth/blob/a302e5437bc58f15c4efd41d3e8222bf24f7d470/assets/examples/im0.jpg)
152
+ is an upstream example crop.
153
+
154
+ ## 🔐 Checksums
155
+
156
+ Verify the checked-out files with:
157
+
158
+ ```sh
159
+ shasum -a 256 -c SHA256SUMS
160
+ ```
161
+
162
+ ## Citation
163
+
164
+ ```bibtex
165
+ @inproceedings{tosi2026zipdepth,
166
+ title = {{ZipDepth}: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device},
167
+ author = {Tosi, Fabio and Bartolomei, Luca and Poggi, Matteo and Mattoccia, Stefano},
168
+ booktitle = {European Conference on Computer Vision (ECCV)},
169
+ year = {2026}
170
+ }
171
+ ```
SHA256SUMS ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ 0007e2ff761f1b89ad89327870b807cd4de00cb657b442b62de2f92fdc87d508 LICENSE
2
+ 3e2361de413265dcef61f7758a29297776d8ecaa72d7041917f033d46c05cba0 README.md
3
+ 0ae0b8c374363c6b82434632143b1c0dfbf171d0cdf779f71c597ceeb3abd34e benchmarks/zipdepth-base-executorch-fp16-vivo-x300-baseline.yaml
4
+ 47ad0825549f7e8fa6b4424e595a801437cf3d1be53ee37e3beaf6d18083f670 benchmarks/zipdepth-base-executorch-fp16-vivo-x300.yaml
5
+ 47f267cad4bb732b7672c8237e6de1ecd7d53b24259c280dac7bc929fa4744f7 metadata.yaml
6
+ ac960f4f6fecc05ce47411ad030974510138e209aef370662aa4d81fc26decde samples/im0.jpg
7
+ 33f72f8c10ae5449872816e77f7e84f055a35c55588fbfbd1b41dd633e146a6f zipdepth_base_fp16.pte
8
+ 16b6bec7550ed15d10ede2dd456e8d7b7fc61d07019eddf5cf0d25fd2219cf42 zipdepth_manifest.json
9
+ b59e2cf4db36efa2d7cdfd3695f000bd8e1b36bf551f5eb4bc0971db505d5997 zipdepth_prepare_image.pte
10
+ 1e37a26736d90ac80a269e77f8a58e232cdb5311f4c504bc387828866b3af78b zipdepth_restore_depth.pte
benchmarks/zipdepth-base-executorch-fp16-vivo-x300-baseline.yaml ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SPDX-FileCopyrightText: Copyright 2026 Arm Limited and/or its affiliates <open-source-office@arm.com>
2
+ # SPDX-License-Identifier: Apache-2.0
3
+
4
+ version: 1.2.0
5
+ report_type: image-depth-estimation
6
+ profile: Baseline
7
+ created_at: '2026-09-17T11:11:10Z'
8
+ context:
9
+ target_ref: vivo-x300
10
+ runtime:
11
+ name: litert
12
+ execution_backend: cpu
13
+ config:
14
+ n_threads: 1
15
+ optimisations:
16
+ - XNNPACK
17
+ version: 2.1.6
18
+ dataset:
19
+ name: NYU Depth V2
20
+ version: V2
21
+ slice: labeled test split (Eigen crop)
22
+ sample_count: 654
23
+ reference_url: https://cs.nyu.edu/~fergus/datasets/nyu_depth_v2.html
24
+ benchmark:
25
+ batch_size: 1
26
+ image_width: 384
27
+ image_height: 384
28
+ num_runs: 30
29
+ warmup_runs: 5
30
+ mode: offline
31
+ pipeline_scope: depth-model-only
32
+ performance:
33
+ end_to_end_latency_ms:
34
+ p50: 41.987
35
+ p90: 42.612
36
+ p99: 43.236
37
+ throughput_images_per_second: 23.8167
38
+ accuracy:
39
+ delta1: 0.770555
40
+ absolute_relative_error: 0.157965
41
+ alignment: per-image least-squares scale and shift in disparity space
42
+ valid_depth_range_m:
43
+ minimum: 0.001
44
+ maximum: 10.0
45
+ sample_count: 654
benchmarks/zipdepth-base-executorch-fp16-vivo-x300.yaml ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SPDX-FileCopyrightText: Copyright 2026 Arm Limited and/or its affiliates <open-source-office@arm.com>
2
+ # SPDX-License-Identifier: Apache-2.0
3
+
4
+ version: 1.2.0
5
+ report_type: image-depth-estimation
6
+ profile: Arm-Optimized
7
+ created_at: '2026-09-17T11:10:17Z'
8
+ context:
9
+ target_ref: vivo-x300
10
+ runtime:
11
+ name: executorch
12
+ execution_backend: cpu
13
+ config:
14
+ n_threads: 1
15
+ optimisations:
16
+ - XNNPACK
17
+ version: d13c78971338b35a219e0a27f7045b29c7f4e8c1
18
+ dataset:
19
+ name: NYU Depth V2
20
+ version: V2
21
+ slice: labeled test split (Eigen crop)
22
+ sample_count: 654
23
+ reference_url: https://cs.nyu.edu/~fergus/datasets/nyu_depth_v2.html
24
+ benchmark:
25
+ batch_size: 1
26
+ image_width: 384
27
+ image_height: 384
28
+ num_runs: 30
29
+ warmup_runs: 5
30
+ mode: offline
31
+ pipeline_scope: depth-model-only
32
+ performance:
33
+ end_to_end_latency_ms:
34
+ p50: 20.431
35
+ p90: 21.342
36
+ p99: 27.181
37
+ throughput_images_per_second: 48.9444
38
+ accuracy:
39
+ delta1: 0.770092
40
+ absolute_relative_error: 0.157853
41
+ alignment: per-image least-squares scale and shift in disparity space
42
+ valid_depth_range_m:
43
+ minimum: 0.001
44
+ maximum: 10.0
45
+ sample_count: 654
metadata.yaml ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SPDX-FileCopyrightText: Copyright 2026 Arm Limited and/or its affiliates <open-source-office@arm.com>
2
+ # SPDX-License-Identifier: Apache-2.0
3
+
4
+ schema_version: 1.0.0
5
+ report_type: image-depth-estimation
6
+ task_type: monocular-depth-estimation
7
+ title: ZipDepth FP16 ExecuTorch — Compute Flow
8
+ description: >-
9
+ Monocular relative-depth package derived from the unfold-free ZipDepth Base mobile/NPU
10
+ checkpoint. The package uses a static ExecuTorch depth model with FP32 tensor interfaces,
11
+ FP16 activations, FP32 serialized convolution weights, and dynamic ExecuTorch transforms to
12
+ preserve the source image shape at its boundaries.
13
+ family_name: ZipDepth
14
+ id: Arm/zipdepth-fp16-executorch
15
+ filename: zipdepth_base_fp16.pte
16
+ base_model_id: fabiotosi92/ZipDepth
17
+ vendor: University of Bologna
18
+ base_model_url: https://github.com/fabiotosi92/ZipDepth
19
+ weight_dtype: fp32
20
+ profile: Arm-Optimized
21
+ model_size_mb: 24.609984
22
+ parameter_count: 6142058
23
+ format: executorch
samples/im0.jpg ADDED
zipdepth_manifest.json ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "brick": {
3
+ "type": "monocular_depth_estimation",
4
+ "name": "ZipDepth",
5
+ "version": "1.0"
6
+ },
7
+ "models": {
8
+ "resize_input": {
9
+ "runner": "executorch",
10
+ "filename": "zipdepth_prepare_image.pte",
11
+ "inputs": {
12
+ "image": {
13
+ "index": 0,
14
+ "dtype": "float32",
15
+ "shape": [1, 3, -1, -1]
16
+ }
17
+ },
18
+ "outputs": {
19
+ "image": {
20
+ "index": 0,
21
+ "dtype": "float32",
22
+ "shape": [1, 3, 384, 384]
23
+ }
24
+ },
25
+ "runner_params": {}
26
+ },
27
+ "zip_depth": {
28
+ "runner": "executorch",
29
+ "filename": "zipdepth_base_fp16.pte",
30
+ "inputs": {
31
+ "image": {
32
+ "index": 0,
33
+ "dtype": "float32",
34
+ "shape": [1, 3, 384, 384]
35
+ }
36
+ },
37
+ "outputs": {
38
+ "relative_depth": {
39
+ "index": 0,
40
+ "dtype": "float32",
41
+ "shape": [1, 1, 384, 384]
42
+ }
43
+ },
44
+ "runner_params": {}
45
+ },
46
+ "resize_output": {
47
+ "runner": "executorch",
48
+ "filename": "zipdepth_restore_depth.pte",
49
+ "inputs": {
50
+ "relative_depth": {
51
+ "index": 0,
52
+ "dtype": "float32",
53
+ "shape": [1, 1, 384, 384]
54
+ },
55
+ "image": {
56
+ "index": 1,
57
+ "dtype": "float32",
58
+ "shape": [1, 3, -1, -1]
59
+ }
60
+ },
61
+ "outputs": {
62
+ "relative_depth": {
63
+ "index": 0,
64
+ "dtype": "float32",
65
+ "shape": [1, 1, -1, -1]
66
+ }
67
+ },
68
+ "runner_params": {}
69
+ }
70
+ },
71
+ "constants": {
72
+ "maximum_dynamic_dimension": {
73
+ "type": "int64",
74
+ "value": 4096
75
+ }
76
+ }
77
+ }