| --- |
| library_name: executorch |
| license: mit |
| base_model: fabiotosi92/ZipDepth |
| display_name: ZipDepth Base FP16 — ExecuTorch |
| tags: |
| - depth-estimation |
| - monocular-depth-estimation |
| - zipdepth |
| - executorch |
| - xnnpack |
| - fp16 |
| - arm |
| - edge-ai |
| pipeline_tag: depth-estimation |
| --- |
| |
| <!-- |
| SPDX-FileCopyrightText: Copyright 2026 Arm Limited and/or its affiliates <open-source-office@arm.com> |
| |
| SPDX-License-Identifier: Apache-2.0 |
| --> |
| |
| # ZipDepth Base FP16 — ExecuTorch |
|
|
| This repository packages the unfold-free ZipDepth Base mobile/NPU checkpoint as a static 384×384 |
| ExecuTorch model. The depth network uses FP16 activations with FP32 serialized convolution weights |
| and external tensors. |
|
|
| ## ✨ Key Highlights |
|
|
| - **2.06× faster on Android™ Vivo X300** — one-thread p50 depth-model latency is **20.431 ms**, |
| compared with **41.987 ms** for FP32 under the same inference-only protocol. |
| - **Negligible quality change** — across all 654 NYUv2 test images, δ1 changes by only |
| **−0.046 percentage points** and AbsRel improves by **0.011 percentage points** relative to the |
| FP32 depth model under identical fixed-shape preprocessing and evaluation. |
|
|
| ## 📦 Model Details |
|
|
| ZipDepth predicts affine-invariant inverse depth from one RGB image. The package accepts arbitrary |
| image dimensions up to 4096×4096, crops the largest top-left square, resizes it to the static |
| 384×384 depth-model input, and restores the result to the original tensor shape with zero padding. |
|
|
| - **Developed by:** Fabio Tosi, Luca Bartolomei, Matteo Poggi, and Stefano Mattoccia |
| - **Model type:** Zero-shot monocular relative-depth estimation |
| - **License:** MIT |
| - **Base model:** [ZipDepth](https://github.com/fabiotosi92/ZipDepth) |
| - **Source revision:** [`a302e543`](https://github.com/fabiotosi92/ZipDepth/tree/a302e5437bc58f15c4efd41d3e8222bf24f7d470) |
| - **Source checkpoint:** `zipdepth_base_npu.pth` |
| - **ExecuTorch revision:** `d13c78971338b35a219e0a27f7045b29c7f4e8c1` |
| - **Parameters:** Approximately 6.1 million after inference-time fusion |
| - **Artifact size:** 23.47 MiB, 0.27% smaller than the 23.53 MiB FP32 artifact |
|
|
| ## 🚀 Get Started with the Model |
|
|
| ### 🔓 Compute Flow — Early Access |
|
|
| The inference engine for this model package is available through the Compute Flow Early Access |
| Program. Contact ai-early-access@arm.com to request access. |
|
|
| ## 📊 Quality evaluation |
|
|
| Quality was evaluated on all 654 images in the NYUv2 test split. Both models received the same |
| top-left square crop resized to 384×384. Their predictions were resized to the ground-truth shape, |
| aligned independently in inverse-depth space using a least-squares scale and shift, and evaluated |
| over the NYUv2 Eigen crop and 0.001–10 m depth range. This isolates the static depth models; it is |
| not an evaluation of the dynamic prepare/restore transforms. |
|
|
| | Metric | FP32 | FP16 ExecuTorch | Change | |
| | --- | ---: | ---: | ---: | |
| | AbsRel ↓ | 15.796% | **15.785%** | 0.011 percentage points lower | |
| | δ1 ↑ | 77.055% | 77.009% | 0.046 percentage points lower | |
|
|
| The FP16 candidate uses a different official ZipDepth checkpoint because the mobile/NPU checkpoint |
| replaces convex unfold upsampling with a deployment-friendly learned blend. The comparison |
| therefore measures the complete mobile graph and FP16 conversion, not FP16 rounding in isolation. |
| On the distributed 384×384 sample, its output has 0.99645 Pearson correlation with the FP32 output. |
|
|
| ## 🎯 Performance evaluation |
|
|
| The static depth models were profiled using the distributed `samples/im0.jpg` image, which is |
| already 384×384. The dynamic prepare and restore models were therefore bypassed. |
|
|
| - **CPU:** One Arm CPU thread. |
| - **Runs:** One unmeasured in-process inference to initialize each runtime, followed by 5 warmup |
| and 30 measured fresh-process runs with 1 second between processes. |
| - **Timing scope:** The setup call is not included. |
| - **Runtimes:** LiteRT 2.1.6 FP32 baseline and ExecuTorch `d13c7897` FP16 candidate, both with the |
| XNNPACK CPU backend. |
| - **Target:** Android™ Vivo X300 smartphone (`V2502A`) on Android™ 16, USB powered at 100% battery. |
|
|
| | Metric | Android™ Vivo X300: FP32 | Android™ Vivo X300: FP16 | Uplift | |
| | --- | ---: | ---: | ---: | |
| | p50 inference latency | 41.987 ms | **20.431 ms** | **2.06× faster** | |
| | p90 inference latency | 42.612 ms | **21.342 ms** | **2.00× faster** | |
| | p99 inference latency | 43.236 ms | **27.181 ms** | **1.59× faster** | |
|
|
| ## 🛠️ Technical Specifications |
|
|
| ### Runtime Architecture |
|
|
| | Component role | Framework / format | |
| | --- | --- | |
| | Input square crop and resize | ExecuTorch / PTE | |
| | ZipDepth Base inference | ExecuTorch / PTE | |
| | Output resize and zero padding | ExecuTorch / PTE | |
|
|
| ### Precision |
|
|
| - Internal depth-model activations use FP16. |
| - Public input and output tensors remain FP32 to preserve the existing brick contract. |
| - Dynamic image-shape transforms use FP32. |
| - The fixed 24×24-to-12×12 adaptive average pool is represented as an equivalent 2×2, |
| stride-2 average pool supported by the deployed ExecuTorch kernel set. |
|
|
| ### Input Specification |
|
|
| | Input | Data type | Shape | Value range | |
| | --- | --- | --- | --- | |
| | RGB image | FP32 | `3 × H × W`, with `1 ≤ H,W ≤ 4096` | `[0, 1]` | |
|
|
| ### Output Specification |
|
|
| | Output | Data type | Shape | Description | |
| | --- | --- | --- | --- | |
| | Relative depth | FP32 | `H × W` | Affine-invariant inverse depth; padded regions are zero | |
|
|
| ### Repository Contents |
|
|
| - `zipdepth_base_fp16.pte` — static FP16 ZipDepth Base mobile/NPU depth model. |
| - `zipdepth_prepare_image.pte` — dynamic input crop-and-resize transform. |
| - `zipdepth_restore_depth.pte` — dynamic output resize-and-padding transform. |
| - `zipdepth_manifest.json` — model tensor contracts and runtime configuration. |
| - `samples/im0.jpg` — 384×384 upstream sample used for profiling. |
| - `metadata.yaml` and `benchmarks/` — model and benchmark metadata. |
| - `LICENSE` — upstream MIT license. |
| - `SHA256SUMS` — reproducibility checksums. |
|
|
| ## ⚠️ Known Limitations |
|
|
| - The square-crop package pipeline does not preserve the aspect-ratio preprocessing used for |
| ZipDepth's published benchmark results. |
| - The output is affine-invariant relative depth, not metric depth. |
| - The model has no temporal-consistency mechanism and may flicker when applied independently to |
| video frames. |
|
|
| ## 🗂️ Model and Asset Origin |
|
|
| - [`zipdepth_base_npu.pth`](https://github.com/fabiotosi92/ZipDepth/tree/a302e5437bc58f15c4efd41d3e8222bf24f7d470/checkpoints) |
| is the source checkpoint for `zipdepth_base_fp16.pte`. |
| - [`samples/im0.jpg`](https://github.com/fabiotosi92/ZipDepth/blob/a302e5437bc58f15c4efd41d3e8222bf24f7d470/assets/examples/im0.jpg) |
| is an upstream example crop. |
|
|
| ## 🔐 Checksums |
|
|
| Verify the checked-out files with: |
|
|
| ```sh |
| shasum -a 256 -c SHA256SUMS |
| ``` |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{tosi2026zipdepth, |
| title = {{ZipDepth}: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device}, |
| author = {Tosi, Fabio and Bartolomei, Luca and Poggi, Matteo and Mattoccia, Stefano}, |
| booktitle = {European Conference on Computer Vision (ECCV)}, |
| year = {2026} |
| } |
| ``` |
|
|
