lc700x commited on
Commit
8b709ba
·
verified ·
1 Parent(s): 5812dad

Add EdgePad ONNX models and original model card

Browse files
LICENSE CHANGED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Fabio Tosi (ZipDepth, the weights these models derive from)
4
+ Copyright (c) 2026 The Nightfall authors (the fine-tuned weights and the
5
+ exported graphs)
6
+
7
+ Permission is hereby granted, free of charge, to any person obtaining a copy
8
+ of this software and associated documentation files (the "Software"), to deal
9
+ in the Software without restriction, including without limitation the rights
10
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
11
+ copies of the Software, and to permit persons to whom the Software is
12
+ furnished to do so, subject to the following conditions:
13
+
14
+ The above copyright notice and this permission notice shall be included in all
15
+ copies or substantial portions of the Software.
16
+
17
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
18
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
19
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
20
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
21
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
22
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
23
+ SOFTWARE.
README.md CHANGED
@@ -1,5 +1,142 @@
1
  ---
2
- license: other
3
- license_name: mit
4
- license_link: LICENSE
 
 
 
 
 
5
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: mit
3
+ pipeline_tag: depth-estimation
4
+ tags:
5
+ - depth-estimation
6
+ - monocular-depth
7
+ - zipdepth
8
+ - edgepad
9
+ - onnx
10
  ---
11
+
12
+ # EdgePad depth models (ZipDepth for mobile GPUs)
13
+
14
+ This repository mirrors the **PC ONNX** EdgePad depth models published by the
15
+ **Nightfall team**. The model research, fine-tuning, graph optimization, and
16
+ exports were created by Nightfall and are credited to its authors and to
17
+ Fabio Tosi, the author of ZipDepth.
18
+
19
+ The model-card text below is based closely on the original release notes,
20
+ with only small changes to describe the files mirrored here.
21
+
22
+ ## About these models
23
+
24
+ The monocular depth models Nightfall uses for AI 3D are published for anyone
25
+ to use: TFLite for phones and headsets (Nightfall runs them on Quest), and
26
+ ONNX for PCs (Nightfall Meteor runs them there). They are MIT licensed; see
27
+ [`LICENSE`](LICENSE).
28
+
29
+ All of them are **EdgePad** builds of
30
+ [ZipDepth](https://github.com/fabiotosi92/ZipDepth) by Fabio Tosi (MIT): a
31
+ 6.1M-parameter convolutional network distilled from Depth Anything V2 Large,
32
+ with no attention or transformer operations, which is what allows it to run
33
+ entirely on a mobile GPU. There are two generations:
34
+
35
+ - **Square EdgePad** (shipped in Nightfall v0.7.x): ZipDepth's own Standard
36
+ checkpoint, **not retrained**, exported through exact graph rewrites so the
37
+ whole network runs on the GPU.
38
+ - **Widescreen EdgePad** (Nightfall's next release): a light fine-tune of the
39
+ same checkpoint for 16:9 video and steadier depth from frame to frame,
40
+ exported at exact 16:9 sizes.
41
+
42
+ ## Files in this mirror
43
+
44
+ | File | Generation | Input | Output | SHA-256 |
45
+ | --- | --- | --- | --- | --- |
46
+ | `zipdepth_edgepad_384.onnx` | Square | 384x384 | 384x384 | `78efcf08caded436c626af0b7b5980db47bc07653ec549a59b7e48e73b3e035f` |
47
+ | `zipdepth_wide_512x288.onnx` | Widescreen | 512x288 | 512x288 | `197d598f50ee3270b12831096dbab260c6bd5979bab13b87f01a28fad0268274` |
48
+ | `zipdepth_wide_672x384.onnx` | Widescreen | 672x384 | 672x384 | `f812b10b10309c6759266cbe888fa9c0f70668350fdf69e7eda9cc0412ca1a21` |
49
+
50
+ Sizes are width x height. These are the PC ONNX exports used by Nightfall
51
+ Meteor. This Hugging Face mirror currently contains the three ONNX files above;
52
+ the original release also provides TFLite exports for mobile and headset use.
53
+
54
+ ## Using them
55
+
56
+ - **Input:** RGB as float32 in 0 to 1 (`pixel / 255`). ImageNet mean and
57
+ standard deviation are applied inside the graph. The ONNX model takes NCHW
58
+ `[1, 3, H, W]` named `image`. Resize the frame to the model input size;
59
+ Nightfall uses an area filter.
60
+ - **Output:** relative inverse depth: larger is nearer, with no fixed scale or
61
+ offset. Normalise each frame before use. Nightfall stretches between low and
62
+ high percentiles and smooths the result over time. The ONNX models output the
63
+ full-size map, already unpacked and clamped.
64
+
65
+ ## Square EdgePad: optimisation only
66
+
67
+ The weights are ZipDepth's Standard checkpoint (`zipdepth_base.pth`), unchanged,
68
+ including its learned convex reconstruction head, which proved essential for
69
+ clean full-screen geometry in the headset. Every change in the Square EdgePad
70
+ export is to the graph and is mathematically exact:
71
+
72
+ 1. **Materialised attention broadcasts.** ZipDepth's strip, channel, and
73
+ global-context attention broadcast small tensors across the feature map.
74
+ These tensors are expanded before the operation for GPU-delegate
75
+ compatibility.
76
+ 2. **Small reductions.** Large spatial average pools are split into exact 2x
77
+ and 3x stages.
78
+ 3. **Packed reconstruction head.** The head's 5-dimensional softmax and
79
+ depth-to-space tail becomes four ordinary 9-channel softmax branches, the
80
+ 36-channel mask convolution becomes four 9-channel convolutions, and the
81
+ weighted sums become fixed 1x1 convolutions.
82
+ 4. **EdgePad.** The head's replicate padding is rebuilt from edge slices and
83
+ concatenation so the whole graph can run without splitting and copying
84
+ intermediate tensors back to the CPU.
85
+
86
+ Against the original Standard graph, Nightfall reports correlation
87
+ `0.9999999999999908` and normalised mean absolute error `3.6e-8`.
88
+
89
+ ## Widescreen EdgePad: a light fine-tune
90
+
91
+ One checkpoint (SHA-256
92
+ `2dc8fddcef9ef22f45b6a1573b9cf66de0f39b1b5172e4284dd693790bd6309e`)
93
+ was trained from ZipDepth's Standard checkpoint in two short stages. Only the
94
+ decoder was trained: 154,005 of 6.1 million parameters. The encoder is
95
+ ZipDepth's.
96
+
97
+ - **Stage 1** (5,000 steps): shapes alternated between 384x384, 256x256,
98
+ 224x224, and 512x288, so one set of weights serves every size. Training used
99
+ 2,000 frames from 25 videos: gameplay/YouTube, film, live-action/CG, and
100
+ animation. Labels were the original Square EdgePad-384 output, Video Depth
101
+ Anything Small for frame-to-frame consistency, and Marigold V2 for spatial
102
+ detail on selected keyframes. A texture-leakage term at weight 0.5
103
+ discourages copying image texture into depth.
104
+ - **Stage 2** (1,000 steps at learning rate `5e-5`): decoder-only training for
105
+ the exact 16:9 sizes.
106
+
107
+ The model has no memory and processes each frame independently; temporal
108
+ behaviour was learned only through the losses.
109
+
110
+ Nightfall's held-out video results against the untouched ZipDepth export at
111
+ the same size:
112
+
113
+ | Model | Compared with | Flicker on static scenes | Flow-warped error | Frame-to-frame change |
114
+ | --- | --- | --- | --- | --- |
115
+ | 512x288 | stock 512x288 | -8.6% | -5.8% | -8.4% |
116
+ | 320x180 | stock 224x224 | -42.4% | -43.9% | -39.2% |
117
+
118
+ The trade-off is about 11% higher texture leakage than the stock 224x224 model
119
+ at 320x180. Nightfall notes that the 320x180 model is also visibly less
120
+ detailed than the 384x384 model, as its size implies.
121
+
122
+ Nightfall reports that the TFLite exports match their ONNX sources with
123
+ correlation `0.99999` and pass LiteRT's GPU compatibility check without
124
+ warnings. Meteor's ONNX files are the same weights with the 2x2 unpacking and
125
+ clamp added to the graph and the weights embedded. The 672x384 file is an
126
+ export of the same checkpoint at a larger size and was not part of the
127
+ held-out evaluation above.
128
+
129
+ ## Credits and licences
130
+
131
+ - **ZipDepth:** Fabio Tosi, MIT. These models are derivatives of its weights;
132
+ its copyright notice is retained in [`LICENSE`](LICENSE).
133
+ - **Teacher models** used only to make training labels, not included here:
134
+ Video Depth Anything Small (Apache-2.0) and Marigold V2 (Apache-2.0; built
135
+ on Qwen-Image-Edit-2509, Apache-2.0).
136
+ - **Fine-tuning, graph optimisation, and exports:** Nightfall's model research
137
+ team. Sincere thanks to **tB0nE and all Nightfall contributors** for making
138
+ these models and the detailed release notes available.
139
+
140
+ This Hugging Face repository is a mirror provided for convenient access. All
141
+ credit for the original models and release documentation belongs to the
142
+ Nightfall team and Fabio Tosi.
zipdepth_edgepad_384.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:78efcf08caded436c626af0b7b5980db47bc07653ec549a59b7e48e73b3e035f
3
+ size 24608261
zipdepth_wide_512x288.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:197d598f50ee3270b12831096dbab260c6bd5979bab13b87f01a28fad0268274
3
+ size 24625797
zipdepth_wide_672x384.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f812b10b10309c6759266cbe888fa9c0f70668350fdf69e7eda9cc0412ca1a21
3
+ size 24624796