Depth, matte and fill models for Parallax Wallpaper Studio

README.md

@@ -0,0 +1,41 @@
+---
+license: apache-2.0
+library_name: onnx
+tags:
+  - onnx
+  - depth-estimation
+  - image-matting
+  - inpainting
+  - android
+---
+
+# Parallax Wallpaper Studio models
+
+ONNX exports of three open models, made to run on the phone's CPU (ONNX Runtime) inside the Android app
+Parallax Wallpaper Studio. The app downloads them on demand: the depth model for depth-based parallax, and all
+three for "Split into layers", which turns one photo into a layered wallpaper.
+
+| File | Model | Used for | Licence |
+|---|---|---|---|
+| `depth-anything-v2-small-int8.onnx` | [Depth Anything V2 Small](https://huggingface.co/depth-anything/Depth-Anything-V2-Small-hf), int8 dynamic quantisation | Depth map | Apache-2.0 |
+| `vitmatte-small-512-fp16w.onnx` | [ViTMatte Small (Composition-1k)](https://huggingface.co/hustvl/vitmatte-small-composition-1k), fixed 512×512 input, fp16 weights | Refining layer edges | Apache-2.0 |
+| `big-lama-512-fp16w.onnx` | [LaMa](https://github.com/advimman/lama) (big-lama), fixed 512×512 input, fp16 weights | Filling the holes behind cut-out layers | Apache-2.0 |
+
+Notes on the exports:
+
+- The depth model matches the full-precision model to 0.9998 correlation.
+- LaMa's FFTs have no ONNX export, so they are rewritten as constant DFT matrix multiplications. That fixes the
+  input size at 512×512.
+- "fp16 weights" means the weights are stored as float16 and the maths runs in float32.
+
+## Credits
+
+- **Depth Anything V2**: Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, Hengshuang Zhao.
+  *Depth Anything V2*, 2024. Only the Small model is Apache-2.0; Base and Large are not, and are not used here.
+- **ViTMatte**: Jingfeng Yao, Xinggang Wang, Shusheng Yang, Baoyuan Wang. *ViTMatte: Boosting Image Matting with
+  Pretrained Plain Vision Transformers*, 2023 (hustvl).
+- **LaMa**: Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei
+  Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, Victor Lempitsky. *Resolution-robust Large Mask Inpainting
+  with Fourier Convolutions*, WACV 2022 (Samsung AI Center).
+
+These are format conversions of the original weights. The licences and copyright of the original works apply.

big-lama-512-fp16w.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1d765c2c58e62a4d027b988e0faea43d1868e88366317318ca0a7df15defaf0
+size 106442009

depth-anything-v2-small-int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b674426508aa52b0da136a70bb8e8cdd0888a630a4526350d9375caf82b1395b
+size 35621067

vitmatte-small-512-fp16w.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0869db6a4247bf4d632419e68f7e953c8c2ecc15a38bc53102a311c6f62f4d6e
+size 52397598