Card: weights and model code verified against the official MODNet release
Browse files
NOTICE
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
This repository contains a modified version of https://github.com/ZHKKKe/MODNet.
|
| 2 |
|
| 3 |
Modifications by Meta Platforms, Inc.: conversion to the ExecuTorch program format (.pte) for the
|
| 4 |
-
ExecuTorch WebGPU backend
|
| 5 |
|
| 6 |
The original work and these modifications are distributed under the license in LICENSE.
|
|
|
|
| 1 |
This repository contains a modified version of https://github.com/ZHKKKe/MODNet.
|
| 2 |
|
| 3 |
Modifications by Meta Platforms, Inc.: conversion to the ExecuTorch program format (.pte) for the
|
| 4 |
+
ExecuTorch WebGPU backend.
|
| 5 |
|
| 6 |
The original work and these modifications are distributed under the license in LICENSE.
|
README.md
CHANGED
|
@@ -2,6 +2,7 @@
|
|
| 2 |
license: apache-2.0
|
| 3 |
library_name: executorch
|
| 4 |
pipeline_tag: image-segmentation
|
|
|
|
| 5 |
tags:
|
| 6 |
- executorch
|
| 7 |
- webgpu
|
|
@@ -12,14 +13,39 @@ tags:
|
|
| 12 |
|
| 13 |
This repository contains an [ExecuTorch](https://github.com/pytorch/executorch) conversion of
|
| 14 |
https://github.com/ZHKKKe/MODNet for the ExecuTorch **WebGPU** backend, so the model runs entirely in a web browser on
|
| 15 |
-
the visitor's GPU. It powers the corresponding demo in the
|
| 16 |
-
ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)).
|
| 17 |
|
| 18 |
- **Base model:** https://github.com/ZHKKKe/MODNet
|
| 19 |
- **Format:** ExecuTorch `.pte` programs delegated to the WebGPU backend
|
| 20 |
- **Quantization:** fp32 (as exported)
|
| 21 |
- **Total size:** 24.9 MB
|
| 22 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
## Files
|
| 24 |
|
| 25 |
| File | Size | SHA-256 |
|
|
@@ -30,20 +56,37 @@ Full checksums are in `manifest.json`. The playground verifies them after downlo
|
|
| 30 |
|
| 31 |
## Running it
|
| 32 |
|
| 33 |
-
These programs
|
| 34 |
-
([backends/webgpu](https://github.com/pytorch/executorch/tree/main/backends/webgpu))
|
| 35 |
-
|
| 36 |
-
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
## Provenance
|
| 39 |
|
| 40 |
- **Weights:** `boopathiraj/MODNet` @ `a967175f8fd63de67a1c161d8daac966d2b57e3a`, file `modnet/pretrained/modnet_photographic_portrait_matting.ckpt`, sha256 `7c22235f0925deba15d4d63e53afcb654c47055bbcd98f56e393ab2584007ed8` (from a community mirror of the official release)
|
|
|
|
| 41 |
- **Exported with:** executorch 1.5.1+cpu, torch 2.14.1+cpu, torchao 0.18.0+cpu, transformers 5.19.0 (executorch git `3b6068392324`)
|
| 42 |
- **Reproduce:** `python recipes/modnet.py --out out/modnet` (recipe in `export/`)
|
| 43 |
- Re-running the recipe produces byte-identical files.
|
| 44 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
## License
|
| 46 |
|
| 47 |
This is a derivative of the upstream model and is distributed under the same terms
|
| 48 |
(`apache-2.0`); see `LICENSE`.
|
| 49 |
-
Modifications (conversion to the ExecuTorch format
|
|
|
|
| 2 |
license: apache-2.0
|
| 3 |
library_name: executorch
|
| 4 |
pipeline_tag: image-segmentation
|
| 5 |
+
inference: false
|
| 6 |
tags:
|
| 7 |
- executorch
|
| 8 |
- webgpu
|
|
|
|
| 13 |
|
| 14 |
This repository contains an [ExecuTorch](https://github.com/pytorch/executorch) conversion of
|
| 15 |
https://github.com/ZHKKKe/MODNet for the ExecuTorch **WebGPU** backend, so the model runs entirely in a web browser on
|
| 16 |
+
the visitor's GPU. It powers the corresponding demo in the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)).
|
|
|
|
| 17 |
|
| 18 |
- **Base model:** https://github.com/ZHKKKe/MODNet
|
| 19 |
- **Format:** ExecuTorch `.pte` programs delegated to the WebGPU backend
|
| 20 |
- **Quantization:** fp32 (as exported)
|
| 21 |
- **Total size:** 24.9 MB
|
| 22 |
|
| 23 |
+
## About the original model
|
| 24 |
+
|
| 25 |
+
A real-time portrait matting network that separates people from the background without a trimap.
|
| 26 |
+
|
| 27 |
+
- **Developed by:** Zhanghan Ke et al.
|
| 28 |
+
- **Paper:** [MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition](https://arxiv.org/abs/2011.11961)
|
| 29 |
+
- 8.7M parameters in the released checkpoint
|
| 30 |
+
- Photographic portrait matting checkpoint
|
| 31 |
+
|
| 32 |
+
For training data, evaluation results and full details, see the original model card: https://github.com/ZHKKKe/MODNet.
|
| 33 |
+
|
| 34 |
+
## Intended use and limitations
|
| 35 |
+
|
| 36 |
+
Cutting people out of photos or camera frames, for example to blur or replace the background.
|
| 37 |
+
|
| 38 |
+
Trained for portraits: results degrade on non-human subjects, several overlapping people, or unusual poses and lighting.
|
| 39 |
+
|
| 40 |
+
## What changed in this conversion
|
| 41 |
+
|
| 42 |
+
- Fixed 256x256 input; the matte is upsampled to the frame size outside the program
|
| 43 |
+
|
| 44 |
+
## Inputs and outputs
|
| 45 |
+
|
| 46 |
+
- Input `[1, 3, 256, 256]` float32 RGB normalized to [-1, 1]: `(x / 255 - 0.5) / 0.5`
|
| 47 |
+
- Output `[1, 1, 256, 256]` float32 alpha matte in [0, 1]
|
| 48 |
+
|
| 49 |
## Files
|
| 50 |
|
| 51 |
| File | Size | SHA-256 |
|
|
|
|
| 56 |
|
| 57 |
## Running it
|
| 58 |
|
| 59 |
+
These programs were exported with the ExecuTorch WebGPU backend
|
| 60 |
+
([backends/webgpu](https://github.com/pytorch/executorch/tree/main/backends/webgpu)) and validated with the
|
| 61 |
+
WebAssembly runtime build used by the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)). That runtime build is not published yet; until it is,
|
| 62 |
+
the playground is the supported way to run them. It downloads these files once, caches them in the browser,
|
| 63 |
+
and runs everything locally.
|
| 64 |
+
|
| 65 |
+
Requirements: a browser with WebGPU enabled.
|
| 66 |
|
| 67 |
## Provenance
|
| 68 |
|
| 69 |
- **Weights:** `boopathiraj/MODNet` @ `a967175f8fd63de67a1c161d8daac966d2b57e3a`, file `modnet/pretrained/modnet_photographic_portrait_matting.ckpt`, sha256 `7c22235f0925deba15d4d63e53afcb654c47055bbcd98f56e393ab2584007ed8` (from a community mirror of the official release)
|
| 70 |
+
- **Checked against the official checkpoint:** `modnet_photographic_portrait_matting.ckpt` is byte-identical to the official checkpoint linked from github.com/ZHKKKe/MODNet (sha256 `7c22235f0925deba15d4d63e53afcb654c47055bbcd98f56e393ab2584007ed8`), and the model code used for the export is identical to the official `src/models/` (one import path differs), checked by the project owner on 2026-10-16
|
| 71 |
- **Exported with:** executorch 1.5.1+cpu, torch 2.14.1+cpu, torchao 0.18.0+cpu, transformers 5.19.0 (executorch git `3b6068392324`)
|
| 72 |
- **Reproduce:** `python recipes/modnet.py --out out/modnet` (recipe in `export/`)
|
| 73 |
- Re-running the recipe produces byte-identical files.
|
| 74 |
|
| 75 |
+
## Citation
|
| 76 |
+
|
| 77 |
+
If you use this model, please cite the original work:
|
| 78 |
+
|
| 79 |
+
```bibtex
|
| 80 |
+
@inproceedings{ke2022modnet,
|
| 81 |
+
title={MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition},
|
| 82 |
+
author={Ke, Zhanghan and Sun, Jiayu and Li, Kaican and Yan, Qiong and Lau, Rynson W.H.},
|
| 83 |
+
booktitle={AAAI},
|
| 84 |
+
year={2022}
|
| 85 |
+
}
|
| 86 |
+
```
|
| 87 |
+
|
| 88 |
## License
|
| 89 |
|
| 90 |
This is a derivative of the upstream model and is distributed under the same terms
|
| 91 |
(`apache-2.0`); see `LICENSE`.
|
| 92 |
+
Modifications (conversion to the ExecuTorch format) by Meta Platforms, Inc.
|