| --- |
| license: other |
| license_name: per-file |
| license_link: https://huggingface.co/openpixels-app/browser-models/tree/main/LICENSES |
| library_name: onnx |
| pipeline_tag: image-to-image |
| tags: |
| - onnx |
| - onnxruntime-web |
| - super-resolution |
| - real-esrgan |
| - gfpgan |
| - face-restoration |
| - ddcolor |
| - colorization |
| - background-removal |
| - u2net |
| - modnet |
| - yunet |
| - face-detection |
| --- |
| |
| # OpenPixels models |
|
|
| The ONNX files the [OpenPixels](https://openpixels.app) web app downloads to |
| run in the visitor's own browser: photo upscaling, denoising, face |
| restoration, colourisation and background removal. Nothing here is new |
| research. Every file is an existing open model, converted or repackaged so a |
| browser can run it with onnxruntime-web, and each keeps its own upstream |
| licence, listed below and included in full in [`LICENSES/`](LICENSES). |
|
|
| The app downloads each file once, from a URL that names a commit of this |
| repository rather than a branch, and checks it against the sha256 below |
| before using it. Photos are never sent anywhere: they are processed on the |
| visitor's device. |
|
|
| **Files here are never replaced.** A changed model is published under a new |
| file name, so an app pinned to an earlier commit keeps working and every |
| checksum below stays true. |
|
|
| ## Files |
|
|
| | File | Bytes | Task | Licence | |
| |---|---:|---|---| |
| | `realesr_general_x4v3.onnx` | 4,866,422 | 4× upscale, strong denoise | BSD-3-Clause | |
| | `realesr_general_x4v3_dn50.onnx` | 4,866,422 | 4× upscale, medium denoise | BSD-3-Clause | |
| | `realesr_general_wdn_x4v3.onnx` | 4,866,422 | 4× upscale, light denoise | BSD-3-Clause | |
| | `realesrgan_x4plus_w16.onnx` | 33,724,311 | 4× upscale, detailed | BSD-3-Clause | |
| | `realesrgan_x4plus_anime_6b_w16.onnx` | 9,023,048 | 4× upscale, drawings and anime | BSD-3-Clause | |
| | `gfpgan_1.4_w16.onnx` | 170,305,678 | Face restoration | Apache-2.0, with the notices in GFPGAN's LICENSE (below) | |
| | `ddcolor_paper_tiny_w16.onnx` | 115,401,645 | Colourisation | Apache-2.0 | |
| | `u2netp.onnx` | 4,574,861 | Background removal | Apache-2.0 | |
| | `modnet_photographic_portrait_matting.onnx` | 25,888,640 | Background removal for portraits | Apache-2.0 | |
| | `face_detection_yunet_2023mar.onnx` | 232,589 | Face detection | MIT | |
|
|
| ``` |
| 0e121299bf9b23a764d3f9e2120d0d0727d5ac5032e03a46cb78d28bcc7c6872 realesr_general_x4v3.onnx |
| b345bc59c38dbb78d871b5adf9519c475fe243c198f70cc9d0b97a464715d50b realesr_general_x4v3_dn50.onnx |
| be536211515f088de05f1aa799a8079e92f49978b47aad6cec007f9fca62bb68 realesr_general_wdn_x4v3.onnx |
| 29fbb9bc95708f9f63966804a389be77fe060dbdda882402bd1592a10908d364 realesrgan_x4plus_w16.onnx |
| 26e4edb17ecd25676f58ffca1af3352cbd928bd4bc2eaec1461b7b72be0a2c41 realesrgan_x4plus_anime_6b_w16.onnx |
| 008e67bccdcbdb611273f106dca6f88703cd02884b3ac35dbb8b3785c99a2831 gfpgan_1.4_w16.onnx |
| 87854113d29989a6fc3c680b92b21f2cd5377a54b5babd0c5d2a53469f59244a ddcolor_paper_tiny_w16.onnx |
| 309c8469258dda742793dce0ebea8e6dd393174f89934733ecc8b14c76f4ddd8 u2netp.onnx |
| 07c308cf0fc7e6e8b2065a12ed7fc07e1de8febb7dc7839d7b7f15dd66584df9 modnet_photographic_portrait_matting.onnx |
| 8f2383e4dd3cfbb4553ea8718107fc0423210dc964f9f4280604804ed2552fa4 face_detection_yunet_2023mar.onnx |
| ``` |
|
|
| The scripts that produced every converted file, and the full provenance |
| notes, are in the app's source repository: |
| [MODELS.md](https://gitlab.com/openpixelsapp/OpenPixels/-/blob/main/MODELS.md). |
|
|
| ## What `_w16` means |
| |
| The four `*_w16.onnx` files are the float32 networks with every weight of |
| 1,024 values or more stored as float16 and followed by a `Cast` back to |
| float32. Nothing else in the graph changes. onnxruntime folds the casts into |
| ordinary float32 weights when it builds the session, so all arithmetic is |
| still float32; what is lost is the rounding of each weight to 11 significant |
| bits. Measured against the float32 files over ten real photographs, the |
| outputs agree to 59–69 dB PSNR on onnxruntime's CPU backend, with no pixel |
| more than 3 of 255 levels apart, and to 59–67 dB in Chromium on |
| onnxruntime-web's WebGPU and WASM backends, with at most 5 of 255. The |
| download is half the size. |
|
|
| ## Where each file comes from |
|
|
| **Real-ESRGAN** — Xintao Wang et al., <https://github.com/xinntao/Real-ESRGAN>. |
| BSD 3-Clause License, Copyright (c) 2021, Xintao Wang |
| ([`LICENSES/Real-ESRGAN.txt`](LICENSES/Real-ESRGAN.txt)). Real-ESRGAN |
| publishes PyTorch weights; these ONNX files are exports of them, checked |
| against the PyTorch forward pass. |
|
|
| - `realesr_general_x4v3.onnx` — exported from `realesr-general-x4v3.pth` |
| (release v0.2.5.0). |
| - `realesr_general_wdn_x4v3.onnx` — exported from |
| `realesr-general-wdn-x4v3.pth` (release v0.2.5.0). |
| - `realesr_general_x4v3_dn50.onnx` — the two state dicts above interpolated |
| 50/50 before export, which is what Real-ESRGAN's own |
| `--denoise_strength 0.5` does at run time. |
| - `realesrgan_x4plus_w16.onnx` — exported from `RealESRGAN_x4plus.pth` |
| (release v0.1.0), weights stored at half size as described above. |
| - `realesrgan_x4plus_anime_6b_w16.onnx` — exported from |
| `RealESRGAN_x4plus_anime_6B.pth` (release v0.2.2.4), likewise. |
|
|
| **GFPGAN v1.4** — Tencent ARC, <https://github.com/TencentARC/GFPGAN>. |
| `gfpgan_1.4_w16.onnx` is `gfpgan_1.4.onnx` as published in |
| [facefusion/models-3.0.0](https://huggingface.co/facefusion/models-3.0.0) |
| (sha256 `accc4757…1385`), with its weights stored at half size as described |
| above. That repository publishes the file without a model card; its name |
| corresponds to the GFPGAN v1.4 weights, which Tencent ARC distributes as |
| `GFPGANv1.4.pth` in GFPGAN release v1.3.0. |
|
|
| GFPGAN's LICENSE ([`LICENSES/GFPGAN.txt`](LICENSES/GFPGAN.txt), included |
| unchanged) reads: *"GFPGAN is licensed under the Apache License Version 2.0 |
| except for the third-party components listed below."* Those components are |
| source code in the GFPGAN repository, and two of them carry non-commercial |
| terms: StyleGAN2 code, modified from rosinality's stylegan2-pytorch, whose |
| upstream (NVlabs/stylegan2) is under the NVIDIA licence reproduced there, |
| including its section 3.3 use limitation; and code "largely modified from" |
| DFDNet, under CC BY-NC-SA 4.0. `gfpgan_1.4_w16.onnx` contains no source |
| code: it is the trained GFPGAN v1.4 weights and an ONNX graph of the network |
| they belong to, `GFPGANv1Clean`, whose generator follows the StyleGAN2 |
| architecture that the listed StyleGAN2 code implements. Read the full terms |
| before relying on this file for commercial use. GFPGAN's README separately offers a "clean" |
| version of the architecture that does not require its customised CUDA |
| extensions. The full text is included so the terms can be read as they are |
| written rather than as summarised here. |
|
|
| **DDColor** — Xiaoyang Kang et al., <https://github.com/piddnad/DDColor>. |
| Apache License 2.0 ([`LICENSES/DDColor.txt`](LICENSES/DDColor.txt)). |
| `ddcolor_paper_tiny_w16.onnx` comes from the `ddcolor_paper_tiny` ONNX |
| export in Qualcomm AI Hub Models release v0.63.0 (`ddcolor-onnx-float.zip`; |
| model card licence apache-2.0, AI Hub Models code under BSD-3-Clause, |
| [`LICENSES/qai-hub-models.txt`](LICENSES/qai-hub-models.txt)). That release |
| is a graph plus a separate weights file; they were merged into one file, |
| which gives output identical to the published pair, and the weights were |
| then stored at half size as described above. |
|
|
| **U²-Net-p** — Xuebin Qin et al., <https://github.com/xuebinqin/U-2-Net>. |
| Apache License 2.0 ([`LICENSES/U-2-Net.txt`](LICENSES/U-2-Net.txt)). |
| `u2netp.onnx` is the file rembg publishes in its release v0.0.0, unchanged |
| (<https://github.com/danielgatis/rembg>, MIT, |
| [`LICENSES/rembg.txt`](LICENSES/rembg.txt)). |
|
|
| **MODNet** — Zhanghan Ke et al., <https://github.com/ZHKKKe/MODNet>, whose |
| README states that "the code, models, and demos in this repository … are |
| released under the Apache License 2.0" |
| ([`LICENSES/MODNet.txt`](LICENSES/MODNet.txt)). |
| `modnet_photographic_portrait_matting.onnx` is the official photographic |
| portrait matting weights as redistributed, unchanged, by HivisionIDPhotos in |
| its release `pretrained-model` |
| (<https://github.com/Zeyi-Lin/HivisionIDPhotos>, Apache-2.0, |
| [`LICENSES/HivisionIDPhotos.txt`](LICENSES/HivisionIDPhotos.txt)). |
|
|
| **YuNet** — Shiqi Yu et al., OpenCV Zoo, |
| <https://github.com/opencv/opencv_zoo/tree/main/models/face_detection_yunet>. |
| MIT License, Copyright (c) 2020 Shiqi Yu |
| ([`LICENSES/YuNet.txt`](LICENSES/YuNet.txt)). |
| `face_detection_yunet_2023mar.onnx` is the file published there, unchanged. |
|
|
| ## Changes made to the upstream works |
|
|
| As required by the Apache License 2.0, section 4(b): the files marked |
| `_w16` have had their weights re-stored as float16 with a cast back to |
| float32; `realesr_general_x4v3_dn50.onnx` is an interpolation of two |
| upstream state dicts; the Real-ESRGAN files are ONNX exports of PyTorch |
| weights; and the DDColor file merges a graph with its external weights. |
| Every other file is redistributed byte for byte as its upstream published it. |
|
|
