Add DDColor paper tiny INT8 (default colouriser), DDColor build scripts, update model card

1004324 verified 1 day ago

9.02 kB

---
license: other
license_name: mixed-apache-2.0-bsd-3-clause
license_link: LICENSE
library_name: onnx
pipeline_tag: image-to-image
tags:
- onnx
- gfpgan
- real-esrgan
- u2net
- face-restoration
- super-resolution
- background-removal
- colorization
- ddcolor
- on-device
---
# OptiMix models

ONNX model files downloaded on first use by the OptiMix Android app (`com.momoapps.optimix`, MomoApps).
Every file is a mirror or a derivative of third-party open-source weights. All credit goes to the original authors listed below.
Each file keeps its upstream licence. This repository adds no new licence terms.

Pin a commit in your download URL (`/resolve/<commit>/<file>`) and verify the SHA-256 before loading.

## Files

| File | Size (bytes) | SHA-256 | Model | Upstream source | Licence |
|---|---|---|---|---|---|
| `gfpgan-v1.4-int8.onnx` | 85,567,123 | `9771b4b4eb9568d4458bbee4ebb59ef9380827a6f91a5814603b08bf356db91b` | GFPGAN v1.4, weight-only INT8 (default enhancer) | Built by MomoApps from the official `GFPGANv1.4.pth` ([TencentARC/GFPGAN release v1.3.0](https://github.com/TencentARC/GFPGAN/releases/tag/v1.3.0), `params_ema`) exported with the clean `GFPGANv1Clean` architecture (tag v1.3.4), then per-channel INT8-quantised (weights only, including the StyleGAN modulated-conv kernels). | Apache-2.0, © TencentARC (Xintao Wang et al.) |
| `gfpgan-v1.4.onnx` | 170,362,246 | `6b12e9428c2a0983b2726604f09c4eec9e2e250370c79946a31b139f9befc1ea` | GFPGAN v1.4, fp16 weights / fp32 I/O (HD option) | Unmodified copy of `gfpgan14-fp16.onnx` from [sidkonduru/gfpgan-v1.4-onnx](https://huggingface.co/sidkonduru/gfpgan-v1.4-onnx) (commit `3777aaa98f90f704884e977670d8abfc15ede03f`), a conversion of the official TencentARC `GFPGANv1.4.pth` | Apache-2.0, © TencentARC (Xintao Wang et al.) |
| `realesr-general-x4v3.onnx` | 4,866,420 | `7b10e80021ec3e0ff35017ae5022791879d93a799b2e201e1010a37510202266` | Real-ESRGAN general x4v3 (SRVGGNetCompact), fp32 | Unmodified copy of `onnx/model.onnx` from [vlad-grig/realesr-general-x4v3-onnx](https://huggingface.co/vlad-grig/realesr-general-x4v3-onnx) (commit `0e95d93e454cf6b5de9bdf5a1c9e7ec340d38231`), an export of the official `realesr-general-x4v3.pth` from [xinntao/Real-ESRGAN release v0.2.5.0](https://github.com/xinntao/Real-ESRGAN/releases/tag/v0.2.5.0) | BSD-3-Clause, © 2021 Xintao Wang |
| `u2netp.onnx` | 4,574,861 | `309c8469258dda742793dce0ebea8e6dd393174f89934733ecc8b14c76f4ddd8` | U²-Netp (small) salient-object matte (default background model) | Unmodified copy of `u2netp.onnx` from [danielgatis/rembg release v0.0.0](https://github.com/danielgatis/rembg/releases/tag/v0.0.0) (rembg is MIT), converted from the weights of [xuebinqin/U-2-Net](https://github.com/xuebinqin/U-2-Net) | Apache-2.0, © Xuebin Qin et al. |
| `u2net.onnx` | 175,997,641 | `8d10d2f3bb75ae3b6d527c77944fc5e7dcd94b29809d47a739a7a728a912b491` | U²-Net (full) salient-object matte (HD option) | Unmodified copy of `u2net.onnx` from [danielgatis/rembg release v0.0.0](https://github.com/danielgatis/rembg/releases/tag/v0.0.0) (rembg is MIT), converted from the weights of [xuebinqin/U-2-Net](https://github.com/xuebinqin/U-2-Net) | Apache-2.0, © Xuebin Qin et al. |
| `ddcolor-paper-tiny.onnx` | 220,552,834 | `5479b1b00d8ba2b3c312e5378e75dc93eac6f80c8497e90169801896e1eeb68d` | DDColor-T (`ddcolor_paper_tiny`), ConvNeXt-T encoder + multi-scale colour decoder, fp32, opset 17 (photo colourisation) | Built by MomoApps from the official weights [piddnad/ddcolor_paper_tiny](https://huggingface.co/piddnad/ddcolor_paper_tiny) (commit `cf9fd99c1d7472689ec7413441c1b799a51866a3`, `pytorch_model.bin` sha256 `8a1277bc90a1bfbb6d2d83933a9a6bc821931879ca93e26e4fcec12165d41fce`) and the architecture in [piddnad/DDColor](https://github.com/piddnad/DDColor) (commit `2adb63f2656ac41cbdf7b894cddd94121a3faf13`). The weights load with no missing or unexpected keys. The 8 spectral-norm layers are folded into plain weights, which is exact in eval mode. Exported with `torch.onnx.export` (torch 2.15.0). | Apache-2.0, © Xiaoyang Kang et al. (DAMO Academy, Alibaba) |
| `ddcolor-paper-tiny-int8.onnx` | 56,619,279 | `29f3a5f5f794959faebaf034a142c34712833623f9ce672f6ef8f0a8ae8e5f5a` | DDColor-T, weight-only per-channel INT8 (default colouriser) | Built by MomoApps from `ddcolor-paper-tiny.onnx` above with `scripts/quantize_ddcolor_int8.py`. The 161 Conv/MatMul/Gemm weight tensors are stored as symmetric INT8 with one scale per output channel and expanded by `DequantizeLinear`. All compute stays float32, and Einsum, Erf, LayerNormalization, norms and biases stay in float. Against the float32 ONNX on two DDColor test photos, the mean ab difference is 0.41 / 0.32. Visually indistinguishable apart from edge artifacts the float model already has. | Apache-2.0, © Xiaoyang Kang et al. (DAMO Academy, Alibaba) |

## Input and output contracts

- **GFPGAN (both files):** input `input`, float32 `[1,3,512,512]` RGB in −1..1 (one aligned face crop). Output `output`, the same shape and range.
- **Real-ESRGAN general x4v3:** input `input`, float32 `[1,3,H,W]` RGB in 0..1, with dynamic H and W. Output `output`, `[1,3,4H,4W]`. Clamp the output to 0..1.
- **DDColor paper tiny (both files):** input `input`, float32 `[1,3,512,512]` RGB in 0..1. Build it as DDColor's `ColorizationPipeline` does: resize the photo to 512×512, convert to Lab (OpenCV float), keep L, set a = b = 0, and convert back to RGB. The result is the greyscale L channel replicated as RGB. Output `output`, float32 `[1,2,512,512]`: the predicted Lab a and b channels on OpenCV's float Lab scale (roughly −128..127; clamp them). Resize a and b to the original size, combine them with the original L, and convert Lab to RGB. Ops include `Einsum`, `Erf` and `LayerNormalization`, which the full ONNX Runtime Android package supports. A reduced-operator ORT Mobile build must include them. The INT8 file also uses `DequantizeLinear` (opset 13+, per-axis). ONNX Runtime's mobile checker recommends the CPU execution provider for both files, and both convert to `.ort` format.
- **U²-Net / U²-Netp:** input `input.1`, float32 `[1,3,320,320]` RGB, with ImageNet normalisation applied after dividing by the peak pixel. The first output is the matte.

## Build scripts (DDColor)

`scripts/` holds the scripts that produced the DDColor files. They are deterministic: re-running them on the box gave byte-identical files.

- `export_ddcolor_onnx.py DDCOLOR_REPO WEIGHTS_DIR OUT.onnx TEST_IMAGE [PREVIEW.jpg]` exports the float32 ONNX (opset 17) from the official weights. It folds spectral norm into plain weights and checks the result against PyTorch: max |Δab| was 0.0022 on the Audrey Hepburn test photo.
- `quantize_ddcolor_int8.py SRC.onnx DEST.onnx` produces the weight-only per-channel INT8 file.
- `compare_ddcolor.py REF.onnx TEST.onnx OUTDIR IMG...` reports the ab differences and writes side-by-side colourised JPGs.

The tools used were torch 2.15.0 (CPU), onnx 1.23.2, onnxruntime 1.31.1 and opencv-python-headless.

## Training data note: FFHQ

GFPGAN was trained on the [FFHQ](https://github.com/NVlabs/ffhq-dataset) face dataset. FFHQ's images are published under a
**non-commercial** dataset licence (Creative Commons BY-NC-SA 4.0 for the dataset compilation, plus per-image licences).
The GFPGAN code and weights are released under Apache-2.0. Whether the dataset's terms carry over to the trained weights is legally unsettled.
Commercial users of the GFPGAN files should make their own assessment.

## Training data note: ImageNet

DDColor `ddcolor_paper_tiny` was trained on [ImageNet](https://www.image-net.org/) (ILSVRC 2012), according to the DDColor model zoo. ImageNet's terms of access limit the images to non-commercial research and educational use.
The DDColor code and weights are released under Apache-2.0. As with FFHQ above, whether the dataset's terms carry over to the trained weights is legally unsettled.

## Attribution and citations

- GFPGAN: Xintao Wang, Yu Li, Honglun Zhang, Ying Shan. *Towards Real-World Blind Face Restoration with Generative Facial Prior.* CVPR 2021. https://github.com/TencentARC/GFPGAN (Apache-2.0)
- Real-ESRGAN: Xintao Wang, Liangbin Xie, Chao Dong, Ying Shan. *Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data.* ICCVW 2021. https://github.com/xinntao/Real-ESRGAN (BSD-3-Clause)
- U²-Net: Xuebin Qin, Zichen Zhang, Chenyang Huang, Masood Dehghan, Osmar R. Zaiane, Martin Jagersand. *U²-Net: Going Deeper with Nested U-Structure for Salient Object Detection.* Pattern Recognition 2020. https://github.com/xuebinqin/U-2-Net (Apache-2.0)
- DDColor: Xiaoyang Kang, Tao Yang, Wenqi Ouyang, Peiran Ren, Lingzhi Li, Xuansong Xie. *DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders.* ICCV 2023. https://github.com/piddnad/DDColor (Apache-2.0)
- ONNX conversions: rembg by Daniel Gatis (MIT), sidkonduru/gfpgan-v1.4-onnx, vlad-grig/realesr-general-x4v3-onnx.

See `LICENSE` for the full licence texts.

	---
	license: other
	license_name: mixed-apache-2.0-bsd-3-clause
	license_link: LICENSE
	library_name: onnx
	pipeline_tag: image-to-image
	tags:
	- onnx
	- gfpgan
	- real-esrgan
	- u2net
	- face-restoration
	- super-resolution
	- background-removal
	- colorization
	- ddcolor
	- on-device
	---

	# OptiMix models

	ONNX model files downloaded on first use by the OptiMix Android app (`com.momoapps.optimix`, MomoApps).
	Every file is a mirror or a derivative of third-party open-source weights. All credit goes to the original authors listed below.
	Each file keeps its upstream licence. This repository adds no new licence terms.

	Pin a commit in your download URL (`/resolve/<commit>/<file>`) and verify the SHA-256 before loading.

	## Files

	\| File \| Size (bytes) \| SHA-256 \| Model \| Upstream source \| Licence \|
	\|---\|---\|---\|---\|---\|---\|
	\| `gfpgan-v1.4-int8.onnx` \| 85,567,123 \| `9771b4b4eb9568d4458bbee4ebb59ef9380827a6f91a5814603b08bf356db91b` \| GFPGAN v1.4, weight-only INT8 (default enhancer) \| Built by MomoApps from the official `GFPGANv1.4.pth` ([TencentARC/GFPGAN release v1.3.0](https://github.com/TencentARC/GFPGAN/releases/tag/v1.3.0), `params_ema`) exported with the clean `GFPGANv1Clean` architecture (tag v1.3.4), then per-channel INT8-quantised (weights only, including the StyleGAN modulated-conv kernels). \| Apache-2.0, © TencentARC (Xintao Wang et al.) \|
	\| `gfpgan-v1.4.onnx` \| 170,362,246 \| `6b12e9428c2a0983b2726604f09c4eec9e2e250370c79946a31b139f9befc1ea` \| GFPGAN v1.4, fp16 weights / fp32 I/O (HD option) \| Unmodified copy of `gfpgan14-fp16.onnx` from [sidkonduru/gfpgan-v1.4-onnx](https://huggingface.co/sidkonduru/gfpgan-v1.4-onnx) (commit `3777aaa98f90f704884e977670d8abfc15ede03f`), a conversion of the official TencentARC `GFPGANv1.4.pth` \| Apache-2.0, © TencentARC (Xintao Wang et al.) \|
	\| `realesr-general-x4v3.onnx` \| 4,866,420 \| `7b10e80021ec3e0ff35017ae5022791879d93a799b2e201e1010a37510202266` \| Real-ESRGAN general x4v3 (SRVGGNetCompact), fp32 \| Unmodified copy of `onnx/model.onnx` from [vlad-grig/realesr-general-x4v3-onnx](https://huggingface.co/vlad-grig/realesr-general-x4v3-onnx) (commit `0e95d93e454cf6b5de9bdf5a1c9e7ec340d38231`), an export of the official `realesr-general-x4v3.pth` from [xinntao/Real-ESRGAN release v0.2.5.0](https://github.com/xinntao/Real-ESRGAN/releases/tag/v0.2.5.0) \| BSD-3-Clause, © 2021 Xintao Wang \|
	\| `u2netp.onnx` \| 4,574,861 \| `309c8469258dda742793dce0ebea8e6dd393174f89934733ecc8b14c76f4ddd8` \| U²-Netp (small) salient-object matte (default background model) \| Unmodified copy of `u2netp.onnx` from [danielgatis/rembg release v0.0.0](https://github.com/danielgatis/rembg/releases/tag/v0.0.0) (rembg is MIT), converted from the weights of [xuebinqin/U-2-Net](https://github.com/xuebinqin/U-2-Net) \| Apache-2.0, © Xuebin Qin et al. \|
	\| `u2net.onnx` \| 175,997,641 \| `8d10d2f3bb75ae3b6d527c77944fc5e7dcd94b29809d47a739a7a728a912b491` \| U²-Net (full) salient-object matte (HD option) \| Unmodified copy of `u2net.onnx` from [danielgatis/rembg release v0.0.0](https://github.com/danielgatis/rembg/releases/tag/v0.0.0) (rembg is MIT), converted from the weights of [xuebinqin/U-2-Net](https://github.com/xuebinqin/U-2-Net) \| Apache-2.0, © Xuebin Qin et al. \|
	\| `ddcolor-paper-tiny.onnx` \| 220,552,834 \| `5479b1b00d8ba2b3c312e5378e75dc93eac6f80c8497e90169801896e1eeb68d` \| DDColor-T (`ddcolor_paper_tiny`), ConvNeXt-T encoder + multi-scale colour decoder, fp32, opset 17 (photo colourisation) \| Built by MomoApps from the official weights [piddnad/ddcolor_paper_tiny](https://huggingface.co/piddnad/ddcolor_paper_tiny) (commit `cf9fd99c1d7472689ec7413441c1b799a51866a3`, `pytorch_model.bin` sha256 `8a1277bc90a1bfbb6d2d83933a9a6bc821931879ca93e26e4fcec12165d41fce`) and the architecture in [piddnad/DDColor](https://github.com/piddnad/DDColor) (commit `2adb63f2656ac41cbdf7b894cddd94121a3faf13`). The weights load with no missing or unexpected keys. The 8 spectral-norm layers are folded into plain weights, which is exact in eval mode. Exported with `torch.onnx.export` (torch 2.15.0). \| Apache-2.0, © Xiaoyang Kang et al. (DAMO Academy, Alibaba) \|
	\| `ddcolor-paper-tiny-int8.onnx` \| 56,619,279 \| `29f3a5f5f794959faebaf034a142c34712833623f9ce672f6ef8f0a8ae8e5f5a` \| DDColor-T, weight-only per-channel INT8 (default colouriser) \| Built by MomoApps from `ddcolor-paper-tiny.onnx` above with `scripts/quantize_ddcolor_int8.py`. The 161 Conv/MatMul/Gemm weight tensors are stored as symmetric INT8 with one scale per output channel and expanded by `DequantizeLinear`. All compute stays float32, and Einsum, Erf, LayerNormalization, norms and biases stay in float. Against the float32 ONNX on two DDColor test photos, the mean ab difference is 0.41 / 0.32. Visually indistinguishable apart from edge artifacts the float model already has. \| Apache-2.0, © Xiaoyang Kang et al. (DAMO Academy, Alibaba) \|

	## Input and output contracts

	- GFPGAN (both files): input `input`, float32 `[1,3,512,512]` RGB in −1..1 (one aligned face crop). Output `output`, the same shape and range.
	- Real-ESRGAN general x4v3: input `input`, float32 `[1,3,H,W]` RGB in 0..1, with dynamic H and W. Output `output`, `[1,3,4H,4W]`. Clamp the output to 0..1.
	- DDColor paper tiny (both files): input `input`, float32 `[1,3,512,512]` RGB in 0..1. Build it as DDColor's `ColorizationPipeline` does: resize the photo to 512×512, convert to Lab (OpenCV float), keep L, set a = b = 0, and convert back to RGB. The result is the greyscale L channel replicated as RGB. Output `output`, float32 `[1,2,512,512]`: the predicted Lab a and b channels on OpenCV's float Lab scale (roughly −128..127; clamp them). Resize a and b to the original size, combine them with the original L, and convert Lab to RGB. Ops include `Einsum`, `Erf` and `LayerNormalization`, which the full ONNX Runtime Android package supports. A reduced-operator ORT Mobile build must include them. The INT8 file also uses `DequantizeLinear` (opset 13+, per-axis). ONNX Runtime's mobile checker recommends the CPU execution provider for both files, and both convert to `.ort` format.
	- U²-Net / U²-Netp: input `input.1`, float32 `[1,3,320,320]` RGB, with ImageNet normalisation applied after dividing by the peak pixel. The first output is the matte.

	## Build scripts (DDColor)

	`scripts/` holds the scripts that produced the DDColor files. They are deterministic: re-running them on the box gave byte-identical files.

	- `export_ddcolor_onnx.py DDCOLOR_REPO WEIGHTS_DIR OUT.onnx TEST_IMAGE [PREVIEW.jpg]` exports the float32 ONNX (opset 17) from the official weights. It folds spectral norm into plain weights and checks the result against PyTorch: max \|Δab\| was 0.0022 on the Audrey Hepburn test photo.
	- `quantize_ddcolor_int8.py SRC.onnx DEST.onnx` produces the weight-only per-channel INT8 file.
	- `compare_ddcolor.py REF.onnx TEST.onnx OUTDIR IMG...` reports the ab differences and writes side-by-side colourised JPGs.

	The tools used were torch 2.15.0 (CPU), onnx 1.23.2, onnxruntime 1.31.1 and opencv-python-headless.

	## Training data note: FFHQ

	GFPGAN was trained on the [FFHQ](https://github.com/NVlabs/ffhq-dataset) face dataset. FFHQ's images are published under a
	non-commercial dataset licence (Creative Commons BY-NC-SA 4.0 for the dataset compilation, plus per-image licences).
	The GFPGAN code and weights are released under Apache-2.0. Whether the dataset's terms carry over to the trained weights is legally unsettled.
	Commercial users of the GFPGAN files should make their own assessment.

	## Training data note: ImageNet

	DDColor `ddcolor_paper_tiny` was trained on [ImageNet](https://www.image-net.org/) (ILSVRC 2012), according to the DDColor model zoo. ImageNet's terms of access limit the images to non-commercial research and educational use.
	The DDColor code and weights are released under Apache-2.0. As with FFHQ above, whether the dataset's terms carry over to the trained weights is legally unsettled.

	## Attribution and citations

	- GFPGAN: Xintao Wang, Yu Li, Honglun Zhang, Ying Shan. Towards Real-World Blind Face Restoration with Generative Facial Prior. CVPR 2021. https://github.com/TencentARC/GFPGAN (Apache-2.0)
	- Real-ESRGAN: Xintao Wang, Liangbin Xie, Chao Dong, Ying Shan. Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data. ICCVW 2021. https://github.com/xinntao/Real-ESRGAN (BSD-3-Clause)
	- U²-Net: Xuebin Qin, Zichen Zhang, Chenyang Huang, Masood Dehghan, Osmar R. Zaiane, Martin Jagersand. U²-Net: Going Deeper with Nested U-Structure for Salient Object Detection. Pattern Recognition 2020. https://github.com/xuebinqin/U-2-Net (Apache-2.0)
	- DDColor: Xiaoyang Kang, Tao Yang, Wenqi Ouyang, Peiran Ren, Lingzhi Li, Xuansong Xie. DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders. ICCV 2023. https://github.com/piddnad/DDColor (Apache-2.0)
	- ONNX conversions: rembg by Daniel Gatis (MIT), sidkonduru/gfpgan-v1.4-onnx, vlad-grig/realesr-general-x4v3-onnx.

	See `LICENSE` for the full licence texts.