| --- |
| license: mit |
| pipeline_tag: image-to-image |
| library_name: executorch |
| tags: |
| - inpainting |
| --- |
| |
| # migan |
|
|
| This repository hosts **MI-GAN** (image inpainting) exported for the |
| [React Native ExecuTorch](https://www.npmjs.com/package/react-native-executorch) |
| library as ExecuTorch `.pte` programs, ready to run on device. |
|
|
| Upstream model: [Picsart-AI-Research/MI-GAN](https://github.com/Picsart-AI-Research/MI-GAN), |
| revision `2b793c5ece43f4253e32d4afc257120a5deed6f5`, checkpoints |
| `migan_256_places2.pt`, `migan_512_places2.pt` (Places2, scenes) and |
| `migan_256_ffhq.pt` (FFHQ, faces). |
|
|
| ## Inputs and outputs |
|
|
| `forward(image, mask) -> image`, all `float32`: |
|
|
| - `image` `[1, 3, H, W]`, RGB in `[0, 1]` |
| - `mask` `[1, 1, H, W]`, `1` = known pixel, `0` = hole |
| - output `[1, 3, H, W]`, RGB in `[0, 1]`, the whole frame with the hole filled |
|
|
| `H = W = 256` or `512`. Cropping around the hole and blending the result back |
| into the full-resolution photo happen outside the model, as in upstream's |
| `scripts/create_onnx_pipeline.py`. |
|
|
| ## Variants |
|
|
| | Path | Backend | Precision | Checkpoint | |
| | --- | --- | --- | --- | |
| | `coreml/256/migan_256_coreml_fp16.pte` | coreml | fp16 | Places2 256 | |
| | `coreml/512/migan_512_coreml_fp16.pte` | coreml | fp16 | Places2 512 | |
| | `coreml/faces_256/migan_faces_256_coreml_fp16.pte` | coreml | fp16 | FFHQ 256 | |
| | `vulkan/256/migan_256_vulkan_fp16.pte` | vulkan | fp16 | Places2 256 | |
| | `vulkan/512/migan_512_vulkan_fp16.pte` | vulkan | fp16 | Places2 512 | |
| | `vulkan/faces_256/migan_faces_256_vulkan_fp16.pte` | vulkan | fp16 | FFHQ 256 | |
| | `xnnpack/256/migan_256_xnnpack_fp16.pte` | xnnpack | fp16 | Places2 256 | |
| | `xnnpack/512/migan_512_xnnpack_fp16.pte` | xnnpack | fp16 | Places2 512 | |
| | `xnnpack/faces_256/migan_faces_256_xnnpack_fp16.pte` | xnnpack | fp16 | FFHQ 256 | |
|
|
| ## Repository structure |
|
|
| ``` |
| config.json 27 B |
| coreml/256/config.json 1171 B |
| coreml/256/migan_256_coreml_fp16.pte 12.8 MB |
| coreml/512/config.json 1171 B |
| coreml/512/migan_512_coreml_fp16.pte 14.0 MB |
| coreml/faces_256/config.json 1177 B |
| coreml/faces_256/migan_faces_256_coreml_fp16.pte 12.8 MB |
| vulkan/256/config.json 1171 B |
| vulkan/256/migan_256_vulkan_fp16.pte 12.4 MB |
| vulkan/512/config.json 1171 B |
| vulkan/512/migan_512_vulkan_fp16.pte 14.1 MB |
| vulkan/faces_256/config.json 1177 B |
| vulkan/faces_256/migan_faces_256_vulkan_fp16.pte 12.4 MB |
| xnnpack/256/config.json 1173 B |
| xnnpack/256/migan_256_xnnpack_fp16.pte 24.0 MB |
| xnnpack/512/config.json 1173 B |
| xnnpack/512/migan_512_xnnpack_fp16.pte 25.2 MB |
| xnnpack/faces_256/config.json 1179 B |
| xnnpack/faces_256/migan_faces_256_xnnpack_fp16.pte 24.0 MB |
| ``` |
|
|
| ## Compatibility |
|
|
| These files are published for the **ExecuTorch v1.4.1** runtime. The XNNPACK |
| and Vulkan files need the React Native ExecuTorch native libraries |
| `v0.10.7-libs` or newer. ExecuTorch gives no forward compatibility guarantee, |
| so an older runtime may fail to load them. |
|
|
| For React Native ExecuTorch model loading, see the |
| [documentation](https://docs.swmansion.com/react-native-executorch/docs/fundamentals/downloading-models). |
| To load these files in your own runtime, read the |
| [compatibility note](https://github.com/pytorch/executorch/blob/main/runtime/COMPATIBILITY.md) |
| first. |
|
|
