File size: 2,824 Bytes
8f90b04 1a1735a 8f90b04 1a1735a 8f90b04 a08b39c 8f90b04 ecdb76b a08b39c 8f90b04 ecdb76b 8f90b04 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | ---
license: other
license_name: mixed
license_link: LICENSE.md
library_name: executorch
tags:
- executorch
- on-device
- image-restoration
- inpainting
- colorization
- super-resolution
---
# Yesteryear models
On-device ExecuTorch builds of the models used by **Yesteryear**, a private photo restoration
app. Photos are processed on the phone; these files are downloaded by the app the first time
a feature is used.
Each model is a re-export of a published open model, converted for
[ExecuTorch](https://github.com/pytorch/executorch) 1.4 with no retraining. `*_xnnpack.pte`
files run on the CPU (XNNPACK, float32) on Android and iOS; `*_coreml.pte` files run on iOS
(Core ML, float16). LaMa and DDColor ship as XNNPACK only: their float16 Core ML builds were
not accurate enough, so iOS uses the XNNPACK files for them.
| File | Model | Purpose | Upstream license |
|---|---|---|---|
| `realesrgan_x4v3_t256_{xnnpack,coreml}.pte` | [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) general x4v3 | 4x upscaling, 256 px tiles | BSD-3-Clause |
| `bopbtl_scratch_t384_{xnnpack,xnnpack_int8,coreml}.pte` | Scratch detection from [Bringing Old Photos Back to Life](https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life) | Finds scratches and tears, 384 px | MIT |
| `lama_big_t512_xnnpack.pte` | [LaMa](https://github.com/advimman/lama) (big-lama) | Inpaints the damage, 512 px tiles | Apache-2.0 |
| `ddcolor_paper_tiny_t512_xnnpack.pte` | [DDColor](https://github.com/piddnad/DDColor) tiny | Colorization, 512 px | Apache-2.0 |
Full license texts and copyright notices are in [`licenses/`](licenses). See
[`LICENSE.md`](LICENSE.md) for details and changes made.
## Changes from upstream
The networks and weights are unchanged. To run under ExecuTorch, some operations are rewritten
as mathematically equivalent forms (verified against the upstream PyTorch models):
- LaMa: the Fourier transform is computed as matrix multiplications against fixed DFT tables.
- DDColor: layer norm, attention and pooling are written with operations XNNPACK supports;
spectral normalization is folded into the weights.
- All: reflection padding is built from slices; inputs and outputs are wrapped for the app
(for example, LaMa takes RGB plus a mask channel and composites its own output).
## Notes
- `bopbtl_scratch_t384_xnnpack_int8.pte` uses 8-bit weights with dynamically quantized
activations (38 MB instead of 150 MB). Its scratch masks match the float model with IoU 0.95
to 0.98 on test photos; the app uses it on Android.
- The LaMa and DDColor weights were trained on Places365 and ImageNet respectively; those
datasets have their own terms of use for the images themselves.
- The Bringing Old Photos Back to Life repository states that its code and pretrained models
are MIT licensed.
|