Image-to-Image
PyTorch
ONNX
TensorRT
English
super-resolution
image-restoration
sisr
real-world-restoration
spandrel
chainner
transformer
attention
Instructions to use Phips/HEART with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- TensorRT
How to use Phips/HEART with TensorRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
no unverified claims, CC0 data section, embedded examples
Browse files
README.md
CHANGED
|
@@ -16,9 +16,10 @@ pipeline_tag: image-to-image
|
|
| 16 |
|
| 17 |
**Hybrid Efficient Attention with Rank-factorized bias Transformer**
|
| 18 |
|
| 19 |
-
A super-resolution architecture that is simple to train
|
| 20 |
-
|
| 21 |
-
**
|
|
|
|
| 22 |
|
| 23 |
Part of the **BODY** suite by **Philip Hofmann** — networks built to be
|
| 24 |
real-world usable first, not benchmark-chasing. See
|
|
@@ -27,6 +28,12 @@ real-world usable first, not benchmark-chasing. See
|
|
| 27 |
> Short version: copy `heart_arch.py` into traiNNer-redux, train with
|
| 28 |
> `lq_size: 96`, export to ONNX, use anywhere. Details below.
|
| 29 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
---
|
| 31 |
|
| 32 |
## Quickstart
|
|
@@ -65,8 +72,11 @@ your training config (e.g. `models/heart_4x_pretrain.safetensors` for a 4x run).
|
|
| 65 |
|
| 66 |
### As a user (want to upscale images)
|
| 67 |
|
| 68 |
-
|
| 69 |
-
|
|
|
|
|
|
|
|
|
|
| 70 |
|
| 71 |
- Input: RGB, any size (dynamic H/W).
|
| 72 |
- Scale: 4x (`heart_4x_*`), 2x (`heart_2x`).
|
|
@@ -164,6 +174,16 @@ in [`docs/EXPERIMENTS.md`](docs/EXPERIMENTS.md). Highlights:
|
|
| 164 |
- **OTF GAN lq96 window alignment**: diagnosed the bright band + tile grid and
|
| 165 |
fixed it via the `lq_size` multiple-of-32 rule.
|
| 166 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 167 |
## Relation to NERVE
|
| 168 |
|
| 169 |
HEART and NERVE are the two members of the **BODY** suite (by Philip Hofmann):
|
|
|
|
| 16 |
|
| 17 |
**Hybrid Efficient Attention with Rank-factorized bias Transformer**
|
| 18 |
|
| 19 |
+
A super-resolution architecture that is simple to train and fast to run.
|
| 20 |
+
Released models are **verified** with **traiNNer-redux** (training) and
|
| 21 |
+
**ONNX Runtime** (inference); TensorRT and other runtimes are expected to work
|
| 22 |
+
(same standard op set) but are not tested here.
|
| 23 |
|
| 24 |
Part of the **BODY** suite by **Philip Hofmann** — networks built to be
|
| 25 |
real-world usable first, not benchmark-chasing. See
|
|
|
|
| 28 |
> Short version: copy `heart_arch.py` into traiNNer-redux, train with
|
| 29 |
> `lq_size: 96`, export to ONNX, use anywhere. Details below.
|
| 30 |
|
| 31 |
+

|
| 32 |
+
*Input (as-is) | bicubic x4 | HEART 4x release | HEART 4x OTF GAN.*
|
| 33 |
+
|
| 34 |
+

|
| 35 |
+

|
| 36 |
+
|
| 37 |
---
|
| 38 |
|
| 39 |
## Quickstart
|
|
|
|
| 72 |
|
| 73 |
### As a user (want to upscale images)
|
| 74 |
|
| 75 |
+
- **ONNX Runtime** (verified): run the `onnx/*.onnx` files (dynamic H/W).
|
| 76 |
+
TensorRT should also work (same ops) but is untested here.
|
| 77 |
+
- **chaiNNer**: load the `onnx/*.onnx` files directly. Loading the
|
| 78 |
+
`.safetensors` via spandrel needs HEART registered in your chaiNNer/spandrel
|
| 79 |
+
build (not upstream yet).
|
| 80 |
|
| 81 |
- Input: RGB, any size (dynamic H/W).
|
| 82 |
- Scale: 4x (`heart_4x_*`), 2x (`heart_2x`).
|
|
|
|
| 174 |
- **OTF GAN lq96 window alignment**: diagnosed the bright band + tile grid and
|
| 175 |
fixed it via the `lq_size` multiple-of-32 rule.
|
| 176 |
|
| 177 |
+
## Training data & license
|
| 178 |
+
|
| 179 |
+
All released HEART models were trained **only** on
|
| 180 |
+
[`Phips/lucid-cc0-v2-hc-512`](https://huggingface.co/datasets/Phips/lucid-cc0-v2-hc-512),
|
| 181 |
+
a **CC0** dataset — the released weights are clean to use commercially.
|
| 182 |
+
|
| 183 |
+
HEART ties HAT on quality at a much lower cost, but we did **not** chase the
|
| 184 |
+
benchmark leaderboard. If higher PSNR/LPIPS is your goal, training on larger
|
| 185 |
+
academic datasets (DIV2K, Flickr2K, LSDIR, ...) is expected to help.
|
| 186 |
+
|
| 187 |
## Relation to NERVE
|
| 188 |
|
| 189 |
HEART and NERVE are the two members of the **BODY** suite (by Philip Hofmann):
|