Image-to-Image
PyTorch
ONNX
TensorRT
English
super-resolution
image-restoration
sisr
real-world-restoration
spandrel
chainner
transformer
attention
Instructions to use Phips/HEART with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- TensorRT
How to use Phips/HEART with TensorRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
add 'What we actually tested' section
Browse files
README.md
CHANGED
|
@@ -127,6 +127,27 @@ scripts/export_heart.py checkpoint → dynamic ONNX exporter
|
|
| 127 |
`examples/inputs/` also contains the original Real-ESRGAN test images, so you
|
| 128 |
can inspect full-size inputs and run the models yourself.
|
| 129 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 130 |
## Relation to NERVE
|
| 131 |
|
| 132 |
HEART and NERVE are the two members of the **BODY** suite (by Philip Hofmann):
|
|
|
|
| 127 |
`examples/inputs/` also contains the original Real-ESRGAN test images, so you
|
| 128 |
can inspect full-size inputs and run the models yourself.
|
| 129 |
|
| 130 |
+
## What we actually tested
|
| 131 |
+
|
| 132 |
+
Every design choice came from a measured experiment, not a guess. The full log
|
| 133 |
+
(including dead ends, benchmarks and the "considered and decided" register) is
|
| 134 |
+
in [`docs/EXPERIMENTS.md`](docs/EXPERIMENTS.md). Highlights:
|
| 135 |
+
|
| 136 |
+
- **HEART vs HAT_M** (30k, identical settings): effectively a tie on quality
|
| 137 |
+
(25.02 vs 25.05 dB, within noise) at **~half the peak VRAM** (1.44 vs
|
| 138 |
+
2.84 GB), and **~1.16-1.24x faster + ~2.7x less VRAM at inference in bf16**
|
| 139 |
+
(the deployment-relevant precision).
|
| 140 |
+
- **ATD**: OOM'd at iter 1 on a 12 GB card on the same config — a real
|
| 141 |
+
deployment result.
|
| 142 |
+
- **torch.compile**: ~1.5x training speed, quality-neutral (validated).
|
| 143 |
+
- **i-LN** (norm choice) for stable bf16 AMP with no fp32 gymnastics.
|
| 144 |
+
- **RIB** replaces HAT's relative-position-bias table + OCAB with a
|
| 145 |
+
FlashAttention-friendly dot product.
|
| 146 |
+
- **Shifted-window bug** found and fixed during ablations (it was quietly
|
| 147 |
+
costing quality).
|
| 148 |
+
- **OTF GAN lq96 window alignment**: diagnosed the bright band + tile grid and
|
| 149 |
+
fixed it via the `lq_size` multiple-of-32 rule.
|
| 150 |
+
|
| 151 |
## Relation to NERVE
|
| 152 |
|
| 153 |
HEART and NERVE are the two members of the **BODY** suite (by Philip Hofmann):
|