inYourOwnBrowser
/

inYourOwnBrowser Phips commited on
Commit
fb6c625
·
0 Parent(s):

Duplicate from Phips/PULSE

Browse files

Co-authored-by: Philip Hofmann <Phips@users.noreply.huggingface.co>

.gitattributes ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ examples/4x_pulse_urban100_img001_ab.png filter=lfs diff=lfs merge=lfs -text
37
+ examples/4x_pulse_urban100_img004_ab.png filter=lfs diff=lfs merge=lfs -text
38
+ examples/4x_pulse_urban100_img006_ab.png filter=lfs diff=lfs merge=lfs -text
39
+ onnx/2x_pulse.onnx.data filter=lfs diff=lfs merge=lfs -text
40
+ onnx/2x_pulse_fast.onnx.data filter=lfs diff=lfs merge=lfs -text
41
+ onnx/4x_pulse.onnx.data filter=lfs diff=lfs merge=lfs -text
42
+ onnx/4x_pulse_fast.onnx.data filter=lfs diff=lfs merge=lfs -text
43
+ examples/4x_pulse_urban100_img_001_input_vs_pretrain.png filter=lfs diff=lfs merge=lfs -text
44
+ examples/4x_pulse_urban100_img_004_input_vs_pretrain.png filter=lfs diff=lfs merge=lfs -text
45
+ examples/4x_pulse_urban100_img_006_input_vs_pretrain.png filter=lfs diff=lfs merge=lfs -text
ATTRIBUTION.md ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Attribution
2
+
3
+ PULSE uses a fixed **Magic Kernel Sharp 2013 (MKS2013)** base upsampler.
4
+
5
+ - Algorithm and original reference: **John Costella**, *Magic Kernel Sharp*,
6
+ https://johncostella.com/magic/
7
+ - Public reference implementation: `felixr/magic-kernel-sharp` (MIT),
8
+ https://github.com/felixr/magic-kernel-sharp
9
+
10
+ PULSE contains an independent PyTorch implementation of the integer-scale
11
+ MKS2013 phase kernels. It is not a copy of the reference C implementation.
12
+ The formula and the MIT-licensed reference are attributed here for clarity.
13
+
14
+ The PULSE release is MIT-licensed. This notice is attribution, not a legal
15
+ opinion; verify the current upstream terms before redistribution.
LICENSE ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Philip Hofmann
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
PAPER.md ADDED
@@ -0,0 +1,245 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # PULSE technical report
2
+
3
+ *Private draft. Companion to the model card.*
4
+
5
+
6
+ # PULSE
7
+
8
+ **A tiny, deployment-first super-resolution network.**
9
+
10
+ PULSE is the **tiny/fast sibling of NERVE**: a much smaller convolutional SISR
11
+ model with the same deployment-first philosophy. It trades capacity for size and
12
+ speed, so it is less capable than NERVE and is intended for light cleanup and
13
+ clean upscaling rather than heavy restoration. It trains on a consumer GPU,
14
+ converts without surprises, and embeds almost anywhere.
15
+
16
+ - `pulse`: 48 channels, **353,856 parameters**
17
+ - `pulse_fast`: 32 channels, **162,192 parameters**
18
+ - 2x and 4x models
19
+ - PyTorch `.safetensors` + dynamic ONNX
20
+ - **MIT** code and weights, trained on **CC0 / public-domain images only**
21
+
22
+ **Links:** [HEART](https://huggingface.co/Phips/HEART) · [NERVE](https://huggingface.co/Phips/NERVE) · [PULSE demo](https://huggingface.co/spaces/Phips/PULSE-demo) · [PULSE ONNX converter](https://huggingface.co/spaces/Phips/PULSE-convert)
23
+
24
+ > Short version: load a released `.safetensors` with `pulse_arch.py`, or an
25
+ > `.onnx` with ONNX Runtime. Start with 2x for browser/CPU use.
26
+
27
+ ---
28
+
29
+ ## Why PULSE
30
+
31
+ PULSE is not different for the sake of being different. Each constraint removes
32
+ a real deployment or maintenance problem.
33
+
34
+ - **No `Resize` op in the exported graph.** The scale is done by a fixed MKS2013
35
+ base plus PixelShuffle. ONNX, TensorRT, NCNN, and other runtimes all see the
36
+ same standard graph; there is no interpolation implementation mismatch.
37
+ - **Only standard ops.** `Conv / Add / ReLU / Pad / DepthToSpace`. No custom
38
+ kernels, no fused/unfused checkpoint pair, no fragile converter step. That
39
+ makes it easy to audit, port, optimize with `onnxsim`, and compile with
40
+ TensorRT.
41
+ - **Fixed MKS2013 base.** The base upsampler has no learned parameters and no
42
+ dynamic branch. It anchors the output, reduces what the network must learn,
43
+ and keeps the graph compact and deterministic.
44
+ - **No attention and no normalization.** That means stable bf16 AMP training,
45
+ fewer memory spikes, and simpler maintenance. The whole model is readable.
46
+ - **Tiny.** 162K–354K parameters. It fits browser/WASM budgets, small GPUs,
47
+ CPU fallbacks, and embedded pipelines where a transformer does not.
48
+ - **Fast training.** The pretrain uses Charbonnier only: no discriminator, no
49
+ perceptual stack, no fragile GAN schedule. It trains on a single RTX 3060.
50
+ - **Dynamic ONNX.** One model works at any H/W; no per-resolution exports.
51
+
52
+ ## What it is for
53
+
54
+ PULSE is a **local, low-cost upscaler**. The released pretrains are clean
55
+ 2x/4x models, not a general blind-restoration model.
56
+
57
+ - **Browser or desktop upscaling.** ONNX Runtime Web/WASM/WebGPU can run the 2x
58
+ models; 4x is heavier but still small.
59
+ - **Image viewers, galleries, thumbnail pipelines.** A fast local pass before
60
+ display, with no server round-trip.
61
+ - **Anime and illustration stills.** Clean 2x/4x upscaling; a compression-trained
62
+ variant is a separate model.
63
+ - **Batch/offline preprocessing.** Small enough to share a consumer GPU or CPU.
64
+ - **Embedding in apps.** One architecture file, one op set, no fuse step.
65
+
66
+ Compression cleanup is a real use case, but it needs the separate
67
+ **PULSE-Web** finetune. The released clean pretrain was not trained on
68
+ JPEG/WebP/AVIF/H.265/GIF artifacts.
69
+
70
+ A temporal/video cleaner is **not part of this release**. Per-frame PULSE can
71
+ be a component, but flicker-free video needs temporal conditioning.
72
+
73
+ ## When to use NERVE or HEART instead
74
+
75
+ | Need | Use |
76
+ |---|---|
77
+ | Browser/CPU, tiny model, fast 2x/4x | **PULSE** |
78
+ | Better quality with a ~1.8M conv-only edge model | **NERVE** |
79
+ | Highest quality / desktop-server GPU budget | **HEART** |
80
+ | Blind OTF restoration, heavy blur/noise | NERVE/HEART OTF or a larger restoration model |
81
+ | Text/UI/screenshot upscaling | a text-aware restoration model, not PULSE |
82
+ | Temporal video restoration | a temporal model; PULSE is per-frame only |
83
+
84
+ PULSE is the **tiny/fast NERVE sibling**: the same practical philosophy, but
85
+ far fewer parameters and a lighter operating envelope. NERVE is the more
86
+ capable restoration model; PULSE is for light degradation, clean upscaling,
87
+ browser/CPU use, and cases where the model itself must be almost free.
88
+
89
+ PULSE is not competing with HEART on PSNR.
90
+
91
+ ## Pick a model
92
+
93
+ | Want | Use | Scale | Params |
94
+ |---|---|---:|---:|
95
+ | Best small-model quality | `pulse` | 4x / 2x | 353,856 |
96
+ | Fastest / lowest compute | `pulse_fast` | 4x / 2x | 162,192 |
97
+ | CPU / ORT / engine | `onnx/*_fp16.onnx` | 4x / 2x | same |
98
+
99
+ All released models are pretrained only on **CC0 / public-domain images**. They
100
+ are MIT-licensed and permissive.
101
+
102
+ ### Urban100 validation
103
+
104
+ | Model | PSNR | SSIM | TopIQ | LPIPS |
105
+ |---|---:|---:|---:|---:|
106
+ | 4x `pulse` | **25.6523** | **0.7717** | **0.5907** | **0.2452** |
107
+ | 2x `pulse` | **31.4423** | **0.9221** | **0.8614** | **0.0721** |
108
+ | 4x `pulse_fast` | **25.4239** | **0.7638** | **0.5748** | **0.2553** |
109
+ | 2x `pulse_fast` | **31.1371** | **0.9188** | **0.8551** | **0.0755** |
110
+
111
+ EMA weights, standard Urban100 validation. PSNR/SSIM are Y-channel with a 4px border crop; TopIQ/LPIPS are RGB. No TTA.
112
+
113
+ ### Inference speed
114
+
115
+ RTX 3060, batch 1, fp16. TensorRT engine built locally.
116
+
117
+ | Model | 256² eager | 720p eager | 1080p eager | 256² TRT | 720p TRT | 1080p TRT |
118
+ |---|---:|---:|---:|---:|---:|---:|
119
+ | `pulse_fast` | 3.65 | 47.00 | 103.88 | 1.43 | 17.52 | 39.60 |
120
+ | `pulse` | 6.71 | 90.20 | 199.28 | 2.82 | 35.35 | 79.65 |
121
+
122
+ ### Training speed
123
+
124
+ CC0/public-domain 200k fidelity pretrain, RTX 3060, batch 8, bf16:
125
+
126
+ - `pulse` 4x: ~29 it/s
127
+ - `pulse` 2x: ~9.8 it/s
128
+ - `pulse_fast` 4x: ~36 it/s
129
+ - `pulse_fast` 2x: ~20 it/s
130
+
131
+ Training is Charbonnier-only. Perceptual branches are experimental and are not
132
+ part of the pretrain release.
133
+
134
+ ## Quickstart
135
+
136
+ ### ONNX Runtime
137
+
138
+ ```python
139
+ import onnxruntime as ort
140
+ sess = ort.InferenceSession("onnx/4x_pulse_fp16.onnx", providers=["CPUExecutionProvider"])
141
+ # input: float32 [1, 3, H, W] in [0, 1], RGB, dynamic H/W
142
+ ```
143
+
144
+ The fp16 file is self-contained. Use `4x_pulse.onnx` if your runtime lacks fp16.
145
+
146
+ ### PyTorch / spandrel
147
+
148
+ 1. Copy `pulse_arch.py` into `traiNNer/archs/` (or use the file here).
149
+ 2. Load the `.safetensors` with spandrel once PULSE is registered.
150
+ 3. Input RGB `[0,1]`, output 2x/4x.
151
+
152
+ ### Train
153
+
154
+ ```bash
155
+ cp pulse_arch.py <traiNNer-redux>/traiNNer/archs/
156
+ cp configs/4x_pulse_pretrain.yml <traiNNer-redux>/options/
157
+ cd <traiNNer-redux>
158
+ python train.py -opt options/4x_pulse_pretrain.yml
159
+ ```
160
+
161
+ The sample configs use placeholder data paths and can be trained from scratch.
162
+ The released pretrains are provided as warm starts.
163
+
164
+ ## Visual examples
165
+
166
+ Urban100 4x. Left: bicubic input. Right: released PULSE 4x pretrain.
167
+
168
+ ![Urban100 example 1](examples/4x_pulse_urban100_img_001_input_vs_pretrain.png)
169
+ ![Urban100 example 2](examples/4x_pulse_urban100_img_004_input_vs_pretrain.png)
170
+ ![Urban100 example 3](examples/4x_pulse_urban100_img_006_input_vs_pretrain.png)
171
+
172
+ ## Files
173
+
174
+ ```text
175
+ pulse_arch.py
176
+ models/
177
+ pulse/4x_pretrain.safetensors
178
+ pulse/2x_pretrain.safetensors
179
+ pulse_fast/4x_pretrain.safetensors
180
+ pulse_fast/2x_pretrain.safetensors
181
+ onnx/
182
+ *_fp16.onnx
183
+ *.onnx
184
+ configs/
185
+ 4x_pulse_pretrain.yml
186
+ 2x_pulse_pretrain.yml
187
+ 4x_pulse_fast_pretrain.yml
188
+ 2x_pulse_fast_pretrain.yml
189
+ PAPER.md
190
+ examples/
191
+ ```
192
+
193
+ ## Limitations
194
+
195
+ - Not a blind-OTF model; no degradation embedding.
196
+ - Not a text/UI upscaler.
197
+ - The clean pretrain does not clean compression artifacts; use a web/anime
198
+ finetune for that.
199
+ - No temporal/video model is included.
200
+ - PSNR is below HEART; that is the intended size/speed tradeoff.
201
+
202
+ ## Attribution
203
+
204
+ The fixed base uses **Magic Kernel Sharp 2013** by **John Costella**
205
+ (https://johncostella.com/magic/). The public reference implementation
206
+ (`felixr/magic-kernel-sharp`, MIT) is credited in `ATTRIBUTION.md`. PULSE uses
207
+ an independent PyTorch implementation of the integer-scale phase kernels, not
208
+ a copy of the reference C code.
209
+
210
+ ## License
211
+
212
+ The code and released weights are **MIT-licensed** and permissive. Keep the
213
+ MIT copyright and permission notice with copies or substantial portions of the
214
+ software. See `LICENSE`.
215
+
216
+ The released weights were trained on a curated CC0 / public-domain image set.
217
+ This is a provenance statement, not a guarantee; source-image rights and
218
+ jurisdiction-specific rules can still matter. No warranty is provided.
219
+
220
+ ```bibtex
221
+ @misc{pulse_body_2026,
222
+ title = {PULSE: A Deployment-First Single-Image Super-Resolution Network},
223
+ author = {Philip Hofmann and contributors},
224
+ year = {2026},
225
+ note = {BODY suite}
226
+ }
227
+ ```
228
+
229
+
230
+ ---
231
+
232
+ ## Appendix: ablation context
233
+
234
+ PULSE was selected through equal-budget internal experiments. The published
235
+ weights are the 200k pretrains trained only on CC0/public-domain images.
236
+
237
+ - 48 channels / 8 blocks beat 32 channels / 8 blocks on quality at 2.2x the
238
+ parameters; both keep the same deployment graph.
239
+ - MKS2013 and bicubic bases tied on quality; MKS was kept because it removes
240
+ `Resize` from ONNX.
241
+ - RepConv/PULSE-R gained PSNR but cost 2.1x training time, 3.3x training VRAM,
242
+ 10x training parameters, and a fuse step; rejected.
243
+ - Partial depthwise, 5x5 stem, and tiny variants were rejected.
244
+ - Perceptual/compression branches are separate and are not part of this
245
+ pretrain release.
README.md ADDED
@@ -0,0 +1,318 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - en
5
+ library_name: pytorch
6
+ pipeline_tag: image-to-image
7
+ tags:
8
+ - super-resolution
9
+ - image-restoration
10
+ - image-to-image
11
+ - sisr
12
+ - pytorch
13
+ - onnx
14
+ - onnxruntime
15
+ - tensorrt
16
+ - spandrel
17
+ - chainner
18
+ - edge
19
+ - deployment
20
+ - mit
21
+ ---
22
+
23
+ # PULSE
24
+
25
+ **A tiny, deployment-first super-resolution network.**
26
+
27
+ PULSE is the **tiny/fast sibling of NERVE**: a much smaller convolutional SISR
28
+ model with the same deployment-first philosophy. It trades capacity for size and
29
+ speed, so it is less capable than NERVE and is intended for light cleanup and
30
+ clean upscaling rather than heavy restoration. It trains on a consumer GPU,
31
+ converts without surprises, and embeds almost anywhere.
32
+
33
+ - `pulse`: 48 channels, **353,856 parameters**
34
+ - `pulse_fast`: 32 channels, **162,192 parameters**
35
+ - 2x and 4x models
36
+ - PyTorch `.safetensors` + dynamic ONNX
37
+ - **MIT** code and weights, trained on **CC0 / public-domain images only**
38
+
39
+ **Links:** [HEART](https://huggingface.co/Phips/HEART) · [NERVE](https://huggingface.co/Phips/NERVE) · [PULSE demo](https://huggingface.co/spaces/Phips/PULSE-demo) · [PULSE ONNX converter](https://huggingface.co/spaces/Phips/PULSE-convert)
40
+
41
+ > Short version: load a released `.safetensors` with `pulse_arch.py`, or an
42
+ > `.onnx` with ONNX Runtime. Start with 2x for browser/CPU use.
43
+
44
+ ---
45
+
46
+ ## Why PULSE
47
+
48
+ PULSE is not different for the sake of being different. Each constraint removes
49
+ a real deployment or maintenance problem.
50
+
51
+ - **No `Resize` op in the exported graph.** The scale is done by a fixed MKS2013
52
+ base plus PixelShuffle. ONNX, TensorRT, NCNN, and other runtimes all see the
53
+ same standard graph; there is no interpolation implementation mismatch.
54
+ - **Only standard ops.** `Conv / Add / ReLU / Pad / DepthToSpace`. No custom
55
+ kernels, no fused/unfused checkpoint pair, no fragile converter step. That
56
+ makes it easy to audit, port, optimize with `onnxsim`, and compile with
57
+ TensorRT.
58
+ - **Fixed MKS2013 base.** The base upsampler has no learned parameters and no
59
+ dynamic branch. It anchors the output, reduces what the network must learn,
60
+ and keeps the graph compact and deterministic.
61
+ - **No attention and no normalization.** That means stable bf16 AMP training,
62
+ fewer memory spikes, and simpler maintenance. The whole model is readable.
63
+ - **Tiny.** 162K–354K parameters. It fits browser/WASM budgets, small GPUs,
64
+ CPU fallbacks, and embedded pipelines where a transformer does not.
65
+ - **Fast training.** The pretrain uses Charbonnier only: no discriminator, no
66
+ perceptual stack, no fragile GAN schedule. It trains on a single RTX 3060.
67
+ - **Dynamic ONNX.** One model works at any H/W; no per-resolution exports.
68
+
69
+ ## What it is for
70
+
71
+ PULSE is a **local, low-cost upscaler**. The released pretrains are clean
72
+ 2x/4x models, not a general blind-restoration model.
73
+
74
+ - **Browser or desktop upscaling.** ONNX Runtime Web/WASM/WebGPU can run the 2x
75
+ models; 4x is heavier but still small.
76
+ - **Image viewers, galleries, thumbnail pipelines.** A fast local pass before
77
+ display, with no server round-trip.
78
+ - **Anime and illustration stills.** Clean 2x/4x upscaling; a compression-trained
79
+ variant is a separate model.
80
+ - **Batch/offline preprocessing.** Small enough to share a consumer GPU or CPU.
81
+ - **Embedding in apps.** One architecture file, one op set, no fuse step.
82
+
83
+ Compression cleanup is a real use case, but it needs the separate
84
+ **PULSE-Web** finetune. The released clean pretrain was not trained on
85
+ JPEG/WebP/AVIF/H.265/GIF artifacts.
86
+
87
+ A temporal/video cleaner is **not part of this release**. Per-frame PULSE can
88
+ be a component, but flicker-free video needs temporal conditioning.
89
+
90
+ ## When to use NERVE or HEART instead
91
+
92
+ | Need | Use |
93
+ |---|---|
94
+ | Browser/CPU, tiny model, fast 2x/4x | **PULSE** |
95
+ | Better quality / heavier restoration with a ~1.8M conv-only edge model | **NERVE** |
96
+ | Highest quality / desktop-server GPU budget | **HEART** |
97
+ | Blind OTF restoration, heavy blur/noise | NERVE/HEART OTF or a larger restoration model |
98
+ | Tiny/fast NERVE-like model for light cleanup | **PULSE** |
99
+ | Text/UI/screenshot upscaling | a text-aware restoration model, not PULSE |
100
+ | Temporal video restoration | a temporal model; PULSE is per-frame only |
101
+
102
+ PULSE is the **tiny/fast NERVE sibling**: the same practical philosophy, but
103
+ far fewer parameters and a lighter operating envelope. NERVE is the more
104
+ capable restoration model; PULSE is for light degradation, clean upscaling,
105
+ browser/CPU use, and cases where the model itself must be almost free.
106
+
107
+ PULSE is not competing with HEART on PSNR.
108
+
109
+ ## Pick a model
110
+
111
+ | Want | Use | Scale | Params |
112
+ |---|---|---:|---:|
113
+ | Best small-model quality | `pulse` | 4x / 2x | 353,856 |
114
+ | Fastest / lowest compute | `pulse_fast` | 4x / 2x | 162,192 |
115
+ | CPU / ORT / engine | `onnx/*_fp16.onnx` | 4x / 2x | same |
116
+
117
+ All released models are pretrained only on **CC0 / public-domain images**. They
118
+ are MIT-licensed and permissive.
119
+
120
+ ### Urban100 validation
121
+
122
+ | Model | PSNR | SSIM | TopIQ | LPIPS |
123
+ |---|---:|---:|---:|---:|
124
+ | 4x `pulse` | **25.6523** | **0.7717** | **0.5907** | **0.2452** |
125
+ | 2x `pulse` | **31.4423** | **0.9221** | **0.8614** | **0.0721** |
126
+ | 4x `pulse_fast` | **25.4239** | **0.7638** | **0.5748** | **0.2553** |
127
+ | 2x `pulse_fast` | **31.1371** | **0.9188** | **0.8551** | **0.0755** |
128
+
129
+ EMA weights, standard Urban100 validation. PSNR/SSIM are Y-channel with a 4px border crop; TopIQ/LPIPS are RGB. No TTA.
130
+
131
+ ### Inference speed
132
+
133
+ RTX 3060, batch 1, fp16. TensorRT engine built locally.
134
+
135
+ | Model | 256² eager | 720p eager | 1080p eager | 256² TRT | 720p TRT | 1080p TRT |
136
+ |---|---:|---:|---:|---:|---:|---:|
137
+ | `pulse_fast` | 3.65 | 47.00 | 103.88 | 1.43 | 17.52 | 39.60 |
138
+ | `pulse` | 6.71 | 90.20 | 199.28 | 2.82 | 35.35 | 79.65 |
139
+
140
+ ### Training speed
141
+
142
+ CC0/public-domain 200k fidelity pretrain, RTX 3060, batch 8, bf16:
143
+
144
+ - `pulse` 4x: ~29 it/s
145
+ - `pulse` 2x: ~9.8 it/s
146
+ - `pulse_fast` 4x: ~36 it/s
147
+ - `pulse_fast` 2x: ~20 it/s
148
+
149
+ Training is Charbonnier-only. Perceptual branches are experimental and are not
150
+ part of the pretrain release.
151
+
152
+ ## Quickstart
153
+
154
+ ### ONNX Runtime
155
+
156
+ ```python
157
+ import onnxruntime as ort
158
+ sess = ort.InferenceSession("onnx/4x_pulse_fp16.onnx", providers=["CPUExecutionProvider"])
159
+ # input: float32 [1, 3, H, W] in [0, 1], RGB, dynamic H/W
160
+ ```
161
+
162
+ The fp16 file is self-contained. Use `4x_pulse.onnx` if your runtime lacks fp16.
163
+
164
+ ### PyTorch / spandrel
165
+
166
+ 1. Copy `pulse_arch.py` into `traiNNer/archs/` (or use the file here).
167
+ 2. Load the `.safetensors` with spandrel once PULSE is registered.
168
+ 3. Input RGB `[0,1]`, output 2x/4x.
169
+
170
+ ### Train
171
+
172
+ ```bash
173
+ cp pulse_arch.py <traiNNer-redux>/traiNNer/archs/
174
+ cp configs/4x_pulse_pretrain.yml <traiNNer-redux>/options/
175
+ cd <traiNNer-redux>
176
+ python train.py -opt options/4x_pulse_pretrain.yml
177
+ ```
178
+
179
+ The sample configs use placeholder data paths and can be trained from scratch.
180
+ The released pretrains are provided as warm starts.
181
+
182
+ ## Visual examples
183
+
184
+ Urban100 4x. Left: bicubic input. Right: released PULSE 4x pretrain.
185
+
186
+ ![Urban100 example 1](examples/4x_pulse_urban100_img_001_input_vs_pretrain.png)
187
+ ![Urban100 example 2](examples/4x_pulse_urban100_img_004_input_vs_pretrain.png)
188
+ ![Urban100 example 3](examples/4x_pulse_urban100_img_006_input_vs_pretrain.png)
189
+
190
+ ## Files
191
+
192
+ ```text
193
+ pulse_arch.py
194
+ models/
195
+ pulse/4x_pretrain.safetensors
196
+ pulse/2x_pretrain.safetensors
197
+ pulse_fast/4x_pretrain.safetensors
198
+ pulse_fast/2x_pretrain.safetensors
199
+ onnx/
200
+ *_fp16.onnx
201
+ *.onnx
202
+ configs/
203
+ 4x_pulse_pretrain.yml
204
+ 2x_pulse_pretrain.yml
205
+ 4x_pulse_fast_pretrain.yml
206
+ 2x_pulse_fast_pretrain.yml
207
+ PAPER.md
208
+ examples/
209
+ ```
210
+
211
+ ## Limitations
212
+
213
+ - Not a blind-OTF model; no degradation embedding.
214
+ - Not a text/UI upscaler.
215
+ - The clean pretrain does not clean compression artifacts; use a web/anime
216
+ finetune for that.
217
+ - No temporal/video model is included.
218
+ - PSNR is below HEART; that is the intended size/speed tradeoff.
219
+
220
+ ## Attribution
221
+
222
+ The fixed base uses **Magic Kernel Sharp 2013** by **John Costella**
223
+ (https://johncostella.com/magic/). The public reference implementation
224
+ (`felixr/magic-kernel-sharp`, MIT) is credited in `ATTRIBUTION.md`. PULSE uses
225
+ an independent PyTorch implementation of the integer-scale phase kernels, not
226
+ a copy of the reference C code.
227
+
228
+ ## License
229
+
230
+ The code and released weights are **MIT-licensed** and permissive. Keep the
231
+ MIT copyright and permission notice with copies or substantial portions of the
232
+ software. See `LICENSE`.
233
+
234
+ The released weights were trained on a curated CC0 / public-domain image set.
235
+ This is a provenance statement, not a guarantee; source-image rights and
236
+ jurisdiction-specific rules can still matter. No warranty is provided.
237
+
238
+ ```bibtex
239
+ @misc{pulse_body_2026,
240
+ title = {PULSE: A Deployment-First Single-Image Super-Resolution Network},
241
+ author = {Philip Hofmann and contributors},
242
+ year = {2026},
243
+ note = {BODY suite}
244
+ }
245
+ ```
246
+
247
+ ---
248
+
249
+ ---
250
+
251
+ ## BODY suite benchmark — 2026-11-09
252
+
253
+ The **BODY suite** is a set of networks with different deployment goals, not one
254
+ network scaled up and down:
255
+
256
+ - **PULSE_fast** — smallest and fastest, 2x/4x.
257
+ - **PULSE** — ultralight quality step, 2x/4x.
258
+ - **NERVE** — lightweight ~1.8M conv-only edge model.
259
+ - **HEART** — heavier ~16.7M quality model for desktop/workstation GPUs.
260
+
261
+ All numbers use the latest released **CC0 / public-domain pretrains**.
262
+ Validation: [lucid-val-512](https://huggingface.co/datasets/Phips/lucid-val-512),
263
+ full images, Y-channel PSNR/SSIM with a 4px crop, RGB TopIQ/LPIPS/DISTS, EMA,
264
+ no TTA. All models are **4x**.
265
+
266
+ ### Quality
267
+
268
+ | Model | Params | Urban100 PSNR / SSIM / TopIQ / LPIPS / DISTS | lucid-val-512 | BHI100 |
269
+ |---|---:|---|---|---|
270
+ | PULSE_fast | 162,192 | 25.4263 / 0.7640 / 0.5749 / 0.2558 / 0.1866 | 28.1909 / 0.7807 / 0.5674 / 0.2825 / 0.1886 | 23.3966 / 0.6801 / 0.4917 / 0.3472 / 0.2275 |
271
+ | PULSE | 353,856 | 25.6559 / 0.7719 / 0.5908 / 0.2456 / 0.1821 | 28.2892 / 0.7842 / 0.5777 / 0.2782 / 0.1859 | 23.5135 / 0.6877 / 0.5041 / 0.3363 / 0.2236 |
272
+ | NERVE | 1,801,920 | 26.2685 / 0.7912 / 0.6230 / 0.2259 / 0.1735 | 28.5381 / 0.7919 / 0.5933 / 0.2710 / 0.1815 | 23.8057 / 0.7052 / 0.5275 / 0.3180 / 0.2156 |
273
+ | HEART | 16,677,399 | **28.0458 / 0.8372 / 0.7055 / 0.1758 / 0.1499** | **29.0150 / 0.8073 / 0.6268 / 0.2484 / 0.1698** | **24.4877 / 0.7419 / 0.5820 / 0.2724 / 0.1945** |
274
+
275
+ ### Training speed
276
+
277
+ 1k iterations, Charbonnier only, lq96, batch 2, bf16, sequential, RTX 3060.
278
+
279
+ | Model | it/s | Peak VRAM |
280
+ |---|---:|---:|
281
+ | PULSE_fast | 107.59 | 0.05 GB |
282
+ | PULSE | 126.99 | 0.06 GB |
283
+ | NERVE | 54.49 | 0.16 GB |
284
+ | HEART | 1.137 | 1.03 GB |
285
+
286
+ Training is **not strictly monotonic** with size: at this tiny scale, data/launch
287
+ and memory overhead can outweigh the parameter difference, so PULSE is slightly
288
+ faster than PULSE_fast in this 1k microbenchmark even though PULSE_fast is faster
289
+ at inference.
290
+
291
+ ### Inference — full lucid-val-512
292
+
293
+ 136 images, 128×128 input → 512×512 output, batch 1, no TTA, RTX 3060.
294
+ ONNX is static 128² fp16 + onnxsim, ORT CUDA; TensorRT is native fp16.
295
+ Peak GPU memory includes the CUDA/desktop baseline (~1.3 GB).
296
+
297
+ | Model | Runtime | Total s | images/s | ms/img | Peak GPU MB |
298
+ |---|---|---:|---:|---:|---:|
299
+ | PULSE_fast | eager | 0.151 | 900.9 | 1.11 | 1316 |
300
+ | PULSE_fast | ORT CUDA | 0.190 | 717.6 | 1.39 | 1290 |
301
+ | PULSE_fast | TensorRT | **0.042** | **3249.8** | **0.308** | 1340 |
302
+ | PULSE | eager | 0.402 | 338.1 | 2.96 | 1336 |
303
+ | PULSE | ORT CUDA | 0.329 | 413.7 | 2.42 | 1298 |
304
+ | PULSE | TensorRT | **0.078** | **1740.5** | **0.575** | 1350 |
305
+ | NERVE | eager | 1.064 | 127.8 | 7.83 | 1485 |
306
+ | NERVE | ORT CUDA | 0.946 | 143.8 | 6.95 | 1317 |
307
+ | NERVE | TensorRT | **0.259** | **525.8** | **1.90** | 1374 |
308
+ | HEART | eager | 48.444 | 2.81 | 356.2 | 10454 |
309
+ | HEART | ORT CUDA | 28.706 | 4.74 | 211.1 | 5508 |
310
+ | HEART | TensorRT | **10.149** | **13.40** | **74.63** | 5779 |
311
+
312
+ `torch.compile` was not a win on the full lucid run for these four models and is
313
+ therefore omitted from the summary.
314
+
315
+ **Links:** [PULSE](https://huggingface.co/Phips/PULSE) ·
316
+ [NERVE](https://huggingface.co/Phips/NERVE) ·
317
+ [HEART](https://huggingface.co/Phips/HEART) ·
318
+ [lucid-val-512](https://huggingface.co/datasets/Phips/lucid-val-512)
checksums.txt ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ 24afac4c03337316f37e1c152369eea8c2960227063970d0fdd89dab8ec3c15c ATTRIBUTION.md
2
+ d16d4d07fd524bf5fb633e6677adee564cfecfcbd49df1870eec946334815f50 LICENSE
3
+ 8b20946732f997a51f8758594f85f40215a34abaea75069f156a6139e737f307 PAPER.md
4
+ b3ce65f754c9c60f65d59f2e67d60eac327c911e1873b0e7d8b15c15ba4e2a5a README.md
5
+ 4aef8acc39c8a83e2fb32232320f1a82ee0b272d09507fe27b1a78c761e1e100 configs/2x_pulse_fast_pretrain.yml
6
+ dc4e69f32f180229fddaffbaec1110ad7235f743f598b7f0c4416c25a1f4d5a5 configs/2x_pulse_pretrain.yml
7
+ ab6bcfb92d4243e23a6b84d706f61a3cab0deab14a68933a6fc544e167459a79 configs/4x_pulse_fast_pretrain.yml
8
+ a8360fbf7d6777844f33fb49eafabf54f0d08c7e49fa26280661ee98cca2d498 configs/4x_pulse_pretrain.yml
9
+ 0026397ecef86da75f26861275edaa73b408bba3611bfc293ba62613be1f8cc2 configs/pulse_arch.py
10
+ 7050bea3e3449816d89dd44fb9be367e92db877313bf9952f232317387a6c06d examples/4x_pulse_urban100_img_001_input_vs_pretrain.png
11
+ 664410e9af4ab829fbc000a51cb7cf9335fc9e4915e994603c37f9a2fcf3c962 examples/4x_pulse_urban100_img_004_input_vs_pretrain.png
12
+ ca6b1ff8143503b2a87f327beb6fcb60605c719aa8195e346506cc1d5c410b64 examples/4x_pulse_urban100_img_006_input_vs_pretrain.png
13
+ 3fd891ff8113ac7caf43de985a2759d915d3a6626d50e1db04bc55865e5b5e13 models/pulse/2x_pretrain.safetensors
14
+ 15ac6f2b958b016ad26f1a55700b6530ceeeb08feea920fdf6385029176f2980 models/pulse/4x_pretrain.safetensors
15
+ 2957d4af0069001509ad3222a3dfbad97290b120a40916221d3cc6547097c4b4 models/pulse_fast/2x_pretrain.safetensors
16
+ fa9e477807eb65c7340aeadbee374213853eb3c2fe06210898bae910e6f654a8 models/pulse_fast/4x_pretrain.safetensors
17
+ baeeb9f310aa4dfad5f3a5f7961254270ba09256d6fb49fbcb4b2b5e63c34251 onnx/2x_pulse.onnx
18
+ 66c8905a7671901623a1f74333a259c39e4b6238069b1fc2c64036672d8421fc onnx/2x_pulse_fast.onnx
19
+ 22c277e7cde5d864b8a8ff41d3a74d1afc41ef08ceb4b5349ca272ad525913e8 onnx/2x_pulse_fast_fp16.onnx
20
+ 765559129d740c989027ca28d0c7bbd12c1aadd9411aa1e44aa726950cb35352 onnx/2x_pulse_fp16.onnx
21
+ c616de7389281ad0a5cb9fa09c5ca13b4786f72a33ddf1535c0abefb6893a766 onnx/4x_pulse.onnx
22
+ b2c9703ee9b4c82278599c38f6389495944c769bc43a64221c77a3f906ef5cb5 onnx/4x_pulse_fast.onnx
23
+ c62531bd9f49b402234c861c6dbfa81ff29edee4b547d1522f928be0f41bf3bd onnx/4x_pulse_fast_fp16.onnx
24
+ 185b51606ba170e9feb207c4778c75e5c21f81b4e5b22c749de9a9a1bd5dbfc3 onnx/4x_pulse_fp16.onnx
configs/2x_pulse_fast_pretrain.yml ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: 2x_pulse_fast_PULSE_fast_pretrain
2
+ scale: 2
3
+ use_amp: true
4
+ amp_bf16: true
5
+ use_channels_last: true
6
+ num_gpu: 1
7
+ datasets:
8
+ train:
9
+ name: CC0/public-domain train
10
+ type: pairedimagedataset
11
+ dataroot_gt:
12
+ - datasets/train/cc0/gt
13
+ dataroot_lq:
14
+ - datasets/train/cc0/lr_x2
15
+ lq_size: 256
16
+ use_hflip: true
17
+ use_rot: true
18
+ num_worker_per_gpu: 8
19
+ batch_size_per_gpu: 8
20
+ accum_iter: 1
21
+ val:
22
+ name: Urban100 x4
23
+ type: pairedimagedataset
24
+ dataroot_gt:
25
+ - datasets/val/urban100/hr
26
+ dataroot_lq:
27
+ - datasets/val/urban100/lr_x2
28
+ filename_tmpl: '{}x2'
29
+ network_g:
30
+ type: pulse_fast
31
+ path:
32
+ pretrain_network_g: null
33
+ param_key_g: null
34
+ strict_load_g: false
35
+ resume_state: null
36
+ train:
37
+ ema_decay: 0.999
38
+ ema_power: 0.75
39
+ optim_g:
40
+ type: AdamW
41
+ lr: 0.0001
42
+ weight_decay: 0
43
+ betas:
44
+ - 0.9
45
+ - 0.99
46
+ scheduler:
47
+ type: MultiStepLR
48
+ milestones:
49
+ - 100000
50
+ - 160000
51
+ gamma: 0.5
52
+ total_iter: 200000
53
+ warmup_iter: -1
54
+ losses:
55
+ - type: charbonnierloss
56
+ loss_weight: 1.0
57
+ val:
58
+ val_enabled: true
59
+ val_freq: 5000
60
+ save_img: false
61
+ tile_size: 512
62
+ tile_overlap: 16
63
+ metrics_enabled: true
64
+ metrics:
65
+ psnr:
66
+ type: calculate_psnr
67
+ crop_border: 4
68
+ test_y_channel: true
69
+ ssim:
70
+ type: calculate_ssim
71
+ crop_border: 4
72
+ test_y_channel: true
73
+ topiq:
74
+ type: calculate_topiq
75
+ lpips:
76
+ type: calculate_lpips
77
+ better: lower
78
+ logger:
79
+ print_freq: 250
80
+ save_checkpoint_freq: 5000
81
+ save_checkpoint_format: safetensors
82
+ use_tb_logger: true
configs/2x_pulse_pretrain.yml ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: 2x_pulse_PULSE_pretrain
2
+ scale: 2
3
+ use_amp: true
4
+ amp_bf16: true
5
+ use_channels_last: true
6
+ num_gpu: 1
7
+ datasets:
8
+ train:
9
+ name: CC0/public-domain train
10
+ type: pairedimagedataset
11
+ dataroot_gt:
12
+ - datasets/train/cc0/gt
13
+ dataroot_lq:
14
+ - datasets/train/cc0/lr_x2
15
+ lq_size: 256
16
+ use_hflip: true
17
+ use_rot: true
18
+ num_worker_per_gpu: 8
19
+ batch_size_per_gpu: 8
20
+ accum_iter: 1
21
+ val:
22
+ name: Urban100 x4
23
+ type: pairedimagedataset
24
+ dataroot_gt:
25
+ - datasets/val/urban100/hr
26
+ dataroot_lq:
27
+ - datasets/val/urban100/lr_x2
28
+ filename_tmpl: '{}x2'
29
+ network_g:
30
+ type: pulse
31
+ path:
32
+ pretrain_network_g: null
33
+ param_key_g: null
34
+ strict_load_g: false
35
+ resume_state: null
36
+ train:
37
+ ema_decay: 0.999
38
+ ema_power: 0.75
39
+ optim_g:
40
+ type: AdamW
41
+ lr: 0.0001
42
+ weight_decay: 0
43
+ betas:
44
+ - 0.9
45
+ - 0.99
46
+ scheduler:
47
+ type: MultiStepLR
48
+ milestones:
49
+ - 100000
50
+ - 160000
51
+ gamma: 0.5
52
+ total_iter: 200000
53
+ warmup_iter: -1
54
+ losses:
55
+ - type: charbonnierloss
56
+ loss_weight: 1.0
57
+ val:
58
+ val_enabled: true
59
+ val_freq: 5000
60
+ save_img: false
61
+ tile_size: 512
62
+ tile_overlap: 16
63
+ metrics_enabled: true
64
+ metrics:
65
+ psnr:
66
+ type: calculate_psnr
67
+ crop_border: 4
68
+ test_y_channel: true
69
+ ssim:
70
+ type: calculate_ssim
71
+ crop_border: 4
72
+ test_y_channel: true
73
+ topiq:
74
+ type: calculate_topiq
75
+ lpips:
76
+ type: calculate_lpips
77
+ better: lower
78
+ logger:
79
+ print_freq: 250
80
+ save_checkpoint_freq: 5000
81
+ save_checkpoint_format: safetensors
82
+ use_tb_logger: true
configs/4x_pulse_fast_pretrain.yml ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: 4x_pulse_fast_PULSE_fast_pretrain
2
+ scale: 4
3
+ use_amp: true
4
+ amp_bf16: true
5
+ use_channels_last: true
6
+ num_gpu: 1
7
+ datasets:
8
+ train:
9
+ name: CC0/public-domain train
10
+ type: pairedimagedataset
11
+ dataroot_gt:
12
+ - datasets/train/cc0/gt
13
+ dataroot_lq:
14
+ - datasets/train/cc0/lr_x4
15
+ lq_size: 128
16
+ use_hflip: true
17
+ use_rot: true
18
+ num_worker_per_gpu: 8
19
+ batch_size_per_gpu: 8
20
+ accum_iter: 1
21
+ val:
22
+ name: Urban100 x4
23
+ type: pairedimagedataset
24
+ dataroot_gt:
25
+ - datasets/val/urban100/hr
26
+ dataroot_lq:
27
+ - datasets/val/urban100/lr_x4
28
+ filename_tmpl: '{}x4'
29
+ network_g:
30
+ type: pulse_fast
31
+ path:
32
+ pretrain_network_g: null
33
+ param_key_g: null
34
+ strict_load_g: false
35
+ resume_state: null
36
+ train:
37
+ ema_decay: 0.999
38
+ ema_power: 0.75
39
+ optim_g:
40
+ type: AdamW
41
+ lr: 0.0001
42
+ weight_decay: 0
43
+ betas:
44
+ - 0.9
45
+ - 0.99
46
+ scheduler:
47
+ type: MultiStepLR
48
+ milestones:
49
+ - 100000
50
+ - 160000
51
+ gamma: 0.5
52
+ total_iter: 200000
53
+ warmup_iter: -1
54
+ losses:
55
+ - type: charbonnierloss
56
+ loss_weight: 1.0
57
+ val:
58
+ val_enabled: true
59
+ val_freq: 5000
60
+ save_img: false
61
+ tile_size: 512
62
+ tile_overlap: 16
63
+ metrics_enabled: true
64
+ metrics:
65
+ psnr:
66
+ type: calculate_psnr
67
+ crop_border: 4
68
+ test_y_channel: true
69
+ ssim:
70
+ type: calculate_ssim
71
+ crop_border: 4
72
+ test_y_channel: true
73
+ topiq:
74
+ type: calculate_topiq
75
+ lpips:
76
+ type: calculate_lpips
77
+ better: lower
78
+ logger:
79
+ print_freq: 250
80
+ save_checkpoint_freq: 5000
81
+ save_checkpoint_format: safetensors
82
+ use_tb_logger: true
configs/4x_pulse_pretrain.yml ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: 4x_pulse_PULSE_pretrain
2
+ scale: 4
3
+ use_amp: true
4
+ amp_bf16: true
5
+ use_channels_last: true
6
+ num_gpu: 1
7
+ datasets:
8
+ train:
9
+ name: CC0/public-domain train
10
+ type: pairedimagedataset
11
+ dataroot_gt:
12
+ - datasets/train/cc0/gt
13
+ dataroot_lq:
14
+ - datasets/train/cc0/lr_x4
15
+ lq_size: 128
16
+ use_hflip: true
17
+ use_rot: true
18
+ num_worker_per_gpu: 8
19
+ batch_size_per_gpu: 8
20
+ accum_iter: 1
21
+ val:
22
+ name: Urban100 x4
23
+ type: pairedimagedataset
24
+ dataroot_gt:
25
+ - datasets/val/urban100/hr
26
+ dataroot_lq:
27
+ - datasets/val/urban100/lr_x4
28
+ filename_tmpl: '{}x4'
29
+ network_g:
30
+ type: pulse
31
+ path:
32
+ pretrain_network_g: null
33
+ param_key_g: null
34
+ strict_load_g: false
35
+ resume_state: null
36
+ train:
37
+ ema_decay: 0.999
38
+ ema_power: 0.75
39
+ optim_g:
40
+ type: AdamW
41
+ lr: 0.0001
42
+ weight_decay: 0
43
+ betas:
44
+ - 0.9
45
+ - 0.99
46
+ scheduler:
47
+ type: MultiStepLR
48
+ milestones:
49
+ - 100000
50
+ - 160000
51
+ gamma: 0.5
52
+ total_iter: 200000
53
+ warmup_iter: -1
54
+ losses:
55
+ - type: charbonnierloss
56
+ loss_weight: 1.0
57
+ val:
58
+ val_enabled: true
59
+ val_freq: 5000
60
+ save_img: false
61
+ tile_size: 512
62
+ tile_overlap: 16
63
+ metrics_enabled: true
64
+ metrics:
65
+ psnr:
66
+ type: calculate_psnr
67
+ crop_border: 4
68
+ test_y_channel: true
69
+ ssim:
70
+ type: calculate_ssim
71
+ crop_border: 4
72
+ test_y_channel: true
73
+ topiq:
74
+ type: calculate_topiq
75
+ lpips:
76
+ type: calculate_lpips
77
+ better: lower
78
+ logger:
79
+ print_freq: 250
80
+ save_checkpoint_freq: 5000
81
+ save_checkpoint_format: safetensors
82
+ use_tb_logger: true
configs/pulse_arch.py ADDED
@@ -0,0 +1,165 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """PULSE — fast, deployable single-image super-resolution.
2
+
3
+ A ~162K parameter SISR network (354K at dim=48). The body is a plain conv
4
+ residual stack; the base image is a fixed Magic Kernel Sharp 2013 resampler
5
+ implemented as a phase-kernel grouped convolution plus PixelShuffle. No
6
+ normalization, no attention, no custom ops: the exported graph is
7
+ Pad / Conv / Add / Relu / DepthToSpace, which is friendly to ONNX Runtime,
8
+ TensorRT, and NCNN.
9
+ """
10
+
11
+ from __future__ import annotations
12
+
13
+ import torch
14
+ from spandrel.util import store_hyperparameters
15
+ from torch import Tensor, nn
16
+ from torch.nn import functional as F # noqa: N812
17
+
18
+ from traiNNer.utils.icnr import icnr_init
19
+ from traiNNer.utils.registry import ARCH_REGISTRY, SPANDREL_REGISTRY
20
+
21
+
22
+ def _mks_1d(x: Tensor) -> Tensor:
23
+ """Magic Kernel Sharp 2013, one dimension, support 2.5."""
24
+ x = x.abs()
25
+ return torch.where(
26
+ x <= 0.5,
27
+ 17 / 16 - (7 / 4) * x**2,
28
+ torch.where(
29
+ x <= 1.5,
30
+ 0.25 * (4 * x**2 - 11 * x + 7),
31
+ torch.where(x <= 2.5, -0.125 * (x - 2.5) ** 2, torch.zeros_like(x)),
32
+ ),
33
+ )
34
+
35
+
36
+ class MKS2013Base(nn.Module):
37
+ """Fixed Magic Kernel Sharp 2013 integer upsampler.
38
+
39
+ For an integer scale, each output phase has a fixed 5x5 separable kernel.
40
+ Storing the kernels as a grouped convolution and using PixelShuffle gives
41
+ the exact integer-scale kernel with standard Conv / DepthToSpace ops and no
42
+ Resize. Borders use replicate padding rather than Costella's weight
43
+ renormalization, and the kernel is applied in the input color space (sRGB
44
+ for images), not the linear-light space used by the reference.
45
+ """
46
+
47
+ def __init__(self, channels: int, upscale: int) -> None:
48
+ super().__init__()
49
+ self.channels = channels
50
+ self.upscale = upscale
51
+ if upscale == 1:
52
+ return
53
+
54
+ # One-time constant kernels; no gradients and no state-dict payload.
55
+ offsets = torch.arange(-2, 3, dtype=torch.float32)
56
+ weight = torch.empty(channels * upscale**2, 1, 5, 5)
57
+ for c in range(channels):
58
+ for py in range(upscale):
59
+ for px in range(upscale):
60
+ fy = (py + 0.5) / upscale - 0.5
61
+ fx = (px + 0.5) / upscale - 0.5
62
+ wy = _mks_1d(fy - offsets)
63
+ wx = _mks_1d(fx - offsets)
64
+ weight[c * upscale**2 + py * upscale + px, 0] = torch.outer(wy, wx)
65
+ self.register_buffer("weight", weight, persistent=False)
66
+ self.shuffle = nn.PixelShuffle(upscale)
67
+
68
+ def forward(self, x: Tensor) -> Tensor:
69
+ if self.upscale == 1:
70
+ return x
71
+ x = F.pad(x, (2, 2, 2, 2), mode="replicate")
72
+ y = F.conv2d(x, self.get_buffer("weight"), groups=self.channels)
73
+ return self.shuffle(y)
74
+
75
+
76
+ class PulseBlock(nn.Module):
77
+ """Conv-ReLU-Conv residual block. No normalization, no gating."""
78
+
79
+ def __init__(self, dim: int) -> None:
80
+ super().__init__()
81
+ self.conv1 = nn.Conv2d(dim, dim, 3, padding=1, bias=False)
82
+ self.conv2 = nn.Conv2d(dim, dim, 3, padding=1, bias=False)
83
+ self.act = nn.ReLU(inplace=True)
84
+
85
+ def forward(self, x: Tensor) -> Tensor:
86
+ return x + self.conv2(self.act(self.conv1(x)))
87
+
88
+
89
+ @store_hyperparameters()
90
+ class PULSE(nn.Module):
91
+ """MKS2013 base + residual convs + PixelShuffle correction."""
92
+
93
+ hyperparameters = {} # noqa: RUF012
94
+
95
+ def __init__(
96
+ self,
97
+ *,
98
+ in_ch: int = 3,
99
+ out_ch: int = 3,
100
+ dim: int = 48,
101
+ n_blocks: int = 8,
102
+ upscale: int = 4,
103
+ ) -> None:
104
+ super().__init__()
105
+ # The MKS base is added directly to the output, so channels must match.
106
+ if in_ch != out_ch:
107
+ raise ValueError("PULSE requires in_ch == out_ch for its base residual.")
108
+
109
+ self.upscale = upscale
110
+ self.base = MKS2013Base(in_ch, upscale)
111
+ self.stem = nn.Conv2d(in_ch, dim, 3, padding=1, bias=False)
112
+ self.body = nn.Sequential(*[PulseBlock(dim) for _ in range(n_blocks)])
113
+ # Head bias is intentional: body convs are bias-free and the head
114
+ # learns any residual color shift.
115
+ self.head = nn.Sequential(
116
+ nn.Conv2d(dim, out_ch * upscale**2, 3, padding=1, bias=True),
117
+ nn.PixelShuffle(upscale),
118
+ )
119
+ self._initialize_weights()
120
+
121
+ def _initialize_weights(self) -> None:
122
+ for m in self.modules():
123
+ if isinstance(m, nn.Conv2d):
124
+ nn.init.kaiming_normal_(m.weight, mode="fan_in", nonlinearity="relu")
125
+ if m.bias is not None:
126
+ nn.init.zeros_(m.bias)
127
+
128
+ # ICNR removes phase bias from the PixelShuffle head; the tiny scale
129
+ # keeps the initial output close to the MKS base.
130
+ head = self.head[0]
131
+ assert isinstance(head, nn.Conv2d)
132
+ icnr_init(head, self.upscale)
133
+ with torch.no_grad():
134
+ head.weight.mul_(1e-3)
135
+ if head.bias is not None:
136
+ head.bias.zero_()
137
+
138
+ def forward(self, x: Tensor) -> Tensor:
139
+ base = self.base(x)
140
+ feat = self.stem(x)
141
+ # Direct stem-to-head path; the blocks are already residual.
142
+ feat = feat + self.body(feat)
143
+ return base + self.head(feat)
144
+
145
+
146
+ @SPANDREL_REGISTRY.register()
147
+ @ARCH_REGISTRY.register()
148
+ def pulse(
149
+ scale: int = 4,
150
+ *,
151
+ in_ch: int = 3,
152
+ out_ch: int = 3,
153
+ ) -> PULSE:
154
+ return PULSE(in_ch=in_ch, out_ch=out_ch, dim=48, n_blocks=8, upscale=scale)
155
+
156
+
157
+ @SPANDREL_REGISTRY.register()
158
+ @ARCH_REGISTRY.register()
159
+ def pulse_fast(
160
+ scale: int = 4,
161
+ *,
162
+ in_ch: int = 3,
163
+ out_ch: int = 3,
164
+ ) -> PULSE:
165
+ return PULSE(in_ch=in_ch, out_ch=out_ch, dim=32, n_blocks=8, upscale=scale)
examples/4x_pulse_urban100_img_001_input_vs_pretrain.png ADDED

Git LFS Details

  • SHA256: 7050bea3e3449816d89dd44fb9be367e92db877313bf9952f232317387a6c06d
  • Pointer size: 132 Bytes
  • Size of remote file: 1.49 MB
examples/4x_pulse_urban100_img_004_input_vs_pretrain.png ADDED

Git LFS Details

  • SHA256: 664410e9af4ab829fbc000a51cb7cf9335fc9e4915e994603c37f9a2fcf3c962
  • Pointer size: 132 Bytes
  • Size of remote file: 1.95 MB
examples/4x_pulse_urban100_img_006_input_vs_pretrain.png ADDED

Git LFS Details

  • SHA256: ca6b1ff8143503b2a87f327beb6fcb60605c719aa8195e346506cc1d5c410b64
  • Pointer size: 132 Bytes
  • Size of remote file: 3.04 MB
models/pulse/2x_pretrain.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3fd891ff8113ac7caf43de985a2759d915d3a6626d50e1db04bc55865e5b5e13
3
+ size 1354856
models/pulse/4x_pretrain.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:15ac6f2b958b016ad26f1a55700b6530ceeeb08feea920fdf6385029176f2980
3
+ size 1417208
models/pulse_fast/2x_pretrain.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2957d4af0069001509ad3222a3dfbad97290b120a40916221d3cc6547097c4b4
3
+ size 608920
models/pulse_fast/4x_pretrain.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fa9e477807eb65c7340aeadbee374213853eb3c2fe06210898bae910e6f654a8
3
+ size 650536
onnx/2x_pulse.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:baeeb9f310aa4dfad5f3a5f7961254270ba09256d6fb49fbcb4b2b5e63c34251
3
+ size 1361094
onnx/2x_pulse_fast.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:66c8905a7671901623a1f74333a259c39e4b6238069b1fc2c64036672d8421fc
3
+ size 615172
onnx/2x_pulse_fast_fp16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:22c277e7cde5d864b8a8ff41d3a74d1afc41ef08ceb4b5349ca272ad525913e8
3
+ size 311496
onnx/2x_pulse_fp16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:765559129d740c989027ca28d0c7bbd12c1aadd9411aa1e44aa726950cb35352
3
+ size 684456
onnx/4x_pulse.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c616de7389281ad0a5cb9fa09c5ca13b4786f72a33ddf1535c0abefb6893a766
3
+ size 1427048
onnx/4x_pulse_fast.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b2c9703ee9b4c82278599c38f6389495944c769bc43a64221c77a3f906ef5cb5
3
+ size 660392
onnx/4x_pulse_fast_fp16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c62531bd9f49b402234c861c6dbfa81ff29edee4b547d1522f928be0f41bf3bd
3
+ size 334106
onnx/4x_pulse_fp16.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:185b51606ba170e9feb207c4778c75e5c21f81b4e5b22c749de9a9a1bd5dbfc3
3
+ size 717434