PULSE / PAPER.md

State Y-channel PSNR/SSIM

1751f63 verified 1 day ago

9.26 kB

# PULSE technical report

*Private draft. Companion to the model card.*


# PULSE

**A tiny, deployment-first super-resolution network.**

PULSE is the **tiny/fast sibling of NERVE**: a much smaller convolutional SISR
model with the same deployment-first philosophy. It trades capacity for size and
speed, so it is less capable than NERVE and is intended for light cleanup and
clean upscaling rather than heavy restoration. It trains on a consumer GPU,
converts without surprises, and embeds almost anywhere.

- `pulse`: 48 channels, **353,856 parameters**
- `pulse_fast`: 32 channels, **162,192 parameters**
- 2x and 4x models
- PyTorch `.safetensors` + dynamic ONNX
- **MIT** code and weights, trained on **CC0 / public-domain images only**

**Links:** [HEART](https://huggingface.co/Phips/HEART) · [NERVE](https://huggingface.co/Phips/NERVE) · [PULSE demo](https://huggingface.co/spaces/Phips/PULSE-demo) · [PULSE ONNX converter](https://huggingface.co/spaces/Phips/PULSE-convert)

> Short version: load a released `.safetensors` with `pulse_arch.py`, or an
> `.onnx` with ONNX Runtime. Start with 2x for browser/CPU use.
---
## Why PULSE
PULSE is not different for the sake of being different. Each constraint removes
a real deployment or maintenance problem.
- **No `Resize` op in the exported graph.** The scale is done by a fixed MKS2013
  base plus PixelShuffle. ONNX, TensorRT, NCNN, and other runtimes all see the
  same standard graph; there is no interpolation implementation mismatch.
- **Only standard ops.** `Conv / Add / ReLU / Pad / DepthToSpace`. No custom
  kernels, no fused/unfused checkpoint pair, no fragile converter step. That
  makes it easy to audit, port, optimize with `onnxsim`, and compile with
  TensorRT.
- **Fixed MKS2013 base.** The base upsampler has no learned parameters and no
  dynamic branch. It anchors the output, reduces what the network must learn,
  and keeps the graph compact and deterministic.
- **No attention and no normalization.** That means stable bf16 AMP training,
  fewer memory spikes, and simpler maintenance. The whole model is readable.
- **Tiny.** 162K–354K parameters. It fits browser/WASM budgets, small GPUs,
  CPU fallbacks, and embedded pipelines where a transformer does not.
- **Fast training.** The pretrain uses Charbonnier only: no discriminator, no
  perceptual stack, no fragile GAN schedule. It trains on a single RTX 3060.
- **Dynamic ONNX.** One model works at any H/W; no per-resolution exports.
## What it is for
PULSE is a **local, low-cost upscaler**. The released pretrains are clean
2x/4x models, not a general blind-restoration model.
- **Browser or desktop upscaling.** ONNX Runtime Web/WASM/WebGPU can run the 2x
  models; 4x is heavier but still small.
- **Image viewers, galleries, thumbnail pipelines.** A fast local pass before
  display, with no server round-trip.
- **Anime and illustration stills.** Clean 2x/4x upscaling; a compression-trained
  variant is a separate model.
- **Batch/offline preprocessing.** Small enough to share a consumer GPU or CPU.
- **Embedding in apps.** One architecture file, one op set, no fuse step.
Compression cleanup is a real use case, but it needs the separate
**PULSE-Web** finetune. The released clean pretrain was not trained on
JPEG/WebP/AVIF/H.265/GIF artifacts.
A temporal/video cleaner is **not part of this release**. Per-frame PULSE can
be a component, but flicker-free video needs temporal conditioning.
## When to use NERVE or HEART instead
| Need | Use |
|---|---|
| Browser/CPU, tiny model, fast 2x/4x | **PULSE** |
| Better quality with a ~1.8M conv-only edge model | **NERVE** |
| Highest quality / desktop-server GPU budget | **HEART** |
| Blind OTF restoration, heavy blur/noise | NERVE/HEART OTF or a larger restoration model |
| Text/UI/screenshot upscaling | a text-aware restoration model, not PULSE |
| Temporal video restoration | a temporal model; PULSE is per-frame only |
PULSE is the **tiny/fast NERVE sibling**: the same practical philosophy, but
far fewer parameters and a lighter operating envelope. NERVE is the more
capable restoration model; PULSE is for light degradation, clean upscaling,
browser/CPU use, and cases where the model itself must be almost free.
PULSE is not competing with HEART on PSNR.
## Pick a model
| Want | Use | Scale | Params |
|---|---|---:|---:|
| Best small-model quality | `pulse` | 4x / 2x | 353,856 |
| Fastest / lowest compute | `pulse_fast` | 4x / 2x | 162,192 |
| CPU / ORT / engine | `onnx/*_fp16.onnx` | 4x / 2x | same |

All released models are pretrained only on **CC0 / public-domain images**. They
are MIT-licensed and permissive.

### Urban100 validation

| Model | PSNR | SSIM | TopIQ | LPIPS |
|---|---:|---:|---:|---:|
| 4x `pulse` | **25.6523** | **0.7717** | **0.5907** | **0.2452** |
| 2x `pulse` | **31.4423** | **0.9221** | **0.8614** | **0.0721** |
| 4x `pulse_fast` | **25.4239** | **0.7638** | **0.5748** | **0.2553** |
| 2x `pulse_fast` | **31.1371** | **0.9188** | **0.8551** | **0.0755** |

EMA weights, standard Urban100 validation. PSNR/SSIM are Y-channel with a 4px border crop; TopIQ/LPIPS are RGB. No TTA.

### Inference speed

RTX 3060, batch 1, fp16. TensorRT engine built locally.

| Model | 256² eager | 720p eager | 1080p eager | 256² TRT | 720p TRT | 1080p TRT |
|---|---:|---:|---:|---:|---:|---:|
| `pulse_fast` | 3.65 | 47.00 | 103.88 | 1.43 | 17.52 | 39.60 |
| `pulse` | 6.71 | 90.20 | 199.28 | 2.82 | 35.35 | 79.65 |

### Training speed

CC0/public-domain 200k fidelity pretrain, RTX 3060, batch 8, bf16:

- `pulse` 4x: ~29 it/s
- `pulse` 2x: ~9.8 it/s
- `pulse_fast` 4x: ~36 it/s
- `pulse_fast` 2x: ~20 it/s

Training is Charbonnier-only. Perceptual branches are experimental and are not
part of the pretrain release.

## Quickstart

### ONNX Runtime

```python
import onnxruntime as ort
sess = ort.InferenceSession("onnx/4x_pulse_fp16.onnx", providers=["CPUExecutionProvider"])
# input: float32 [1, 3, H, W] in [0, 1], RGB, dynamic H/W
```

The fp16 file is self-contained. Use `4x_pulse.onnx` if your runtime lacks fp16.

### PyTorch / spandrel

1. Copy `pulse_arch.py` into `traiNNer/archs/` (or use the file here).
2. Load the `.safetensors` with spandrel once PULSE is registered.
3. Input RGB `[0,1]`, output 2x/4x.

### Train

```bash
cp pulse_arch.py <traiNNer-redux>/traiNNer/archs/
cp configs/4x_pulse_pretrain.yml <traiNNer-redux>/options/
cd <traiNNer-redux>
python train.py -opt options/4x_pulse_pretrain.yml
```

The sample configs use placeholder data paths and can be trained from scratch.
The released pretrains are provided as warm starts.

## Visual examples

Urban100 4x. Left: bicubic input. Right: released PULSE 4x pretrain.

![Urban100 example 1](examples/4x_pulse_urban100_img_001_input_vs_pretrain.png)
![Urban100 example 2](examples/4x_pulse_urban100_img_004_input_vs_pretrain.png)
![Urban100 example 3](examples/4x_pulse_urban100_img_006_input_vs_pretrain.png)

## Files

```text
pulse_arch.py
models/
  pulse/4x_pretrain.safetensors
  pulse/2x_pretrain.safetensors
  pulse_fast/4x_pretrain.safetensors
  pulse_fast/2x_pretrain.safetensors
onnx/
  *_fp16.onnx
  *.onnx
configs/
  4x_pulse_pretrain.yml
  2x_pulse_pretrain.yml
  4x_pulse_fast_pretrain.yml
  2x_pulse_fast_pretrain.yml
PAPER.md
examples/
```

## Limitations

- Not a blind-OTF model; no degradation embedding.
- Not a text/UI upscaler.
- The clean pretrain does not clean compression artifacts; use a web/anime
  finetune for that.
- No temporal/video model is included.
- PSNR is below HEART; that is the intended size/speed tradeoff.

## Attribution

The fixed base uses **Magic Kernel Sharp 2013** by **John Costella**
(https://johncostella.com/magic/). The public reference implementation
(`felixr/magic-kernel-sharp`, MIT) is credited in `ATTRIBUTION.md`. PULSE uses
an independent PyTorch implementation of the integer-scale phase kernels, not
a copy of the reference C code.

## License

The code and released weights are **MIT-licensed** and permissive. Keep the
MIT copyright and permission notice with copies or substantial portions of the
software. See `LICENSE`.

The released weights were trained on a curated CC0 / public-domain image set.
This is a provenance statement, not a guarantee; source-image rights and
jurisdiction-specific rules can still matter. No warranty is provided.

```bibtex
@misc{pulse_body_2026,
  title  = {PULSE: A Deployment-First Single-Image Super-Resolution Network},
  author = {Philip Hofmann and contributors},
  year   = {2026},
  note   = {BODY suite}
}
```


---

## Appendix: ablation context

PULSE was selected through equal-budget internal experiments. The published
weights are the 200k pretrains trained only on CC0/public-domain images.

- 48 channels / 8 blocks beat 32 channels / 8 blocks on quality at 2.2x the
  parameters; both keep the same deployment graph.
- MKS2013 and bicubic bases tied on quality; MKS was kept because it removes
  `Resize` from ONNX.
- RepConv/PULSE-R gained PSNR but cost 2.1x training time, 3.3x training VRAM,
  10x training parameters, and a fuse step; rejected.
- Partial depthwise, 5x5 stem, and tiny variants were rejected.
- Perceptual/compression branches are separate and are not part of this
  pretrain release.

	# PULSE technical report

	Private draft. Companion to the model card.


	# PULSE

	A tiny, deployment-first super-resolution network.

	PULSE is the tiny/fast sibling of NERVE: a much smaller convolutional SISR
	model with the same deployment-first philosophy. It trades capacity for size and
	speed, so it is less capable than NERVE and is intended for light cleanup and
	clean upscaling rather than heavy restoration. It trains on a consumer GPU,
	converts without surprises, and embeds almost anywhere.

	- `pulse`: 48 channels, 353,856 parameters
	- `pulse_fast`: 32 channels, 162,192 parameters
	- 2x and 4x models
	- PyTorch `.safetensors` + dynamic ONNX
	- MIT code and weights, trained on CC0 / public-domain images only

	Links: [HEART](https://huggingface.co/Phips/HEART) · [NERVE](https://huggingface.co/Phips/NERVE) · [PULSE demo](https://huggingface.co/spaces/Phips/PULSE-demo) · [PULSE ONNX converter](https://huggingface.co/spaces/Phips/PULSE-convert)

	> Short version: load a released `.safetensors` with `pulse_arch.py`, or an
	> `.onnx` with ONNX Runtime. Start with 2x for browser/CPU use.

	---

	## Why PULSE

	PULSE is not different for the sake of being different. Each constraint removes
	a real deployment or maintenance problem.

	- No `Resize` op in the exported graph. The scale is done by a fixed MKS2013
	base plus PixelShuffle. ONNX, TensorRT, NCNN, and other runtimes all see the
	same standard graph; there is no interpolation implementation mismatch.
	- Only standard ops. `Conv / Add / ReLU / Pad / DepthToSpace`. No custom
	kernels, no fused/unfused checkpoint pair, no fragile converter step. That
	makes it easy to audit, port, optimize with `onnxsim`, and compile with
	TensorRT.
	- Fixed MKS2013 base. The base upsampler has no learned parameters and no
	dynamic branch. It anchors the output, reduces what the network must learn,
	and keeps the graph compact and deterministic.
	- No attention and no normalization. That means stable bf16 AMP training,
	fewer memory spikes, and simpler maintenance. The whole model is readable.
	- Tiny. 162K–354K parameters. It fits browser/WASM budgets, small GPUs,
	CPU fallbacks, and embedded pipelines where a transformer does not.
	- Fast training. The pretrain uses Charbonnier only: no discriminator, no
	perceptual stack, no fragile GAN schedule. It trains on a single RTX 3060.
	- Dynamic ONNX. One model works at any H/W; no per-resolution exports.

	## What it is for

	PULSE is a local, low-cost upscaler. The released pretrains are clean
	2x/4x models, not a general blind-restoration model.

	- Browser or desktop upscaling. ONNX Runtime Web/WASM/WebGPU can run the 2x
	models; 4x is heavier but still small.
	- Image viewers, galleries, thumbnail pipelines. A fast local pass before
	display, with no server round-trip.
	- Anime and illustration stills. Clean 2x/4x upscaling; a compression-trained
	variant is a separate model.
	- Batch/offline preprocessing. Small enough to share a consumer GPU or CPU.
	- Embedding in apps. One architecture file, one op set, no fuse step.

	Compression cleanup is a real use case, but it needs the separate
	PULSE-Web finetune. The released clean pretrain was not trained on
	JPEG/WebP/AVIF/H.265/GIF artifacts.

	A temporal/video cleaner is not part of this release. Per-frame PULSE can
	be a component, but flicker-free video needs temporal conditioning.

	## When to use NERVE or HEART instead

	\| Need \| Use \|
	\|---\|---\|
	\| Browser/CPU, tiny model, fast 2x/4x \| PULSE \|
	\| Better quality with a ~1.8M conv-only edge model \| NERVE \|
	\| Highest quality / desktop-server GPU budget \| HEART \|
	\| Blind OTF restoration, heavy blur/noise \| NERVE/HEART OTF or a larger restoration model \|
	\| Text/UI/screenshot upscaling \| a text-aware restoration model, not PULSE \|
	\| Temporal video restoration \| a temporal model; PULSE is per-frame only \|

	PULSE is the tiny/fast NERVE sibling: the same practical philosophy, but
	far fewer parameters and a lighter operating envelope. NERVE is the more
	capable restoration model; PULSE is for light degradation, clean upscaling,
	browser/CPU use, and cases where the model itself must be almost free.

	PULSE is not competing with HEART on PSNR.

	## Pick a model

	\| Want \| Use \| Scale \| Params \|
	\|---\|---\|---:\|---:\|
	\| Best small-model quality \| `pulse` \| 4x / 2x \| 353,856 \|
	\| Fastest / lowest compute \| `pulse_fast` \| 4x / 2x \| 162,192 \|
	\| CPU / ORT / engine \| `onnx/*_fp16.onnx` \| 4x / 2x \| same \|

	All released models are pretrained only on CC0 / public-domain images. They
	are MIT-licensed and permissive.

	### Urban100 validation

	\| Model \| PSNR \| SSIM \| TopIQ \| LPIPS \|
	\|---\|---:\|---:\|---:\|---:\|
	\| 4x `pulse` \| 25.6523 \| 0.7717 \| 0.5907 \| 0.2452 \|
	\| 2x `pulse` \| 31.4423 \| 0.9221 \| 0.8614 \| 0.0721 \|
	\| 4x `pulse_fast` \| 25.4239 \| 0.7638 \| 0.5748 \| 0.2553 \|
	\| 2x `pulse_fast` \| 31.1371 \| 0.9188 \| 0.8551 \| 0.0755 \|

	EMA weights, standard Urban100 validation. PSNR/SSIM are Y-channel with a 4px border crop; TopIQ/LPIPS are RGB. No TTA.

	### Inference speed

	RTX 3060, batch 1, fp16. TensorRT engine built locally.

	\| Model \| 256² eager \| 720p eager \| 1080p eager \| 256² TRT \| 720p TRT \| 1080p TRT \|
	\|---\|---:\|---:\|---:\|---:\|---:\|---:\|
	\| `pulse_fast` \| 3.65 \| 47.00 \| 103.88 \| 1.43 \| 17.52 \| 39.60 \|
	\| `pulse` \| 6.71 \| 90.20 \| 199.28 \| 2.82 \| 35.35 \| 79.65 \|

	### Training speed

	CC0/public-domain 200k fidelity pretrain, RTX 3060, batch 8, bf16:

	- `pulse` 4x: ~29 it/s
	- `pulse` 2x: ~9.8 it/s
	- `pulse_fast` 4x: ~36 it/s
	- `pulse_fast` 2x: ~20 it/s

	Training is Charbonnier-only. Perceptual branches are experimental and are not
	part of the pretrain release.

	## Quickstart

	### ONNX Runtime

	```python
	import onnxruntime as ort
	sess = ort.InferenceSession("onnx/4x_pulse_fp16.onnx", providers=["CPUExecutionProvider"])
	# input: float32 [1, 3, H, W] in [0, 1], RGB, dynamic H/W
	```

	The fp16 file is self-contained. Use `4x_pulse.onnx` if your runtime lacks fp16.

	### PyTorch / spandrel

	1. Copy `pulse_arch.py` into `traiNNer/archs/` (or use the file here).
	2. Load the `.safetensors` with spandrel once PULSE is registered.
	3. Input RGB `[0,1]`, output 2x/4x.

	### Train

	```bash
	cp pulse_arch.py <traiNNer-redux>/traiNNer/archs/
	cp configs/4x_pulse_pretrain.yml <traiNNer-redux>/options/
	cd <traiNNer-redux>
	python train.py -opt options/4x_pulse_pretrain.yml
	```

	The sample configs use placeholder data paths and can be trained from scratch.
	The released pretrains are provided as warm starts.

	## Visual examples

	Urban100 4x. Left: bicubic input. Right: released PULSE 4x pretrain.

	![Urban100 example 1](examples/4x_pulse_urban100_img_001_input_vs_pretrain.png)
	![Urban100 example 2](examples/4x_pulse_urban100_img_004_input_vs_pretrain.png)
	![Urban100 example 3](examples/4x_pulse_urban100_img_006_input_vs_pretrain.png)

	## Files

	```text
	pulse_arch.py
	models/
	pulse/4x_pretrain.safetensors
	pulse/2x_pretrain.safetensors
	pulse_fast/4x_pretrain.safetensors
	pulse_fast/2x_pretrain.safetensors
	onnx/
	*_fp16.onnx
	*.onnx
	configs/
	4x_pulse_pretrain.yml
	2x_pulse_pretrain.yml
	4x_pulse_fast_pretrain.yml
	2x_pulse_fast_pretrain.yml
	PAPER.md
	examples/
	```

	## Limitations

	- Not a blind-OTF model; no degradation embedding.
	- Not a text/UI upscaler.
	- The clean pretrain does not clean compression artifacts; use a web/anime
	finetune for that.
	- No temporal/video model is included.
	- PSNR is below HEART; that is the intended size/speed tradeoff.

	## Attribution

	The fixed base uses Magic Kernel Sharp 2013 by John Costella
	(https://johncostella.com/magic/). The public reference implementation
	(`felixr/magic-kernel-sharp`, MIT) is credited in `ATTRIBUTION.md`. PULSE uses
	an independent PyTorch implementation of the integer-scale phase kernels, not
	a copy of the reference C code.

	## License

	The code and released weights are MIT-licensed and permissive. Keep the
	MIT copyright and permission notice with copies or substantial portions of the
	software. See `LICENSE`.

	The released weights were trained on a curated CC0 / public-domain image set.
	This is a provenance statement, not a guarantee; source-image rights and
	jurisdiction-specific rules can still matter. No warranty is provided.

	```bibtex
	@misc{pulse_body_2026,
	title = {PULSE: A Deployment-First Single-Image Super-Resolution Network},
	author = {Philip Hofmann and contributors},
	year = {2026},
	note = {BODY suite}
	}
	```


	---

	## Appendix: ablation context

	PULSE was selected through equal-budget internal experiments. The published
	weights are the 200k pretrains trained only on CC0/public-domain images.

	- 48 channels / 8 blocks beat 32 channels / 8 blocks on quality at 2.2x the
	parameters; both keep the same deployment graph.
	- MKS2013 and bicubic bases tied on quality; MKS was kept because it removes
	`Resize` from ONNX.
	- RepConv/PULSE-R gained PSNR but cost 2.1x training time, 3.3x training VRAM,
	10x training parameters, and a fuse step; rejected.
	- Partial depthwise, 5x5 stem, and tiny variants were rejected.
	- Perceptual/compression branches are separate and are not part of this
	pretrain release.