README.md

---

license: cc-by-nc-4.0
pipeline_tag: image-to-image
library_name: pytorch
tags:
- image-super-resolution
- super-resolution
- image-upscaling
- arbitrary-scale-sr
- liif
- esrgan
- pytorch
- torchscript
- safetensors
---


# Universal Upscale - arbitrary-scale super-resolution (LIIF)

One checkpoint for every scale: continuous-coordinate decoding instead of a fixed x2/x4 upsampler.

**Input:** any RGB image · **Output:** the same image upscaled to an arbitrary target size
(2x, 4x, 9.2x, or an explicit size such as "long side = 3000 px"), aspect ratio preserved.

## What this model does

- A convolutional encoder builds features at the input resolution.
- An implicit decoder queries a local feature neighbourhood plus the relative coordinate of every output pixel, so the output grid is **not** tied to a fixed scale factor.
- That is why the same weights produce 2x, 3.7x, 4x, 9.2x or an exact target such as 3000 px.

- Tiled inference with feathered blending is **built into the model file**, so 4K/8K outputs
  do not exhaust memory.
- Runs on CPU or CUDA. No custom operators, no external model zoo.

## Files in this repository

| File | What it is |
|---|---|
| `model.safetensors` | **HF-preferred weights.** Safe tensor format containing the trained network weights. Keys match the architecture described in `config.json`. |
| `model_scripted.pt` | **Runnable artifact.** Self-contained TorchScript module: network graph + trained weights + tiled inference. Load it with `torch.jit.load()` — no training source required. |
| `config.json` | Architecture hyper-parameters, inference defaults, training summary. |
| `upscale.py` | Standalone CLI demo (torch + Pillow + numpy only). |
| `examples/` | Input/output previews, zoomed comparison and benchmark numbers. |
| `app.py` | Gradio demo, ready to deploy as a Hugging Face Space. |

## Quick start (Python)

```python

import numpy as np

import torch

from PIL import Image



model = torch.jit.load("model_scripted.pt")   # graph + weights, self-contained

model.eval()



img = Image.open("photo.jpg").convert("RGB")

lr = torch.from_numpy(np.asarray(img, dtype=np.float32) / 255.0).permute(2, 0, 1).unsqueeze(0)



out = model(lr, img.height * 4, img.width * 4)   # (lr_tensor, target_height, target_width)



arr = (out.squeeze(0).permute(1, 2, 0).numpy() * 255.0).round().astype("uint8")

Image.fromarray(arr).save("photo_4x.png")

```

Download the weights straight from the Hub instead of keeping a local copy:

```python

from huggingface_hub import hf_hub_download



path = hf_hub_download(repo_id="tailot/universal-upscale", filename="model_scripted.pt")

model = torch.jit.load(path)

```

If you only want the HF-native weights, download `model.safetensors`:

```python

from safetensors.torch import load_file



state_dict = load_file("model.safetensors")

```

Use `model_scripted.pt` when you want a runnable model without exposing the private training source.

## CLI demo

```bash

pip install -r requirements.txt



python upscale.py --input photo.jpg --output photo_4x.png --scale 4

python upscale.py --input photo.jpg --output photo_max.png --scale 9.2

python upscale.py --input photo.jpg --output photo_3000.png --target-long-side 3000

python upscale.py --input photo.jpg --output photo_3000.png --target-width 3000 --target-height 3000 --pad-to-target

```

## Colab notebook: billboard-size outputs (Photo billboard)

**Notebook on Google Drive:** <https://drive.google.com/file/d/14B1cu8vW4MFPqzPo-C7Nb4JfxZqFtcEI/view?usp=sharing> → open it in Colab and pick *Runtime → Cambia tipo di runtime → GPU (T4)*.

The notebook runs **the base model published in this repository** (`model_scripted.pt`, fetched with `hf_hub_download`): **no retraining, no fine-tuning, no extra weights**. What it adds is the harness around the model - the image is split into an N×N grid of sectors, each sector is upscaled with a margin of context taken from its neighbours so the seams stay invisible, the sectors are stitched back into one canvas, and when the requested size is still out of reach the whole pass is repeated on that canvas until it hits the exact target.

**Thanks to that implementation the notebook can upscale an image of any resolution** - print/billboard sizes, not just 4K - but the ceiling is the machine it runs on, not the model:

| Resource | What it limits |
|---|---|
| **GPU VRAM** | the size of a single sector (default cap 4096 px per output side) |
| **GPU compute power** | total runtime: every sector is its own forward pass, and every extra pass replays the whole grid |
| **System RAM** | the stitched canvas held in memory (default cap 300 megapixel ≈ 3 byte/px, Colab free tier ≈ 12 GB) |
| **SSD storage space** | the saved output plus the optional per-sector files; Colab disk is temporary, so download before the runtime closes |

> ⚠️ Above **9.2x** the detail is largely *invented*: each pass reinterprets the output of the previous one, so the image grows without becoming more faithful. Check the intermediate previews.

## API contract

```text

forward(lr: Tensor[B, 3, H, W] float32 in [0, 1], target_h: int, target_w: int)

    -> Tensor[B, 3, target_h, target_w] float32 in [0, 1]



forward_single(lr, target_h, target_w)   # single pass, no tiling (small images only)

```

The model accepts any `target_h` / `target_w`. Training sampled scales from 1.5x to 6x, and 9.2x is the largest factor verified end-to-end on the exported artifact. Targets outside the training range still run, but quality degrades.

### Tested scale range

**9.2x is the scale ceiling this repository exposes** in the Space slider and in the CLI example: it is the largest factor recommended for these weights. The exported artifact was nevertheless measured end-to-end at every scale of the sweep below, so the numbers beyond 9.2x are measurements, not guesses - they just are not supported by the demo.

Sweep from `scripts/benchmark.py`: 4 crops of 256px taken from the validation set and degraded with the Real-ESRGAN pipeline, run through tiled inference at 2x-10x. Model and bicubic are measured on the *same* degraded input; higher is better:

| Scale | PSNR model | PSNR bicubic | Δ PSNR | SSIM model | SSIM bicubic | Composite model | Composite bicubic |
|---|---|---|---|---|---|---|---|
| 2x | 29.26 | 27.97 | +1.29 | 0.7552 | 0.7178 | 0.6252 | 0.5903 |
| 3x | 27.07 | 26.33 | +0.73 | 0.6760 | 0.6435 | 0.5289 | 0.5000 |
| 4x | 28.36 | 27.87 | +0.49 | 0.7089 | 0.6968 | 0.5859 | 0.5710 |
| 5x | 27.52 | 27.15 | +0.38 | 0.6778 | 0.6646 | 0.5592 | 0.5457 |
| 6x | 25.36 | 24.71 | +0.65 | 0.6197 | 0.5920 | 0.4725 | 0.4496 |
| 7x | 26.05 | 25.60 | +0.45 | 0.6145 | 0.6020 | 0.4958 | 0.4834 |
| 8x | 24.75 | 24.18 | +0.57 | 0.5756 | 0.5607 | 0.4247 | 0.4031 |
| 9x | 23.71 | 23.38 | +0.33 | 0.5537 | 0.5423 | 0.3875 | 0.3736 |
| 9.2x | 25.11 | 24.60 | +0.52 | 0.5675 | 0.5619 | 0.4482 | 0.4352 |
| 10x | 24.42 | 24.03 | +0.39 | 0.5546 | 0.5491 | 0.4128 | 0.4005 |

Every scale is measured on its own randomly degraded crop, so absolute PSNR wobbles by roughly +/-1 dB between neighbouring scales (9x and 9.2x even land on the same integer LR grid). The delta columns - model vs bicubic on the identical input - are the numbers to trust.

## Benchmarks

Measured on DIV2K validation crops: the HR crop is bicubic-downscaled by 4x,
then the model has to reconstruct it. Higher is better.

| Crop | LR size | PSNR bic | **PSNR model** | SSIM bic | **SSIM model** | Detail bic | **Detail model** |
|---|---|---|---|---|---|---|---|
| 0801.png | 128x128 | 32.35 | **32.63** | 0.8845 | **0.8809** | 0.102 | **0.235** |
| 0802.png | 128x128 | 31.21 | **31.04** | 0.8267 | **0.8128** | 0.068 | **0.100** |
| 0803.png | 128x128 | 30.80 | **31.34** | 0.8733 | **0.8817** | 0.131 | **0.252** |
| 0804.png | 128x128 | 25.84 | **26.07** | 0.7229 | **0.7284** | 0.199 | **0.401** |
| 0805.png | 128x128 | 27.77 | **27.59** | 0.7030 | **0.6846** | 0.134 | **0.176** |

Average over 5 crops of 512x512 px:

| Metric | Bicubic baseline | This model |
|---|---|---|
| PSNR | 29.59 dB | **29.74 dB** |
| SSIM | 0.8021 | **0.7977** |
| Detail (Laplacian energy) | 0.127 | **0.233** |

### Come leggere questi numeri / Reading the numbers

PSNR e SSIM misurano la **fedelta pixel per pixel**, e premiano le ricostruzioni lisce:
una bicubica e difficilissima da battere su queste due metriche.
La colonna "Detail" e l'energia di una risposta Laplaciana (piu alta = piu texture): il modello
rigenera circa 1.8x l'energia ad alta frequenza della bicubica, e questo spiega
perche le immagini risultano piu nitide. Su questo checkpoint la PSNR media e superiore alla bicubica (+0.14 dB) e la SSIM -0.0044. Questo modello e stato addestrato anche con una loss adversariale, che scambia fedelta pixel con texture visibile.

PSNR and SSIM measure **pixel fidelity** and reward smooth interpolation, so a bicubic
resample is hard to beat on them. On this checkpoint the average PSNR is higher than bicubic (+0.14 dB) and SSIM -0.0044. This checkpoint was trained with an adversarial loss, which trades pixel fidelity for visible texture. If you need the numerically closest
reconstruction, use bicubic; if you need visible detail, use the model. Look at
`examples/compare.jpg` rather than trusting a single number.

Exported artifact verified against the original pipeline: max absolute difference
**1.19e-07** (see `examples/verification.json`).

### Demo proof

Generated with the exported `model_scripted.pt` from `a.png`:

- `examples/demo_input.png` (993x747)
- `examples/demo_output_4x.jpg` (3972x2988)
- `examples/demo_compare.jpg` zoomed side-by-side against bicubic

This proves the published artifact runs end-to-end on its own: no training code is required.

## Training code

This model repository is weights-only: no training source is distributed with it. `model_scripted.pt` is self-contained, so the network graph, the tiled inference logic and the trained weights all travel inside the artifact itself, and `model.safetensors` can be loaded by any independent implementation of the architecture described in `config.json`.

## Training summary

What the weights are the product of:

- Architecture: `LIIFNet (simplified Local Implicit Image Function)` (4.60 M parameters)
- Training data: DIV2K train (800 HR images) + Flickr2K (2650 HR images), paired with synthetic LR images generated by a Real-ESRGAN-style degradation pipeline (blur, noise, downsampling, JPEG artefacts)
- Losses: L1 + edge (Sobel gradient) + PatchGAN adversarial
- Optimisation: Adam, 4000 iterations, batch 2, patch 192, lr 4e-05 -> 2e-05 (cosine), FP32, seed 42
- Batch sampling: 16 homogeneous (hr_size, lr_size) buckets, so every sample in a batch shares one implicit-scale grid and the arbitrary-scale decode is exact
- Fine-tuned: warm-started from `checkpoints/liif_v4/best_liif.pth` (previous stage: 8000 iterations at lr 0.0002; the iteration budget above is the fine-tuning stage only)
- Exported from checkpoint step 3500 (best_liif.pth)



## Limitations



- The Space slider stops at **9.2x**, the largest factor recommended for these weights

  (see "Tested scale range" above). Past the training range the model still beats the bicubic

  reference at every measured scale, but the margin shrinks and the output gets smoother.

- Scales beyond the tested maximum are still technically supported (the decoder can sample any

  coordinate grid) but the output becomes progressively smoother/blurrier: the model

  interpolates, it does not hallucinate new detail like a diffusion prior would.

- Inputs with heavy JPEG artefacts or motion blur are handled better than pure bicubic

  downscaling, but the model is not a denoiser.

- Alpha channels are not upscaled: convert RGBA to RGB first.

- These are research checkpoints, not a production replacement for ESRGAN/Real-ESRGAN: on some

  content the generator over-sharpens and invents texture (see the "Detail" column, where the

  model can exceed the ground truth).



## Uso rapido (italiano)



```bash

pip install -r requirements.txt

python upscale.py --input foto.jpg --output foto_4x.png --scale 4
python upscale.py --input foto.jpg --output foto_grande.png --target-long-side 3000

```



Il file `model_scripted.pt` contiene gia architettura e pesi: non serve nessun codice di
training. `model.safetensors` e disponibile se vuoi usare i pesi in una tua implementazione.

## License

- **Weights:** released under `cc-by-nc-4.0`. The training data (DIV2K + Flickr2K) is licensed
  for non-commercial research use, so treat these weights as research-only as well.
- **Code:** no source code is distributed here; the TorchScript artifact `model_scripted.pt` and the `model.safetensors` weights share the same `cc-by-nc-4.0` terms.

## Citation

```bibtex

@misc{universal_upscale,

  author = {Vincenzo Tilotta},

  title  = {Universal Upscale - arbitrary-scale super-resolution (LIIF)},

  year   = {2026},

  url    = {https://huggingface.co/tailot/universal-upscale}

}

```