| --- |
| library_name: pytorch |
| license: cc-by-nc-4.0 |
| tags: |
| - super-resolution |
| - image-restoration |
| - latent-diffusion |
| - pytorch |
| - research |
| --- |
| |
| # LuSIR |
|
|
| **LuSIR: Latent Upscaling via Self-trained Image Restoration** is a |
| vision-only x4 super-resolution research project trained without a pretrained |
| text-to-image diffusion model. |
|
|
| GitHub: <https://github.com/BitIntx/LuSIR> |
|
|
| The repository stores selected research checkpoints, configs, metrics, and |
| sample grids. It does not redistribute training datasets. |
|
|
| ## Current Selected Detail Artifact |
|
|
| The latest public stable detail-branch checkpoint remains: |
|
|
| ```text |
| checkpoints/detail_branch_v1d_deep3m_photo130k_lsdir_best99500.pt |
| ``` |
|
|
| It is a deterministic 3.02M-parameter image-space detail branch on top of the |
| frozen dual-context LSDIR Stage 2 step 98000 condition encoder and frozen Stage |
| 1 decoder. The run completed `100086` micro-steps, exactly three epochs, and |
| selected step `99500` by `eval/detail_score`. |
|
|
| Selected ordinary `photo_detail_mix` val100 result: |
|
|
| ```text |
| aggregate PSNR delta vs frozen base: +0.1646 dB |
| mean PSNR delta vs frozen base: +0.1888 dB |
| SSIM delta vs frozen base: +0.00647 |
| PSNR wins: 99/100 |
| detail wins: 100/100 |
| ``` |
|
|
| Exploratory strict-bicubic DIV2K five-center-crop result: |
|
|
| ```text |
| mean RGB PSNR: 31.9513 dB |
| vs frozen base: +0.2102 dB |
| vs detail v1c: +0.1358 dB |
| wins: 5/5 |
| ``` |
|
|
| The strict-bicubic result is not a formal SOTA benchmark. It uses five |
| 512x512 center crops, PIL bicubic x4 degradation, full-image RGB PSNR, and no |
| border shave. |
|
|
| Formal full-image clean-bicubic benchmark, reported as Y PSNR / Y SSIM: |
|
|
| | Dataset | Dual-context base | Detail v1d | |
| | --- | ---: | ---: | |
| | DIV2K validation | 29.9575 / 0.82887 | **30.1602 / 0.83421** | |
| | Set5 | 31.6621 / 0.88952 | **31.8892 / 0.89440** | |
| | Set14 | 28.2441 / 0.77340 | **28.4123 / 0.77998** | |
| | Urban100 | 25.4816 / 0.76473 | **25.8755 / 0.77875** | |
|
|
| This uses public x4 LR pairs, MATLAB-compatible BT.601 Y, a four-pixel border |
| shave, and MATLAB-style SSIM. V1d improves its frozen base on all four |
| datasets. These clean-bicubic fidelity results are not a claim of classical-SR |
| SOTA or a substitute for real-degradation and perceptual evaluation. |
|
|
| For scale, the official SwinIR classical x4 checkpoint reaches |
| `31.0838 / 0.85228` on the same DIV2K evaluator, `+0.9235 dB` Y PSNR ahead of |
| detail v1d. The next clean-fidelity priority is therefore the Stage 2/base |
| reconstruction path rather than a larger detail branch. |
|
|
| A clean-bicubic Stage 2 continuation improved its task-specific val100 proxy |
| only gradually and plateaued around `25.05`. Learning-rate probes did not |
| change that conclusion: `20x` LR collapsed, while a `5x` from-init run matched |
| the original LR within evaluation noise. These val100 values are not directly |
| comparable with the formal full-image Y-channel benchmark above. |
|
|
| The signed-high-frequency residual diffusion path was evaluated and rejected: |
| longer noise-MSE training collapsed residual magnitude and seed diversity |
| toward zero. The next separate generative research path keeps the |
| deterministic base and validated learned mask frozen, then tests a small |
| bounded mask-weighted patch perceptual/adversarial head with fidelity and |
| artifact guardrails. |
|
|
| ## Latest Masked Detail Research Candidate |
|
|
| The learned-mask-gated v2 candidate is: |
|
|
| ```text |
| checkpoints/detail_branch_v2_masked_photo130k_lsdir_best38000.pt |
| ``` |
|
|
| It combines the frozen 460K-parameter detail-mask predictor step 3250 with the |
| 3.02M-parameter detail branch and a soft-mask floor of `0.05`. On ordinary |
| `photo_detail_mix` val100, selected step 38000 improves the frozen base by |
| `+0.18177 dB` aggregate PSNR, `+0.20432 dB` mean PSNR, and `+0.00755` SSIM, |
| with `100/100` wins. |
|
|
| The score plateaued after step 38000 and fixed grids were nearly |
| indistinguishable from nearby checkpoints. It modestly improves metrics over |
| v1d but does not visibly recover the missing fine texture that motivated the |
| experiment. It is therefore a reproducible research option, not the public |
| default. |
|
|
| On the same formal 219-image clean-bicubic benchmark, masked v2 reaches |
| `30.1636 / 0.83512`, `31.9495 / 0.89534`, `28.4257 / 0.78102`, and |
| `25.8922 / 0.78022` on DIV2K, Set5, Set14, and Urban100. It improves v1d on |
| all four datasets, but the overall gain is only `+0.0114 dB` Y PSNR and |
| `+0.00118` Y SSIM. |
|
|
| ## Download |
|
|
| From a LuSIR GitHub clone: |
|
|
| ```bash |
| python scripts/download_hf_checkpoints.py --preset detail_branch_v1d |
| ``` |
|
|
| Other useful presets include: |
|
|
| ```text |
| stage2_guarded_detail_v2 |
| residual_refiner_v2 |
| stage2_photo130k_lsdir_dual |
| detail_branch_v1b |
| detail_branch_v2_masked |
| photo100k_xl_stage4_edge |
| ``` |
|
|
| The public Colab default is now the T4-friendly guarded-detail Stage 2 v2 |
| checkpoint with tile batch size 1. Residual refiner v2 remains available as the |
| conservative deterministic option. Detail v1d and masked detail v2 remain |
| research options in the Colab WebUI with single-image and tiled inference. |
|
|
| ## Runtime Paths |
|
|
| ```text |
| public deterministic default: |
| LR -> guarded-detail Stage 2 v2 step 10000 -> Stage 1 decoder -> SR |
| |
| conservative deterministic option: |
| LR -> Stage 2 XL -> residual refiner v2 -> Stage 1 decoder -> SR |
| |
| selected detail research path: |
| LR -> dual-context LSDIR Stage 2 -> Stage 1 decoder |
| -> learned detail mask -> masked detail branch v2 -> SR |
| |
| generative comparison: |
| LR -> Stage 2 condition encoder -> Stage 3 OR Stage 4 diffusion U-Net |
| -> Stage 1 decoder -> SR |
| ``` |
|
|
| Stage numbers describe training order. Stage 3 and Stage 4 are alternative |
| diffusion checkpoints, not modules executed sequentially. |
|
|
| ## License |
|
|
| - Checkpoints, generated samples, metrics, and other non-code artifacts: |
| CC BY-NC 4.0. |
| - Source code: PolyForm Noncommercial License 1.0.0. |
|
|
| Commercial use is not permitted without separate written permission. |
|
|
