İvme-Denoise-v1 (İvmetron-Guanine1): 2.97M-param Bayesian shrinkage denoiser
Browse files- .gitattributes +4 -0
- README.md +35 -0
- assets/crops.jpg +3 -0
- assets/example-park.jpg +3 -0
- assets/gains.png +3 -0
- assets/hero.jpg +3 -0
- assets/pareto.png +0 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
assets/crops.jpg filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
assets/example-park.jpg filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
assets/gains.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
assets/hero.jpg filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -29,10 +29,15 @@ It is also the first release built on a *derived* architecture rather than a tun
|
|
| 29 |
|
| 30 |
The codename is a **margay** — a small nocturnal wild cat with the largest eyes-to-body ratio of any cat. It denoises the dark for a living.
|
| 31 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
---
|
| 33 |
|
| 34 |
## Contents
|
| 35 |
|
|
|
|
| 36 |
- [What this is](#what-this-is)
|
| 37 |
- [Model details](#model-details)
|
| 38 |
- [The architecture: İvmetron-Guanine1](#the-architecture-ivmetron-guanine1)
|
|
@@ -68,6 +73,26 @@ The practical consequence: a network that is handed the correct shrinkage law an
|
|
| 68 |
|
| 69 |
---
|
| 70 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
## Model details
|
| 72 |
|
| 73 |
| Parameter | Value |
|
|
@@ -160,6 +185,12 @@ Note what is **absent**: no additive bias after the gain, no normalisation, no a
|
|
| 160 |
|
| 161 |
`Proj` is zero-initialised, so every block starts as the identity and the network starts as a no-op. All the dB are learned, not inherited.
|
| 162 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 163 |
### Design decisions that mattered
|
| 164 |
|
| 165 |
**No bias in the shrinkage.** The single biggest inductive-bias win. A conv with bias can add energy where the prior says there is none; the GSM estimator cannot.
|
|
@@ -263,6 +294,10 @@ At σ = 50 that is **+2.2 dB over BM3D** — and BM3D takes ~3.3 s per image whe
|
|
| 263 |
|
| 264 |
**Accuracy per GFLOP: 0.553 dB/GFLOP** for this model versus 0.225 for Restormer — a **2.5× efficiency advantage**, at 11 % of the parameters and a third of the training time.
|
| 265 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 266 |
Latency, RTX PRO 6000, bf16, batch 1, no tiling:
|
| 267 |
|
| 268 |
| resolution | time |
|
|
|
|
| 29 |
|
| 30 |
The codename is a **margay** — a small nocturnal wild cat with the largest eyes-to-body ratio of any cat. It denoises the dark for a living.
|
| 31 |
|
| 32 |
+

|
| 33 |
+
|
| 34 |
+
*A 12.3 MP high-ISO frame, one pass, no per-image tuning. Left: original. Right: `denoise_auto` with tiled inference, 6.5 s.*
|
| 35 |
+
|
| 36 |
---
|
| 37 |
|
| 38 |
## Contents
|
| 39 |
|
| 40 |
+
- [Examples](#examples)
|
| 41 |
- [What this is](#what-this-is)
|
| 42 |
- [Model details](#model-details)
|
| 43 |
- [The architecture: İvmetron-Guanine1](#the-architecture-ivmetron-guanine1)
|
|
|
|
| 73 |
|
| 74 |
---
|
| 75 |
|
| 76 |
+
## Examples
|
| 77 |
+
|
| 78 |
+
All three crops below are 145 % magnifications of the same frame, straight out of the model with a σ map it estimated itself.
|
| 79 |
+
|
| 80 |
+

|
| 81 |
+
|
| 82 |
+
**Tunnel wall.** The original carries heavy chroma speckle in a region that is essentially black. It is gone, and the wall is black rather than blue-grey. This is the case that a conventional denoiser either leaves noisy or turns into a checkerboard.
|
| 83 |
+
|
| 84 |
+
**Train front.** The headlights are blown to clipping and the livery sits in near-darkness. The edge of the front panel survives, the "34" stays legible, and the highlight keeps its shape instead of blooming.
|
| 85 |
+
|
| 86 |
+
**Tactile strip.** Saturated yellow under sodium light, which is where JPEG chroma subsampling does the most damage. The grain is removed without the plastic-looking flatness you get from a heavy Gaussian blur.
|
| 87 |
+
|
| 88 |
+
A second frame, a park scene in daylight, denoised at σ = 6 — the value its own noise estimator picked:
|
| 89 |
+
|
| 90 |
+

|
| 91 |
+
|
| 92 |
+
The gentle setting is the interesting one: at σ = 6 the model has almost nothing to do, and it correctly does almost nothing. Detail is preserved because the energy gate never closes.
|
| 93 |
+
|
| 94 |
+
---
|
| 95 |
+
|
| 96 |
## Model details
|
| 97 |
|
| 98 |
| Parameter | Value |
|
|
|
|
| 185 |
|
| 186 |
`Proj` is zero-initialised, so every block starts as the identity and the network starts as a no-op. All the dB are learned, not inherited.
|
| 187 |
|
| 188 |
+
Here are the 192 learned gains of one block, plotted against `log e`. This is the *entire* non-linearity of the model — there is nothing else:
|
| 189 |
+
|
| 190 |
+

|
| 191 |
+
|
| 192 |
+
Every curve is a monotone gain that runs from ~1 at low energy to ~0 at high energy, with the family spread out across channels because the learned frame is overcomplete and is not the eigenbasis of the patch covariance. Anything that looks like a familiar activation would have to be justified by the GSM posterior mean; none of them are, so none of them are here.
|
| 193 |
+
|
| 194 |
### Design decisions that mattered
|
| 195 |
|
| 196 |
**No bias in the shrinkage.** The single biggest inductive-bias win. A conv with bias can add energy where the prior says there is none; the GSM estimator cannot.
|
|
|
|
| 294 |
|
| 295 |
**Accuracy per GFLOP: 0.553 dB/GFLOP** for this model versus 0.225 for Restormer — a **2.5× efficiency advantage**, at 11 % of the parameters and a third of the training time.
|
| 296 |
|
| 297 |
+

|
| 298 |
+
|
| 299 |
+
Read the plot as: anything to the upper-left dominates. İvme-Denoise-v1 is above DnCNN at 65 % of its compute, and sits well to the left of the transformer cluster — 0.35 dB costs you 2.5× the FLOPs.
|
| 300 |
+
|
| 301 |
Latency, RTX PRO 6000, bf16, batch 1, no tiling:
|
| 302 |
|
| 303 |
| resolution | time |
|
assets/crops.jpg
ADDED
|
Git LFS Details
|
assets/example-park.jpg
ADDED
|
Git LFS Details
|
assets/gains.png
ADDED
|
Git LFS Details
|
assets/hero.jpg
ADDED
|
Git LFS Details
|
assets/pareto.png
ADDED
|