File size: 24,397 Bytes
ca179f0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
# MASTER PROMPT β€” "LumenX": Recoverability-Aware Adaptive Low-Light Restoration Engine

> Paste this whole file into your coding agent (Claude Code / Cursor / etc.) as the project brief.
> Then build milestone by milestone (Section 12). Do not let the agent skip ahead.

---

## 0. Your role and mission

You are a senior computer-vision + backend engineer building a hackathon system in **24 hours** with a team of 3.
Build **LumenX**, a low-light image restoration engine that:

1. **Diagnoses** each image (darkness, light sources, noise, colour cast, clipping).
2. **Adapts** its parameters per image with a quality-aware self-tuning search (no fixed parameters).
3. **Fuses** classical Retinex processing with a deep model (HVI-CIDNet), pixel by pixel.
4. **Denoises in proportion to the gain**: noise is amplified by the brightening factor, so denoise strength follows the gain map.
5. **Knows what it can't recover**: it outputs a per-pixel *Recoverability Map* (recovered / estimated / unrecoverable) and never invents detail where there was no real signal.
6. **Explains itself**: it streams every stage live and writes a plain-English report a non-technical person can understand.

**Primary principle:** the backend is the product. It must return *rich, structured, visual data* (maps, stages, decisions, candidates, metrics, timings) so that **any** frontend can turn it into a stunning experience. The included frontend is deliberately basic; the team will redesign it later, so it must be fully decoupled.

**Second principle:** a working, reliable system beats an ambitious broken one. Every milestone must end in a runnable state.

---

## 1. Hard constraints

- GPU: **NVIDIA RTX 2050, 4 GB VRAM** (laptop). CPU fallback must work (slower).
- Python 3.10/3.11, PyTorch (CUDA), OpenCV-contrib (`cv2.ximgproc`), scikit-image, numpy, pyiqa, FastAPI, uvicorn, pydantic v2, pyyaml, pandas, matplotlib.
- Everything free/open-source. No paid APIs. No LLM calls at runtime (report text is template-based).
- VRAM safety: FP16 inference, tiled inference for images > 1 MP (tile 512, overlap 32, feathered blending), `torch.no_grad()`, load models once at startup, never hold more than one large model's activations at once.
- **Latency budget (1080p, GPU): ≀ 1.5 s end-to-end** for `/enhance` (excluding first-call warmup). Measure and report per-stage timings.
- Deterministic: fixed seeds, same input gives the same output.

---

## 2. Repository structure (create exactly this)

```

lumenx/

β”œβ”€β”€ README.md                      # setup, run, demo script, architecture diagram (mermaid)

β”œβ”€β”€ config.yaml                    # ALL thresholds/params live here, none hard-coded

β”œβ”€β”€ requirements.txt

β”œβ”€β”€ weights/                       # model weights (gitignored) + README with download links

β”œβ”€β”€ scripts/

β”‚   β”œβ”€β”€ download_weights.py        # downloads what it can; prints manual instructions for the rest

β”‚   β”œβ”€β”€ eval.py                    # benchmark: methods Γ— datasets Γ— metrics β†’ CSV/MD/plots

β”‚   β”œβ”€β”€ ablation.py                # component on/off study β†’ CSV/MD/plots

β”‚   β”œβ”€β”€ failure_mining.py          # auto-collect worst cases + reason tags

β”‚   └── warmup.py

β”œβ”€β”€ lumenx/

β”‚   β”œβ”€β”€ __init__.py

β”‚   β”œβ”€β”€ pipeline.py                # orchestrator; emits stage events; supports ablation flags

β”‚   β”œβ”€β”€ diagnose.py                # illumination, noise Οƒ, SNR, cast, non-uniformity, clipping

β”‚   β”œβ”€β”€ classical.py               # Retinex decomposition/enhancement, CLAHE, colour correction

β”‚   β”œβ”€β”€ deep/

β”‚   β”‚   β”œβ”€β”€ cidnet.py              # HVI-CIDNet wrapper (FP16, tiled)

β”‚   β”‚   └── zerodce.py             # Zero-DCE++ wrapper (live mode)

β”‚   β”œβ”€β”€ fusion.py                  # per-pixel fusion weights + guided-filter smoothing

β”‚   β”œβ”€β”€ denoise.py                 # gain-aware multi-strength guided/bilateral denoise

β”‚   β”œβ”€β”€ recoverability.py          # SNR-based recoverability map + gain capping

β”‚   β”œβ”€β”€ tuner.py                   # quality-aware parameter search on proxy image

β”‚   β”œβ”€β”€ guardrails.py              # overexposure / colour shift / edge loss checks + retry

β”‚   β”œβ”€β”€ metrics.py                 # pyiqa wrappers + custom edge/noise/Ξ”E metrics

β”‚   β”œβ”€β”€ report.py                  # plain-English diagnosis + decision narrative + story steps

β”‚   β”œβ”€β”€ viz.py                     # colormaps, overlays, 3D-ready heightmap export, PNG encoding

β”‚   └── utils.py                   # io, timing context manager, tiling, color conversions

β”œβ”€β”€ api/

β”‚   β”œβ”€β”€ main.py                    # FastAPI app, static serving, CORS

β”‚   β”œβ”€β”€ schemas.py                 # pydantic response models (the frontend contract)

β”‚   β”œβ”€β”€ routes_enhance.py          # POST /enhance, GET /enhance/stream (SSE), GET /runs/{id}

β”‚   β”œβ”€β”€ routes_live.py             # WS /live (webcam, Zero-DCE++)

β”‚   β”œβ”€β”€ routes_meta.py             # /health, /gallery, /benchmarks, /config

β”‚   └── mock.py                    # MOCK MODE: returns realistic fake payloads instantly

β”œβ”€β”€ frontend/                      # BASIC, replaceable (see Section 10)

β”‚   β”œβ”€β”€ index.html

β”‚   β”œβ”€β”€ app.js

β”‚   β”œβ”€β”€ relight.js                 # WebGL relighting shader

β”‚   └── theme.css                  # all visual styling via CSS variables only

β”œβ”€β”€ gallery/                       # 8–10 curated hard demo images + metadata.json

β”œβ”€β”€ runs/                          # per-request outputs (gitignored), served statically

β”œβ”€β”€ results/                       # eval/ablation outputs (CSV, MD, PNG charts, failure gallery)

β”œβ”€β”€ docs/

β”‚   β”œβ”€β”€ API.md                     # full frontend contract with example payloads

β”‚   β”œβ”€β”€ ARCHITECTURE.md

β”‚   └── FAILURE_ANALYSIS.md        # generated + hand-annotated

└── tests/

    β”œβ”€β”€ test_pipeline_shapes.py

    β”œβ”€β”€ test_api_contract.py

    └── test_guardrails.py

```

---

## 3. Pipeline: exact algorithms

All images internally are float32 RGB in [0,1]. `Y` = luminance (BT.709). Every stage records: output image (or map), duration (ms), a one-line human caption, and key numbers.

### 3.1 Diagnose (`diagnose.py`), target < 30 ms at 1080p

Compute on a 512px-long-side proxy where possible, then upsample maps.

- **Illumination map L** (LIME-style): `L0 = max(R,G,B)`, refined with `cv2.ximgproc.guidedFilter(guide=Y, src=L0, radiusβ‰ˆ15, epsβ‰ˆ1e-3)`. Clip to [Ξ΅, 1].
- **Brightness level**: mean(Y), median(Y), % pixels with Y < 0.05 (β€œcrushed shadows”).
- **Noise Οƒ**: `skimage.restoration.estimate_sigma` on Y (or MAD of the Laplacian over low-gradient pixels). Also a **local noise map** (Οƒ in 32Γ—32 blocks over flat regions, interpolated).
- **SNR map**: `SNR = local_mean(Y, 15Γ—15) / (Οƒ_local + Ξ΅)`.
- **Light sources / non-uniformity**: threshold L > 0.85 β†’ connected components β†’ count + bounding boxes (β€œhotspots”). Non-uniformity score = std(log L) normalised.
- **Colour cast**: convert to Lab; mean a*, b* over mid-tones; cast = direction + magnitude (e.g. "warm/orange", "green", "blue").
- **Clipping**: % pixels with any channel > 0.98 (highlights) and < 0.02 (shadows).
- **Severity class**: `extreme` / `severe` / `moderate` / `mild` from thresholds in config.

Output a `Diagnosis` object (numbers + labels + severity per item + hotspot boxes).

### 3.2 Recoverability (`recoverability.py`), the main differentiator

- From the **input** SNR map, classify each pixel:
  - `recovered` (SNR β‰₯ Ο„_high): real signal; enhance fully.

  - `estimated` (Ο„_low ≀ SNR < Ο„_high): partial signal; enhance with stronger denoise.

  - `unrecoverable` (SNR < Ο„_low): essentially noise; **cap the gain**, denoise strongly, and do not let the DL output hallucinate texture there (fusion weight shifts toward a smooth classical result).
- Smooth class boundaries (guided filter on a soft score in [0,1]), then quantise for display.
- Output: soft map (float), 3-class map, % area per class, and an RGBA overlay PNG (green / amber / red, alpha 0.45).
- Report line example: *"18% of this image had too little light to recover real detail. LumenX smoothed those areas instead of inventing texture."*

### 3.3 Classical path (`classical.py`)

- Retinex: `R = I / L` (per channel, L broadcast), clipped. Adjusted illumination `L' = L^Ξ³` (Ξ³ from tuner, typically 0.3–0.7), with highlight protection (where L > 0.8, blend L' back toward L).
- Recompose `E_c = R Β· L'`.
- CLAHE on the Lab L-channel (clipLimit and tile size from tuner).
- Colour correction: partial gray-world in Lab with strength ∝ cast magnitude (never full correction; preserve scene mood such as warm street lights).
- Keep R and L' as outputs (needed for relighting).

### 3.4 Deep path (`deep/cidnet.py`)

- **HVI-CIDNet** (official repo: `github.com/Fediory/HVI-CIDNet`). Load pretrained weights:
  - LOLv2-real weights for `moderate/severe`, **SID (Sony-Total-Dark) weights for `extreme`** darkness. Selection is automatic, based on diagnosis.
- FP16, tiled inference (tile 512, overlap 32, cosine-feathered blending), pad to a multiple of the network stride.
- If weights are missing: log a clear warning and fall back to Zero-DCE++ so the system never crashes.

### 3.5 Live path (`deep/zerodce.py`)

- **Zero-DCE++** (official Zero-DCE extension repo weights). Used ONLY for `/live` webcam streaming. Target β‰₯ 20 FPS at 640Γ—480 on RTX 2050.

### 3.6 Fusion (`fusion.py`)

Per-pixel weights (sum to 1) between `E_dl` (deep), `E_c` (classical), and `I_safe` (input, gamma-lifted gently):
- `w_dl` high in dark regions with recoverable signal: `sigmoid((t_dark βˆ’ L)/s) Β· rec_soft`.
- `w_c` high in mid-tones and in `unrecoverable` zones (smooth, no hallucination).
- `w_safe` high in highlights / hotspots (L > 0.8), to prevent blow-out and halos around lamps.
- **Smooth all weight maps with a guided filter guided by input Y** (not Gaussian), to avoid seams and halos.
- Global fusion bias `Ξ±` comes from the tuner.
- Output: fused image + the 3 weight maps (export as one RGB "fusion map": R=dl, G=classical, B=safe).

### 3.7 Gain-aware denoising (`denoise.py`)

- Gain map `G = Y_fused / (Y_in + Ξ΅)`, clipped.
- Denoise strength map `S = clip(k Β· G Β· Οƒ_local / (Οƒ_ref), 0, 1)`, boosted in `estimated`/`unrecoverable` zones.
- Efficient trick: compute 3 denoised versions (light / medium / strong guided-filter or bilateral; optionally fastNlMeans on the strong one at reduced resolution) and **blend per pixel by S**. Denoise luminance more than chroma, but also suppress chroma noise.
- Edge protection: reduce S where the input gradient magnitude is high and SNR is good.
- Output: denoised image, gain map, strength map.

### 3.8 Self-tuning search (`tuner.py`), the adaptive strategy

- Work on a 256px proxy. Search space (from config): Ξ³, CLAHE clip, denoise k, fusion Ξ±, colour-correction strength.
- Generate ~12–16 candidates (Latin-hypercube sample seeded by diagnosis-based priors, e.g. extreme darkness β†’ lower Ξ³ prior).
- Run the cheap proxy pipeline per candidate (deep output computed ONCE on the proxy and reused; only cheap steps vary).
- Score = `w1Β·CLIP-IQA + w2Β·(βˆ’NIQE_norm) βˆ’ w3Β·clip_pct βˆ’ w4Β·Ξ”E_shift + w5Β·edge_retention` (weights in config).
- Pick the best; run the full-resolution pipeline with it.
- Return **all candidates** (params, sub-scores, total, 128px thumbnail, chosen flag) for the frontend to visualise. Target < 400 ms total.

### 3.9 Guardrails (`guardrails.py`)

Check final output:
- Overexposure: highlight clip % > threshold (and > input's clip % + margin).
- Colour shift: mean Ξ”E2000 between input chroma direction and output (on mid-tones) > threshold.
- Detail loss: edge retention (correlation of Sobel magnitudes, input-normalised) < threshold.
- Halo check: overshoot around hotspot boundaries.
If any check fails, retry once with the next-best candidate or safer params (higher Ξ³, lower Ξ±, stronger highlight protection). Log `{check, value, threshold, action}` and include it in the response and report. Never fail silently.

### 3.10 Report and story (`report.py`)

Template-based, no LLM. Produces:
- `headline`: one sentence (e.g. *"Very dark street scene with 3 bright lamps and heavy sensor noise, restored with shadows lifted 6.2Γ— and lamps protected."*).
- `diagnosis_chips`: short labels with severity (`high/med/low`) and the value.
- `decisions`: 4–7 plain-English bullet points explaining WHAT was done and WHY.
- `story`: ordered list of steps `{title, caption (≀ 20 words, no jargon), image_url, highlight_map_url?}` so any frontend can animate the "watch it think" reveal.
- `recoverability_summary`: % per class + one honest sentence.
- `jargon_free`: true. Every caption must be understandable by a non-technical person; technical names go in a separate `tech_note` field.

---

## 4. Outputs per run

Saved to `runs/<run_id>/` and served at `/runs/<run_id>/...`:
- `input.png`, `final.png` (full-res), `final_preview.jpg` (≀ 1280px)
- Stage images (≀ 768px): `illumination.png` (colormapped), `reflectance.png`, `classical.png`, `deep.png`, `fused.png`, `denoised.png`, `final.png`
- Maps (≀ 512px): `L_gray.png` (8-bit, for WebGL relight), `snr.png`, `recoverability_overlay.png` (RGBA), `gain.png`, `denoise_strength.png`, `fusion_map.png`, `noise_map.png`
- `heightmap.json` or `L_height.png` (downsampled 128Γ—128 illumination for 3D terrain visualisations)
- `candidates/*.jpg` thumbnails
- `result.json` (full response)

---

## 5. API (`api/`): the frontend contract

All responses are typed with pydantic models in `schemas.py` and documented in `docs/API.md` with full example payloads.

- `POST /enhance` (multipart `file`, optional form: `mode=hybrid|classical|dl`, `ablate=[...]` component flags, `mock=bool`) β†’ `EnhanceResponse`
- `POST /enhance/stream` β†’ **Server-Sent Events**. Emit an event as each stage finishes: `{"event":"stage","name":"diagnose","caption":"...","image_url":"...","data":{...},"t_ms":12}`, then `{"event":"candidate",...}` for each tuner candidate, then `{"event":"done","result":EnhanceResponse}`. This drives live "watch it think" UIs. **Required.**
- `GET /runs/{run_id}` β†’ stored result.
- `GET /gallery` β†’ curated demo images with titles and difficulty tags.
- `GET /benchmarks` β†’ eval + ablation tables (JSON) and chart URLs from `results/`.
- `GET /config` β†’ current tunable params (read-only).
- `GET /health` β†’ device, VRAM, models loaded, warm status.
- `WS /live` β†’ client sends JPEG frames, server returns enhanced JPEG frames + fps + latency.

`EnhanceResponse` (minimum fields):
```json

{

  "run_id": "str",

  "mode": "hybrid",

  "input": {"url": "...", "width": 1920, "height": 1080},

  "final": {"url": "...", "preview_url": "..."},

  "headline": "str",

  "diagnosis": {"chips": [{"label":"Very dark","value":0.07,"severity":"high"}], "raw": {...}, "hotspots":[[x,y,w,h]]},

  "stages": [{"name":"illumination","title":"Where the light is","caption":"...","tech_note":"...","image_url":"...","t_ms":9}],

  "maps": {"illumination_gray":"...","snr":"...","recoverability_overlay":"...","gain":"...","fusion":"...","denoise_strength":"...","noise":"...","heightmap":"..."},

  "relight": {"base_url":"final.png","L_url":"L_gray.png","gamma_default":1.0,"exposure_default":1.0},

  "recoverability": {"recovered_pct":71.2,"estimated_pct":17.5,"unrecoverable_pct":11.3,"sentence":"..."},

  "tuner": {"candidates":[{"id":0,"params":{},"scores":{},"total":0.71,"thumb_url":"...","chosen":true}],"search_ms":310},

  "guardrails": [{"check":"overexposure","value":0.4,"threshold":2.0,"passed":true,"action":null}],

  "decisions": ["..."],

  "story": [{"title":"...","caption":"...","image_url":"...","highlight_map_url":null}],

  "metrics": {"before":{"niqe":6.8,"clipiqa":0.22,"brisque":48.1},"after":{"niqe":3.6,"clipiqa":0.67,"brisque":21.4},"edge_retention":0.93,"noise_sigma":{"before":0.041,"after":0.012},"delta_e":3.1},

  "timings_ms": {"diagnose":14,"tuner":310,"deep":420,"classical":35,"fusion":22,"denoise":60,"guardrails":15,"total":910},

  "device": "cuda:0"

}

```

**Mock mode** (`mock=true` or env `LUMENX_MOCK=1`): returns a realistic payload instantly using pre-generated assets, including SSE streaming with artificial delays. The frontend team must be able to build everything before the backend is finished.

---

## 6. Ablation and modes

`pipeline.py` accepts flags to disable each component: `no_tuner` (fixed default params), `no_fusion` (DL only), `no_classical`, `no_dl`, `no_gain_denoise` (uniform denoise), `no_recoverability`, `no_guardrails`, `no_color_correction`. Modes: `classical` (classical path only), `dl` (CIDNet only), `hybrid` (full).

---

## 7. Evaluation (`scripts/eval.py`, `scripts/ablation.py`)

- Datasets: **LOL-v2 Real (test)** (paired), **LOL-v1 test** (paired), **DICM, LIME, MEF, NPE, VV** (unpaired, used as "unseen"), plus 20–50 **own campus night photos** (unseen, real-world).
- Methods: Classical-only, Zero-DCE++, HVI-CIDNet, **LumenX-Hybrid**. (Optional: plain CLAHE and plain gamma as naive baselines.)
- Metrics: PSNR, SSIM, LPIPS (paired, report both normal and GT-mean-adjusted PSNR and state it clearly); NIQE, BRISQUE, CLIP-IQA, MUSIQ (unpaired); edge retention; noise Οƒ before/after; Ξ”E colour shift; highlight clip %; latency (mean, p95) on the RTX 2050.
- Outputs: `results/benchmark.csv`, `results/benchmark.md`, bar/radar charts (PNG), and per-image side-by-side grids for 6 showcase images.
- Ablation: run hybrid with each flag off on LOL-v2 Real + DICM β†’ `results/ablation.csv/.md` + chart showing how much each component contributes.
- Failure mining (`failure_mining.py`): for each method, save the 10 worst images by metric, auto-tag the reason (overexposure / colour shift / noise / halo / detail loss) via guardrail checks, and write `docs/FAILURE_ANALYSIS.md` with images and tags (leave a "Human notes" line under each).
- All eval scripts must be resumable and runnable overnight unattended.

---

## 8. Performance and engineering rules

- `utils.timer("stage")` context manager on every stage β†’ timings go into the response.
- Models load once at startup; `scripts/warmup.py` runs 2 dummy inferences.
- All thresholds/weights in `config.yaml`, with comments explaining each.
- Structured logging (stage, ms, VRAM used).
- Validate uploads (type, max 12 MP; downscale larger images and say so in the response).
- Graceful degradation: missing CIDNet weights β†’ Zero-DCE++; no GPU β†’ CPU with a warning field in the response.
- Type hints and docstrings everywhere; tests must pass: `pytest -q`.

---

## 9. "Wow" requirements (non-negotiable)

These are what make a non-technical viewer surprised. The backend must supply the data for each:

1. **Progressive reveal**: SSE stream lets the UI show the image "being understood" step by step.
2. **Honesty layer**: the recoverability overlay + sentence.
3. **Live relighting**: `L_gray.png` + base image let the browser relight in real time with a WebGL shader: `out = base Β· clamp(L^(Ξ³βˆ’1) Β· exposure)` (both sliders, 60 fps, no server call).
4. **Visible adaptivity**: tuner candidates with scores; the winner is flagged.
5. **Plain-English explanations**: headline, chips, decisions, story captions without jargon.
6. **Proof**: before/after metrics with direction (better/worse) and per-stage timing.
7. **Live webcam** via `/live` that visibly brightens a dark room in real time.

---

## 10. Basic frontend (replaceable by design)

Build a **minimal, clean, functional** single-page UI in plain HTML + vanilla JS (no build step), served by FastAPI at `/`. It exists to prove the API works and to give designers a reference.

Must include:
- Upload / drag-drop + gallery picker.
- SSE-driven progress list showing each stage caption and thumbnail as it arrives.
- Before/after comparison slider.
- Stage strip (click to view full size).
- Overlay toggles: recoverability, gain, fusion map, noise.
- Relight panel (WebGL canvas, Ξ³ + exposure sliders) in `relight.js`.
- Diagnosis chips, headline, decisions list.
- Tuner candidates grid (winner highlighted).
- Metrics table (before/after with arrows) and timing bar.
- Mode switch (classical / dl / hybrid) for side-by-side comparison.
- Live webcam tab.
- Benchmarks tab reading `/benchmarks`.

**Designed for redesign:**
- ALL styling in `theme.css` via CSS variables (`--bg`, `--fg`, `--accent`, `--radius`, `--font`…). No inline styles.
- JS split into small pure functions: `api.js`-style fetch helpers, `render*` functions that take data and return DOM. No framework lock-in.
- Every element has semantic `data-role="..."` attributes so designers can restyle or replace sections.
- The frontend must ONLY use the documented API: no hidden coupling.
- Add a `docs/API.md` section "Ideas for creative frontends" listing which fields can power: a flashlight/torch reveal (final vs input + recoverability), a 3D illumination terrain (heightmap), a story-mode scroll (story[]), and a candidate tournament animation (tuner.candidates).

---

## 11. Demo assets

- `gallery/`: 8–10 hard images (street at night, indoor with one lamp, backlit window, near-black scene, coloured neon cast, high-noise phone shot, a face in dim light, a campus photo). `metadata.json` with title + why it's hard.
- Pre-run all gallery images at startup (cache results) so the demo never waits.
- README includes a **3-minute demo script** in order: hardest image β†’ reveal β†’ recoverability β†’ relight β†’ candidates β†’ ablation chart β†’ lights off + webcam.

---

## 12. Build milestones (stop and verify at each)

| # | Hours | Deliverable | Verify |
|---|---|---|---|
| M1 | 0–2 | Repo skeleton, config, schemas, **mock API + mock SSE**, basic frontend wired to mock | Frontend works fully on mock data |
| M2 | 2–6 | `diagnose`, `classical`, `recoverability` + real `/enhance` (classical mode) | Real maps render in UI; < 100 ms |
| M3 | 4–8 | CIDNet (FP16, tiled, weight selection) + Zero-DCE++ + `/live` | No OOM at 1080p; live β‰₯ 20 FPS |
| M4 | 8–12 | `fusion`, gain-aware `denoise`, `guardrails`, hybrid mode | Visibly fewer halos/noise than DL-only |
| M5 | 10–14 | `tuner` + candidates + `report`/story + SSE real streaming | Full response matches schema; total ≀ 1.5 s |
| M6 | 14–16 | `eval.py`, `ablation.py`, `failure_mining.py` β†’ **run overnight** | CSV/MD/charts generated |
| M7 | 16–20 | Gallery caching, `/benchmarks`, frontend completeness, tests | `pytest` passes; all tabs work |
| M8 | 20–21 | **Feature freeze.** Bug fixes only. Record backup demo video. | Demo script runs cleanly 2Γ— |

---

## 13. Definition of done

- [ ] `uvicorn api.main:app` starts, warms up, and serves the UI at `/`.
- [ ] Any dark image β†’ final result + all maps + story + metrics in ≀ 1.5 s on RTX 2050.
- [ ] SSE streaming works; mock mode works with no GPU/weights.
- [ ] Recoverability overlay and live relight work.
- [ ] Classical vs DL vs Hybrid comparison available in UI and in `results/benchmark.md`.
- [ ] Ablation table shows each component's contribution.
- [ ] Failure analysis doc generated with real failure cases.
- [ ] Never crashes on: tiny image, huge image, already-bright image (should change it minimally and say so), grayscale image, PNG with alpha.
- [ ] README lets a stranger run it in 10 minutes.

---

## 14. Do NOT

- Do not add more deep models (no Retinexformer, diffusion, GANs): one strong model plus the fusion logic is the design.
- Do not hard-code thresholds in code.
- Do not let the frontend compute anything the backend should own (except the WebGL relight).
- Do not use jargon in user-facing captions.
- Do not silently swallow errors; every fallback is reported in the response.
- Do not start a new milestone while the current one is broken.