# PRD: Accelerated ISP + Synthetic Edge-Case Generation

**Feature Name:** Accelerated ISP Preprocessing & Synthetic Edge-Case Generation
**Package:** `sensorflow/vitis` (`isp.py`, `augment.py`) — API under
`/api/vitis/isp/*` and `/api/vitis/augment/*`
**Status:** Implemented on the emulated backend. No FPGA hardware is attached;
all FPGA throughput figures are analytically modeled and labeled
`modeled_not_measured: true`.

## Value Proposition

Two problems, one accelerator. First, ISP pipelines (bad-pixel correction →
demosaic → HDR tone-map → gain → denoise → resize) are the earliest place a
hardware port silently degrades a perception stack — this feature runs the
composable ISP on the reference and Vitis-constrained paths simultaneously
and reports PSNR/SSIM *per stage*, so the exact stage where fixed-point or
streaming constraints bite is visible, alongside a measured-CPU vs
modeled-FPGA throughput comparison that quantifies the acceleration case.
Second, the same accelerated op library mints rare-edge-case evaluation data
at scale: parameterized sensor-noise/low-light/HDR/distortion/blur/glare
augmentations generate stress variants of evaluation frames with complete
lineage, guarded as evaluation-only so synthetic stress data can never leak
into training.

## Architecture & Workflow

```
   +-----------------------------------------------------------------+
   |                     Cloud evaluation platform                    |
   |   evaluation/megaeval record store        raremine scene bank    |
   |   (eval-set supplements, lineage,         (candidate flow for    |
   |    leakage guard destinations)             costume-ped variants) |
   +--------------------^--------------------------------^-----------+
                        | eval-only variant records      | candidates
                        |  (full lineage, protected)     | (try/except)
   +--------------------+--------------------------------+-----------+
   |               sensorflow/vitis: ISP + augment                    |
   |                                                                  |
   |  RAW Bayer -> bad_pixel -> demosaic -> tone_map -> gain          |
   |            -> denoise -> resize        (per-stage PSNR/SSIM)     |
   |                                                                  |
   |  +----------------+        +-------------------------------+     |
   |  | reference CPU  |  vs    | vitis_emulated (ap_fixed<W,I>,|     |
   |  | float32        |        | XFCVDEPTH, LUT div/sqrt,      |     |
   |  +----------------+        | modeled pixels/cycle @ MHz)   |     |
   |                            +---------------+---------------+     |
   |                                            |  << EMULATION      |
   |                            +---------------+---------------+     |
   |                            | (future) vitis_hw: xf::cv ISP |     |
   |                            | kernels on real PL, measured  |     |
   |                            | throughput via XRT profiling  |     |
   |                            +-------------------------------+     |
   +------------------------------------------------------------------+
                        | runs/vitis/{isp,augment}/*.json
                        v
                 dashboard (stage chain, throughput, gallery)
```

## Key Metrics Tracked

From `POST /api/vitis/isp/run`:

- `stage_report[].psnr_db`, `stage_report[].ssim` — emulated vs reference,
  per stage
- `stage_report[].measured_cpu_ms`, `stage_report[].modeled_fpga_ms`,
  `stage_report[].modeled_speedup_x`, `stage_report[].modeled_placement`
  (PL/AIE)
- `throughput.measured_cpu_fps`, `throughput.modeled_fpga_fps_pipelined`
  (dataflow overlap: 1 / max stage latency),
  `throughput.modeled_speedup_x_serial` — all modeled fields flagged
  `modeled_not_measured`
- `summary.min_stage_psnr_db` — the weakest stage at the configured bit-width

From `POST /api/vitis/augment/generate` and `GET /api/vitis/augment/variants`:

- per-variant `lineage` (recipe with resolved params, seed, source frame,
  backend config), `evaluation_only: true`, `training_eligible: false`,
  `recommended_dataset_destination: REGRESSION_EVALUATION_SET`
- `raremine_hook.routed_candidates` (or the documented fallback note when
  `sensorflow.raremine` is not importable)

## Rollout

1. **Emulated (now):** tune stage order, bit-widths, and augmentation
   recipes against per-stage PSNR; mint evaluation-only stress sets.
2. **HIL bench:** synthesize the ISP as an xf::cv dataflow pipeline, replace
   modeled latencies with XRT-profiled measurements, and validate that
   per-stage PSNR on silicon matches the emulator within its documented
   error bounds.
3. **Fleet:** run augmentation generation as a batch service against real
   fleet evaluation frames; generated variants flow into the protected
   evaluation destinations with the same lineage records.

## Risks

- **Modeled throughput is not a benchmark.** pixels/cycle × clock is an
  upper bound that ignores memory bandwidth, AXI stalls, and resource
  contention. Every figure carries `modeled_not_measured`; stage-2 exists
  precisely to replace them.
- **Leakage.** Synthetic stress variants in training would corrupt the
  regression signal the variants exist to protect. Records default to
  `training_eligible: false` and a protected destination; consumers must
  respect the flag (enforced convention, mirrored from raremine's
  `PROTECTED_EVAL_DESTINATIONS`).
- **Schematic realism.** Variants derive from schematic BEV renders, not
  camera frames. The recipes and lineage plumbing are production-shaped; the
  pixels are placeholders until real frames are wired in.
- **In-progress raremine API.** The candidate-routing hook degrades to a
  no-op with a note if the raremine package changes shape mid-flight.