docs/features/confidence-threshold.md

# Confidence Threshold: Apply Only at Analysis and Generation

Status: **Implemented and synced. Final public submission is under
`build-small-hackathon/Snap2Sim` as of June 15, 2026.**
Supersedes the prior implemented version of this file (commit `a0540e9`,
"Add confidence threshold control").
Author of spec: codebase review pass, June 14, 2026.

> **Why this supersedes the prior spec.** The slider already exists, but it was
> built as a *client-authoritative, live re-render* control: moving it instantly
> recomputes the render mode from cached analysis in the browser, and the server
> never sees the chosen threshold. The user has reversed all three of those
> decisions. The threshold must now be **applied only when analysis and
> generation run**, enforced **server-side**, and the slider must be usable
> **before the first upload**.

## Findings — prior behavior before this re-spec

A confidence-threshold slider already existed and worked, but not the way the
user wanted in this re-spec.

1. **Slider UI exists.** `index.html:593-597` — `#confidenceThreshold`
   (`type="range"`, `min=0 max=1 step=0.05 value=0.5`), with a live `#thresholdValue`
   label. It carries the `disabled` attribute and is only enabled after a run.

2. **Slider applies live, client-side, from cache.** The `input` handler at
   `index.html:722-725` calls `updateConfidenceThreshold()` then
   `scheduleThresholdRender()` (`index.html:1279-1286`), which debounces 90 ms and
   re-runs `renderScenePayload(window.lastScenePayload)` against the **cached**
   analysis. No network call — but it re-renders on *every drag*, decoupled from
   any analysis/generation step.

3. **Threshold is applied client-side only.** `chooseRenderMode()`
   (`index.html:821-833`) compares `analysis.confidence` against the JS
   `confidenceThreshold` var. It **ignores** the `render_mode` the server already
   returned in the payload.

4. **The server never receives the threshold.** `/generate_scene`
   (`app.py:101-108` → `_generate_scene` at `app.py:116-117` →
   `InferenceClient.generate_scene` at `backend.py:47-51`) calls
   `select_render_mode(valid_analysis)` (`schema.py:219-236`), which always uses
   the hardcoded `DEFAULT_CONFIDENCE_THRESHOLD = 0.5` (`schema.py:216`). The
   server's `render_mode` is computed with `0.5` regardless of the slider.

5. **Slider availability is gated on a completed run.** It is `disabled` in markup
   (`index.html:595`), disabled in `resetScene()` (`index.html:769`), and
   `setBusy()` (`index.html:1263`) keeps it disabled whenever
   `!window.lastScenePayload`. So it cannot be set before the first analysis.

**Conclusion:** the threshold is currently a *live, browser-only* control that
never reaches the analysis/generation pipeline — the opposite of "only used upon
analysis and generation." This spec changes it to a value that is captured and
enforced **at generation time, server-side**, and that only takes effect on the
next run.

## Product decisions (confirmed with user, June 14, 2026)

1. **Apply timing: next run only.** Moving the slider does **nothing
   immediately** — no live re-render of cached analysis. The new threshold is
   captured and applied only the next time analysis + generation runs (next
   upload / re-run). Remove the live client-side re-render entirely.
2. **Enforcement: server pipeline.** The slider value is sent to
   `/generate_scene`, and the server's `select_render_mode()` uses the user's
   threshold instead of the hardcoded `0.5`. The server's returned `render_mode`
   becomes the source of truth; the client trusts it.
3. **Pre-run state: enabled up front.** The slider is usable before the first
   upload so the chosen threshold is in effect for the very first
   analysis/generation.

Unchanged from prior spec (still true):
- **Downgrade chain stays:** below threshold, skip the 3D Three.js render and
  fall back to `annotate` (annotated source photo), then `unavailable`. The
  slider only moves the cutoff; no new hard-block state.
- **No re-running model inference on slider move** (analysis is the expensive
  Modal GPU call; it is never re-triggered by the slider).

## Implementation plan (for Codex)

Changes span the browser (`index.html`) and the server
(`app.py`, `snap2sim/backend.py`, `snap2sim/schema.py`).

### 1. Server: accept and enforce a threshold at generation

- `snap2sim/schema.py` — `select_render_mode(analysis, threshold=DEFAULT_CONFIDENCE_THRESHOLD)`:
  - Add a `threshold` parameter, defaulting to `DEFAULT_CONFIDENCE_THRESHOLD`.
  - Coerce/clamp: if `threshold` is not a number, fall back to the default; clamp
    into `[0.0, 1.0]`.
  - Replace the hardcoded `DEFAULT_CONFIDENCE_THRESHOLD` in the `low_confidence`
    line (`schema.py:229`) with the (clamped) `threshold`.
- `snap2sim/backend.py` — `InferenceClient.generate_scene(analysis, threshold=None)`
  (`backend.py:47`):
  - Accept an optional `threshold`; when `None`/invalid use
    `DEFAULT_CONFIDENCE_THRESHOLD`.
  - Pass it through to `select_render_mode(valid_analysis, threshold)`.
- `app.py`:
  - `/generate_scene` HTTP route (`app.py:106-108`): read
    `payload.get("confidence_threshold")` and pass to `_generate_scene`.
  - `@app.api(name="generate_scene")` (`app.py:101-103`): add an optional
    `confidence_threshold` parameter (default keeps the existing `/run_pipeline`
    Gradio API backward compatible).
  - `_generate_scene(analysis, threshold)` (`app.py:116-117`): forward the
    threshold to `InferenceClient(...).generate_scene(analysis, threshold)`.
  - Validate at the boundary: coerce to `float`, clamp `[0, 1]`, default on
    missing/invalid. Do not raise on a bad threshold — fall back to the default so
    a malformed client value can't break generation.

Result: `/generate_scene` returns a `render_mode` computed with the user's
threshold. Existing callers that omit `confidence_threshold` still get the `0.5`
default — backward compatible.

### 2. Client: send the threshold at generation, stop live re-render

- **Capture and send at run time.** In `runPipeline()` (`index.html:727-759`),
  include the current threshold in the generate call:
  `postJson("/generate_scene", { analysis, confidence_threshold: confidenceThreshold })`
  (`index.html:749`). The value is read at the moment of the call, so later slider
  drags don't affect the in-flight run.
- **Trust the server's render mode.** Change `renderScenePayload()`
  (`index.html:805-819`) / `chooseRenderMode()` (`index.html:821-833`) so the
  decision uses `payload.render_mode` as the primary choice, with the existing
  capability guards (`hasUsableGeometry`, `hasAnnotations`) only to *downgrade*
  when data is missing — never to upgrade past what the server allowed. Remove the
  client-side `confidence vs confidenceThreshold` comparison (the server now owns
  that). `chooseRenderMode` should take the payload (or render_mode) rather than
  recomputing from confidence.
- **Remove the live re-render.** Delete `scheduleThresholdRender()`
  (`index.html:1279-1286`) and the `thresholdRenderTimer` state
  (`index.html:675`). The slider `input` handler (`index.html:722-725`) should
  now only call `updateConfidenceThreshold()` — update the `confidenceThreshold`
  var and the `#thresholdValue` label. No render, no network call.

### 3. Client: enable the slider up front

- Remove the `disabled` attribute from the markup (`index.html:595`).
- In `resetScene()` (`index.html:761-776`), stop disabling the slider
  (`index.html:769`) — it should remain available between runs.
- In `setBusy()` (`index.html:1259-1264`), disable the slider **only while a
  request is in flight** (`active`), not based on `window.lastScenePayload`
  (`index.html:1263`). This lets the user set the threshold before the first
  upload and adjust it between runs, while preventing edits mid-request.
- Keep the default at `0.5` / `50%` so first-run behavior is unchanged when the
  user never touches the slider.

### 4. Optional: reflect "applies on next run" in the UI

Because the slider no longer re-renders live, consider a subtle affordance so the
change isn't silent — e.g. update the label to hint the value applies to the next
analysis (tooltip or helper text). Low priority; keep it lightweight and
accessible (don't regress the existing `aria-live` label).

## Out of scope / explicitly NOT doing

- **No live re-render from cached analysis** (decision #1 — this is the behavior
  being removed).
- **No re-running model inference (`/analyze_image`) when the slider moves**
  (analysis is the expensive Modal GPU call).
- **No hard-block "confidence too low" state** — the `three -> annotate ->
  unavailable` downgrade chain stays.
- **No model-authored HTML/JS/markup injection.** Rendering stays deterministic
  Three.js from validated JSON (`SECURITY.md` Agent Guidance). The threshold is a
  non-credential UX/quality control; sending it to a same-origin endpoint is fine.

## Verification checklist

- **Server, default:** `/generate_scene` without `confidence_threshold` returns
  the same `render_mode` as today (regression — `0.5` default). Add/extend a unit
  check: `select_render_mode(analysis, 0.9)` downgrades a `0.7`-confidence
  geometry payload to `annotate`/`unavailable`; `select_render_mode(analysis, 0.1)`
  keeps it `three`.
- **Server, clamp/coerce:** out-of-range (`-1`, `5`) and non-numeric thresholds
  fall back/clamp without raising; `/generate_scene` still returns a valid payload.
- **Client, next-run-only:** moving the slider after a run does **not** trigger
  any network request and does **not** change the current render (confirm via
  devtools Network + visual). The new value only takes effect after the next
  upload / re-run.
- **Client, enforced server-side:** raising the threshold above the analysis's
  confidence and re-running downgrades the 3D cutaway to annotated photo / then
  unavailable; lowering it and re-running promotes back to 3D when geometry exists.
- **Client, pre-run:** the slider is interactive before the first upload, disabled
  only while a request is in flight, and re-enabled afterward.
- **Accessibility/layout (regression):** slider keyboard-operable, `#thresholdValue`
  announced, does not block canvas OrbitControls (the pointer-events bug fixed in
  the `docs/reviews/interaction-and-fallback-review.md` pass), and no mobile
  horizontal overflow.
- `INFERENCE_BACKEND=local` sample mode still renders the example analysis with
  the slider present and the threshold honored at generation.
- Existing local checks pass: schema/parser checks and FastAPI `TestClient` for
  `/`, `/analyze_image`, `/generate_scene`.

## Touch points (file/line reference)

- `snap2sim/schema.py:219-236` `select_render_mode()` — add `threshold` param,
  clamp, use it instead of the hardcoded default at `schema.py:229`.
- `snap2sim/backend.py:47-51` `generate_scene()` — accept + forward `threshold`.
- `app.py:101-108` `generate_scene_api` / `generate_scene_http` — accept
  `confidence_threshold`; `app.py:116-117` `_generate_scene` — forward it; clamp
  at the boundary.
- `index.html:595` slider markup — remove `disabled`.
- `index.html:722-725` slider `input` handler — drop the live re-render call.
- `index.html:749` `/generate_scene` call — send `confidence_threshold`.
- `index.html:761-776` `resetScene()` / `index.html:1259-1264` `setBusy()` —
  enable slider up front, disable only while busy.
- `index.html:805-833` `renderScenePayload()` / `chooseRenderMode()` — trust
  server `render_mode`; drop client-side threshold comparison.
- `index.html:675` `thresholdRenderTimer` + `index.html:1279-1286`
  `scheduleThresholdRender()` — remove.

## Implementation result

- Implemented in commit `a6f63e9` (`Enforce confidence threshold during generation`).
- The slider is enabled before the first upload and disabled only while a request
  is in flight.
- The browser sends `confidence_threshold` only with `/generate_scene`; slider
  movement does not call `/analyze_image`, call `/generate_scene`, or re-render
  cached analysis.
- `/generate_scene` clamps/coerces the threshold server-side and returns the
  authoritative `render_mode`; the browser only downgrades when geometry or
  annotation data is missing.
- Local verification passed for default/high/low/malformed/clamped thresholds,
  FastAPI `TestClient`, next-run-only browser behavior, high-threshold
  downgrade, low-threshold promotion, keyboard slider operation, mobile
  no-overflow layout, and canvas pointer targeting.
- GitHub Actions sync run `27515950105` deployed commit `a6f63e9` to the then-private
  Hugging Face Space. The Space reported SHA
  `a6f63e9a0b76315bb223a09a71f4c027a29877fb`.
- Authenticated Space verification passed: the root served the updated
  shell, a synthetic image returned `optical sight` at `0.7` confidence with 3
  parts, high threshold returned `photo` / `annotate`, low threshold returned
  `three` / `three`, and no HTML field was present.