Document server-side confidence threshold

Co-authored-by: OpenAI Codex <codex@openai.com>

FEATURE.md

@@ -1,200 +1,227 @@
-# FEATURE: Confidence Threshold Control for Cutaway Generation
+# FEATURE: Confidence Threshold — Apply Only at Analysis & Generation
 
-Status: Implemented and synced to the private Space, June 14, 2026.
+Status: **Implemented and synced to the private Space, June 14, 2026.**
+Supersedes the prior implemented version of this file (commit `a0540e9`,
+"Add confidence threshold control").
 Author of spec: codebase review pass, June 14, 2026.
 
-## Summary
-
-Add a user-facing **confidence threshold slider** that controls how high the
-model's analysis `confidence` must be before the deterministic Three.js cutaway
-is rendered. Today this threshold exists but is a hardcoded `0.5` magic number,
-duplicated across Python and JavaScript, and not user-controllable.
-
-## Findings (current behavior)
-
-**Q: Does the generation of the Three.js parts have a confidence threshold to
-pass before sending back to the user?**
-
-Partially yes, but it is implicit, hardcoded, and not a true block:
-
-1. The vision model returns a `confidence` value (`0`–`1`) in the analysis JSON.
-   See `snap2sim/schema.py` (`ANALYSIS_SCHEMA`, `confidence` is
-   `number, minimum 0, maximum 1`), `snap2sim/prompts.py`, and
-   `snap2sim/model_io.py` (defaults: coerced `0.55`, generic fallback `0.45`).
-
-2. Confidence gates the **render mode**, not whether output is returned. The
-   threshold is a hardcoded `0.5` and lives in **two** places:
-   - **Python** `snap2sim/schema.py:218` `select_render_mode()`:
-     `low_confidence = _is_number(confidence) and float(confidence) < 0.5`.
-     This is called by `snap2sim/backend.py:47` `InferenceClient.generate_scene()`,
-     which returns `render_mode` in the `/generate_scene` payload.
-   - **JavaScript** `index.html:760` `chooseRenderMode()`:
-     `confidence < 0.5` / `confidence >= 0.5`. Used as a client-side fallback
-     only when the server payload omits `render_mode`.
-
-3. The threshold **downgrades** rather than blocks. The chain is:
-   `three` (3D cutaway) -> `annotate` (annotated source photo overlay) ->
-   `unavailable` (honest "cannot render" state). A low-confidence analysis is
-   still returned to the user, just not as 3D parts. See
-   `index.html:748` `renderScenePayload()` and `index.html:760`
-   `chooseRenderMode()`; mirrored in `schema.py:218` `select_render_mode()`.
-
-4. Confidence is currently only **displayed** (as a percentage) at
-   `index.html:726`; there is no control to change the cutoff.
-
-5. There is **no server-side hard gate**. `modal_app.py` / `app.py` validate and
-   coerce the payload but never reject on low confidence.
-
-**Conclusion:** an implicit confidence threshold (`0.5`) already governs whether
-Three.js parts render, but it is a duplicated magic number with no UI control.
-This feature exposes it as a slider and removes the magic-number duplication.
+> **Why this supersedes the prior spec.** The slider already exists, but it was
+> built as a *client-authoritative, live re-render* control: moving it instantly
+> recomputes the render mode from cached analysis in the browser, and the server
+> never sees the chosen threshold. The user has reversed all three of those
+> decisions. The threshold must now be **applied only when analysis and
+> generation run**, enforced **server-side**, and the slider must be usable
+> **before the first upload**.
+
+## Findings — prior behavior before this re-spec
+
+A confidence-threshold slider already existed and worked, but not the way the
+user wanted in this re-spec.
+
+1. **Slider UI exists.** `index.html:593-597` — `#confidenceThreshold`
+   (`type="range"`, `min=0 max=1 step=0.05 value=0.5`), with a live `#thresholdValue`
+   label. It carries the `disabled` attribute and is only enabled after a run.
+
+2. **Slider applies live, client-side, from cache.** The `input` handler at
+   `index.html:722-725` calls `updateConfidenceThreshold()` then
+   `scheduleThresholdRender()` (`index.html:1279-1286`), which debounces 90 ms and
+   re-runs `renderScenePayload(window.lastScenePayload)` against the **cached**
+   analysis. No network call — but it re-renders on *every drag*, decoupled from
+   any analysis/generation step.
+
+3. **Threshold is applied client-side only.** `chooseRenderMode()`
+   (`index.html:821-833`) compares `analysis.confidence` against the JS
+   `confidenceThreshold` var. It **ignores** the `render_mode` the server already
+   returned in the payload.
+
+4. **The server never receives the threshold.** `/generate_scene`
+   (`app.py:101-108` → `_generate_scene` at `app.py:116-117` →
+   `InferenceClient.generate_scene` at `backend.py:47-51`) calls
+   `select_render_mode(valid_analysis)` (`schema.py:219-236`), which always uses
+   the hardcoded `DEFAULT_CONFIDENCE_THRESHOLD = 0.5` (`schema.py:216`). The
+   server's `render_mode` is computed with `0.5` regardless of the slider.
+
+5. **Slider availability is gated on a completed run.** It is `disabled` in markup
+   (`index.html:595`), disabled in `resetScene()` (`index.html:769`), and
+   `setBusy()` (`index.html:1263`) keeps it disabled whenever
+   `!window.lastScenePayload`. So it cannot be set before the first analysis.
+
+**Conclusion:** the threshold is currently a *live, browser-only* control that
+never reaches the analysis/generation pipeline — the opposite of "only used upon
+analysis and generation." This spec changes it to a value that is captured and
+enforced **at generation time, server-side**, and that only takes effect on the
+next run.
 
 ## Product decisions (confirmed with user, June 14, 2026)
 
-1. **Gate behavior: keep the downgrade chain.** Below threshold, skip the 3D
-   Three.js render and fall back to annotated photo, then unavailable. The
-   slider only moves the cutoff; it does not introduce a new hard-block state.
-2. **Slider apply: live re-render from cached analysis.** Moving the slider after
-   an analysis returns must re-evaluate the render mode against the
-   already-returned analysis JSON and re-render immediately. No new
-   `/analyze_image` or model call — zero added latency or Modal cost.
-3. **Source of truth: client authoritative.** The browser owns the threshold.
-   The client recomputes the render mode from the raw `confidence` plus the
-   slider value, overriding any `render_mode` in the server payload. Rendering is
-   already deterministic browser-side, so this is the minimal change. Per
-   `SECURITY.md`, the threshold is a non-credential UX control, so client
-   authority is acceptable; it is a quality gate, not a security gate.
+1. **Apply timing: next run only.** Moving the slider does **nothing
+   immediately** — no live re-render of cached analysis. The new threshold is
+   captured and applied only the next time analysis + generation runs (next
+   upload / re-run). Remove the live client-side re-render entirely.
+2. **Enforcement: server pipeline.** The slider value is sent to
+   `/generate_scene`, and the server's `select_render_mode()` uses the user's
+   threshold instead of the hardcoded `0.5`. The server's returned `render_mode`
+   becomes the source of truth; the client trusts it.
+3. **Pre-run state: enabled up front.** The slider is usable before the first
+   upload so the chosen threshold is in effect for the very first
+   analysis/generation.
+
+Unchanged from prior spec (still true):
+- **Downgrade chain stays:** below threshold, skip the 3D Three.js render and
+  fall back to `annotate` (annotated source photo), then `unavailable`. The
+  slider only moves the cutoff; no new hard-block state.
+- **No re-running model inference on slider move** (analysis is the expensive
+  Modal GPU call; it is never re-triggered by the slider).
 
 ## Implementation plan (for Codex)
 
-All primary changes are in `index.html`. No backend signature change is required.
-
-### 1. Add the slider UI
-
-- Add a labeled `range` input to the viewport toolbar
-  (`index.html:550` `<div class="toolbar">`) or the readout panel near the
-  confidence metric (`index.html:577` `metric-row`). Recommended: toolbar, next
-  to `resetViewButton`, so it sits with the other live viewport controls.
-- Suggested markup:
-  ```html
-  <label class="threshold-control" for="confidenceThreshold">
-    Min confidence
-    <input id="confidenceThreshold" class="threshold-slider" type="range"
-           min="0" max="1" step="0.05" value="0.5"
-           aria-describedby="thresholdValue">
-    <span id="thresholdValue" aria-live="polite">50%</span>
-  </label>
-  ```
-- Default value `0.5` to preserve current behavior exactly.
-- Style consistent with existing `.tool-button` / toolbar aesthetic; keep it
-  keyboard-accessible (range inputs are by default) and screen-reader labeled,
-  matching the accessibility work already done in this repo (live status,
-  keyboard drop zone).
-
-### 2. Wire the threshold into render-mode selection
-
-- Introduce a single source for the current threshold, e.g.
-  `let confidenceThreshold = 0.5;`, updated from the slider's `input` event.
-- Update the slider value label (`#thresholdValue`) on `input`.
-- Replace the hardcoded `0.5` comparisons in `chooseRenderMode()`
-  (`index.html:760-772`) with `confidenceThreshold`:
-  - `confidence < confidenceThreshold` for the low-confidence branch.
-  - `confidence >= confidenceThreshold` for the `three` branch.
-- Make `renderScenePayload()` (`index.html:748`) **client authoritative**:
-  always compute the render mode via `chooseRenderMode(analysis)` using the
-  current threshold, instead of trusting `payload.render_mode`. Keep the
-  geometry/annotation capability checks (`hasUsableGeometry`, `hasAnnotations`)
-  so a high threshold never forces a 3D render that lacks geometry.
-
-### 3. Live re-render on slider change
-
-- Cache the last analysis (already stored at `index.html:722`
-  `window.lastAnalysis`) and the last scene payload.
-- On slider `change` (or debounced `input`), if an analysis is present and the
-  pipeline is not mid-request, call `renderScenePayload(lastScenePayload)` (or a
-  small `reevaluateRender()` helper that reads `window.lastAnalysis`) to rebuild
-  the scene from cached data. Do **not** call `/analyze_image` or
-  `/generate_scene` again — re-use the cached analysis JSON.
-- Guard against re-render while `setBusy(true)` is active to avoid racing an
-  in-flight request.
-- Reset/standby state should disable or ignore the slider re-render until an
-  analysis exists.
-
-### 4. Remove the magic-number duplication (recommended cleanup)
-
-Because decision #3 makes the client authoritative, the server's
-`select_render_mode()` `0.5` is now advisory only. Two acceptable options:
-
-- **Minimal:** leave `schema.py` `select_render_mode()` as-is (still returns a
-  reasonable default `render_mode`); the client overrides it. Add a code comment
-  noting the client is authoritative for the user-facing threshold.
-- **Cleaner (preferred if time allows):** extract the `0.5` default into a single
-  named constant, e.g. `DEFAULT_CONFIDENCE_THRESHOLD = 0.5` in `schema.py`, and a
-  matching JS constant in `index.html`, so the default lives in one obvious place
-  per layer. Do **not** add a backend signature change to pass the slider value
-  to the server (decision #3 keeps the threshold client-side).
+Changes span the browser (`index.html`) and the server
+(`app.py`, `snap2sim/backend.py`, `snap2sim/schema.py`).
+
+### 1. Server: accept and enforce a threshold at generation
+
+- `snap2sim/schema.py` — `select_render_mode(analysis, threshold=DEFAULT_CONFIDENCE_THRESHOLD)`:
+  - Add a `threshold` parameter, defaulting to `DEFAULT_CONFIDENCE_THRESHOLD`.
+  - Coerce/clamp: if `threshold` is not a number, fall back to the default; clamp
+    into `[0.0, 1.0]`.
+  - Replace the hardcoded `DEFAULT_CONFIDENCE_THRESHOLD` in the `low_confidence`
+    line (`schema.py:229`) with the (clamped) `threshold`.
+- `snap2sim/backend.py` — `InferenceClient.generate_scene(analysis, threshold=None)`
+  (`backend.py:47`):
+  - Accept an optional `threshold`; when `None`/invalid use
+    `DEFAULT_CONFIDENCE_THRESHOLD`.
+  - Pass it through to `select_render_mode(valid_analysis, threshold)`.
+- `app.py`:
+  - `/generate_scene` HTTP route (`app.py:106-108`): read
+    `payload.get("confidence_threshold")` and pass to `_generate_scene`.
+  - `@app.api(name="generate_scene")` (`app.py:101-103`): add an optional
+    `confidence_threshold` parameter (default keeps the existing `/run_pipeline`
+    Gradio API backward compatible).
+  - `_generate_scene(analysis, threshold)` (`app.py:116-117`): forward the
+    threshold to `InferenceClient(...).generate_scene(analysis, threshold)`.
+  - Validate at the boundary: coerce to `float`, clamp `[0, 1]`, default on
+    missing/invalid. Do not raise on a bad threshold — fall back to the default so
+    a malformed client value can't break generation.
+
+Result: `/generate_scene` returns a `render_mode` computed with the user's
+threshold. Existing callers that omit `confidence_threshold` still get the `0.5`
+default — backward compatible.
+
+### 2. Client: send the threshold at generation, stop live re-render
+
+- **Capture and send at run time.** In `runPipeline()` (`index.html:727-759`),
+  include the current threshold in the generate call:
+  `postJson("/generate_scene", { analysis, confidence_threshold: confidenceThreshold })`
+  (`index.html:749`). The value is read at the moment of the call, so later slider
+  drags don't affect the in-flight run.
+- **Trust the server's render mode.** Change `renderScenePayload()`
+  (`index.html:805-819`) / `chooseRenderMode()` (`index.html:821-833`) so the
+  decision uses `payload.render_mode` as the primary choice, with the existing
+  capability guards (`hasUsableGeometry`, `hasAnnotations`) only to *downgrade*
+  when data is missing — never to upgrade past what the server allowed. Remove the
+  client-side `confidence vs confidenceThreshold` comparison (the server now owns
+  that). `chooseRenderMode` should take the payload (or render_mode) rather than
+  recomputing from confidence.
+- **Remove the live re-render.** Delete `scheduleThresholdRender()`
+  (`index.html:1279-1286`) and the `thresholdRenderTimer` state
+  (`index.html:675`). The slider `input` handler (`index.html:722-725`) should
+  now only call `updateConfidenceThreshold()` — update the `confidenceThreshold`
+  var and the `#thresholdValue` label. No render, no network call.
+
+### 3. Client: enable the slider up front
+
+- Remove the `disabled` attribute from the markup (`index.html:595`).
+- In `resetScene()` (`index.html:761-776`), stop disabling the slider
+  (`index.html:769`) — it should remain available between runs.
+- In `setBusy()` (`index.html:1259-1264`), disable the slider **only while a
+  request is in flight** (`active`), not based on `window.lastScenePayload`
+  (`index.html:1263`). This lets the user set the threshold before the first
+  upload and adjust it between runs, while preventing edits mid-request.
+- Keep the default at `0.5` / `50%` so first-run behavior is unchanged when the
+  user never touches the slider.
+
+### 4. Optional: reflect "applies on next run" in the UI
+
+Because the slider no longer re-renders live, consider a subtle affordance so the
+change isn't silent — e.g. update the label to hint the value applies to the next
+analysis (tooltip or helper text). Low priority; keep it lightweight and
+accessible (don't regress the existing `aria-live` label).
 
 ## Out of scope / explicitly NOT doing
 
-- No new server endpoint or `/generate_scene` signature change (client
-  authoritative per decision #3).
-- No hard-block "confidence too low" state — the downgrade chain stays
-  (decision #1).
-- No re-running model inference when the slider moves (decision #2).
-- No model-authored HTML/JS/markup injection; rendering stays deterministic
-  Three.js from validated JSON (`SECURITY.md` Agent Guidance).
-
-## Implementation result
-
-- Added a toolbar confidence threshold slider in `index.html`, defaulting to
-  `0.5` / `50%`.
-- The browser now recomputes render mode from the cached analysis confidence and
-  current slider value, overriding advisory server `render_mode`.
-- Slider input re-renders from `window.lastScenePayload` only; it does not call
-  `/analyze_image` or `/generate_scene`.
-- Added `DEFAULT_CONFIDENCE_THRESHOLD = 0.5` in `snap2sim/schema.py` for the
-  server's advisory fallback.
-- Local verification passed: schema/parser check, FastAPI `TestClient` root /
-  `/analyze_image` / `/generate_scene`, browser slider downgrade/promote with
-  no network calls, annotated-photo fallback, keyboard slider operation, mobile
-  no horizontal overflow, and canvas pointer targeting after normal overlay
-  hiding.
-- GitHub Actions sync run `27515143658` deployed commit `a0540e9` to the
-  private Hugging Face Space. The Space reported the same SHA and remained
-  private. Authenticated deployed verification confirmed the updated root shell
-  contains the slider, a synthetic target image returned `Target Reticle` at
-  `0.7` confidence with 3 parts, and `/generate_scene` returned
-  `renderer: three`, `render_mode: three`, and no HTML field.
+- **No live re-render from cached analysis** (decision #1 — this is the behavior
+  being removed).
+- **No re-running model inference (`/analyze_image`) when the slider moves**
+  (analysis is the expensive Modal GPU call).
+- **No hard-block "confidence too low" state** — the `three -> annotate ->
+  unavailable` downgrade chain stays.
+- **No model-authored HTML/JS/markup injection.** Rendering stays deterministic
+  Three.js from validated JSON (`SECURITY.md` Agent Guidance). The threshold is a
+  non-credential UX/quality control; sending it to a same-origin endpoint is fine.
 
 ## Verification checklist
 
-- Slider defaults to `0.5`; with default value, behavior matches current
-  production exactly (regression check).
-- Raising the threshold above a returned analysis's confidence downgrades a
-  previously-3D render to annotated photo, then to unavailable, live, with no
-  network call (confirm via browser devtools Network tab — no new
-  `/analyze_image` or `/generate_scene` requests on slider move).
-- Lowering the threshold promotes an annotated/unavailable result back to the 3D
-  cutaway, provided usable geometry exists.
-- Slider value label updates and is announced (`aria-live`).
-- Slider is keyboard-operable and does not steal focus from / block the canvas
-  OrbitControls (watch for the pointer-events class of bug fixed in the
-  REVIEW2.md pass).
-- Mobile layout: slider does not introduce horizontal overflow; toolbar still
-  fits.
+- **Server, default:** `/generate_scene` without `confidence_threshold` returns
+  the same `render_mode` as today (regression — `0.5` default). Add/extend a unit
+  check: `select_render_mode(analysis, 0.9)` downgrades a `0.7`-confidence
+  geometry payload to `annotate`/`unavailable`; `select_render_mode(analysis, 0.1)`
+  keeps it `three`.
+- **Server, clamp/coerce:** out-of-range (`-1`, `5`) and non-numeric thresholds
+  fall back/clamp without raising; `/generate_scene` still returns a valid payload.
+- **Client, next-run-only:** moving the slider after a run does **not** trigger
+  any network request and does **not** change the current render (confirm via
+  devtools Network + visual). The new value only takes effect after the next
+  upload / re-run.
+- **Client, enforced server-side:** raising the threshold above the analysis's
+  confidence and re-running downgrades the 3D cutaway to annotated photo / then
+  unavailable; lowering it and re-running promotes back to 3D when geometry exists.
+- **Client, pre-run:** the slider is interactive before the first upload, disabled
+  only while a request is in flight, and re-enabled afterward.
+- **Accessibility/layout (regression):** slider keyboard-operable, `#thresholdValue`
+  announced, does not block canvas OrbitControls (the pointer-events bug fixed in
+  the REVIEW2.md pass), and no mobile horizontal overflow.
 - `INFERENCE_BACKEND=local` sample mode still renders the example analysis with
-  the slider present.
-- Run existing local checks: schema/parser checks and the FastAPI `TestClient`
-  pass for `/`, `/analyze_image`, `/generate_scene`.
+  the slider present and the threshold honored at generation.
+- Existing local checks pass: schema/parser checks and FastAPI `TestClient` for
+  `/`, `/analyze_image`, `/generate_scene`.
 
 ## Touch points (file/line reference)
 
-- `index.html:550` toolbar — add slider markup.
-- `index.html:577` metric-row / `index.html:726` confidence display — optional
-  co-location with confidence readout.
-- `index.html:748` `renderScenePayload()` — make client authoritative.
-- `index.html:760` `chooseRenderMode()` — replace `0.5` with slider value.
-- `index.html:602-622` element refs + `index.html:624` state vars — add slider
-  element ref and `confidenceThreshold` state.
-- `snap2sim/schema.py:218` `select_render_mode()` — optional constant extraction.
-- `snap2sim/backend.py:47` `generate_scene()` — no change required.
+- `snap2sim/schema.py:219-236` `select_render_mode()` — add `threshold` param,
+  clamp, use it instead of the hardcoded default at `schema.py:229`.
+- `snap2sim/backend.py:47-51` `generate_scene()` — accept + forward `threshold`.
+- `app.py:101-108` `generate_scene_api` / `generate_scene_http` — accept
+  `confidence_threshold`; `app.py:116-117` `_generate_scene` — forward it; clamp
+  at the boundary.
+- `index.html:595` slider markup — remove `disabled`.
+- `index.html:722-725` slider `input` handler — drop the live re-render call.
+- `index.html:749` `/generate_scene` call — send `confidence_threshold`.
+- `index.html:761-776` `resetScene()` / `index.html:1259-1264` `setBusy()` —
+  enable slider up front, disable only while busy.
+- `index.html:805-833` `renderScenePayload()` / `chooseRenderMode()` — trust
+  server `render_mode`; drop client-side threshold comparison.
+- `index.html:675` `thresholdRenderTimer` + `index.html:1279-1286`
+  `scheduleThresholdRender()` — remove.
+
+## Implementation result
+
+- Implemented in commit `a6f63e9` (`Enforce confidence threshold during generation`).
+- The slider is enabled before the first upload and disabled only while a request
+  is in flight.
+- The browser sends `confidence_threshold` only with `/generate_scene`; slider
+  movement does not call `/analyze_image`, call `/generate_scene`, or re-render
+  cached analysis.
+- `/generate_scene` clamps/coerces the threshold server-side and returns the
+  authoritative `render_mode`; the browser only downgrades when geometry or
+  annotation data is missing.
+- Local verification passed for default/high/low/malformed/clamped thresholds,
+  FastAPI `TestClient`, next-run-only browser behavior, high-threshold
+  downgrade, low-threshold promotion, keyboard slider operation, mobile
+  no-overflow layout, and canvas pointer targeting.
+- GitHub Actions sync run `27515950105` deployed commit `a6f63e9` to the private
+  Hugging Face Space. The Space reported SHA
+  `a6f63e9a0b76315bb223a09a71f4c027a29877fb` and remained private.
+- Authenticated private-Space verification passed: the root served the updated
+  shell, a synthetic image returned `optical sight` at `0.7` confidence with 3
+  parts, high threshold returned `photo` / `annotate`, low threshold returned
+  `three` / `three`, and no HTML field was present.

README.md

@@ -123,23 +123,25 @@ Runtime flow:
 
 1. Browser encodes the uploaded photo and posts it to `/analyze_image`.
 2. Backend returns the validated mechanism JSON.
-3. Browser posts the analysis to `/generate_scene` for a validated scene
-   descriptor.
+3. Browser posts the analysis plus the current confidence threshold to
+   `/generate_scene` for a validated scene descriptor.
 4. Browser renders deterministic Three.js primitives when geometry is usable,
    or overlays text-only callouts on the uploaded photo when the model only has
    image-space annotations.
 5. A browser-side confidence slider controls the minimum analysis confidence
-   needed for the 3D cutaway. Moving it re-renders from cached analysis data
-   and does not call the model again.
+   needed for the 3D cutaway. Moving it only updates the next-run value; it does
+   not re-render cached analysis or call the model.
 6. `/generate_scene` returns a validated
    `{ "renderer": "...", "render_mode": "...", "analysis": ... }` descriptor
-   instead of model-authored HTML.
+   instead of model-authored HTML. The returned `render_mode` is the source of
+   truth for the visible renderer, with the browser only downgrading when
+   required data is missing.
 
 The shell uses Chakra Petch and Fira Code from Bunny Fonts, an asymmetric
 63/37 viewport/readout split, a blueprint grid, amber/cyan instrument-panel
 colors, explicit Modal cold-start messaging, source-photo preview, and a
 play/pause control. The confidence slider defaults to 50%, matching the
-server's advisory fallback threshold, and the browser is authoritative for the
+server's fallback threshold, and the generation route is authoritative for the
 visible render mode.
 
 The browser no longer injects model-authored HTML into the DOM. The model's