kwabs22 Claude Opus 5.5 commited on
Commit
e6c37a7
Β·
1 Parent(s): b546563

Add the P0-0 context-budget item to the archived QA report

Browse files

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

feedback/vli-qa-feedback-2026-10-30.md CHANGED
@@ -26,6 +26,15 @@ Seed-0 baseline reproduced bit-identically from the first session: uid `4459b90d
26
 
27
  ## 2. Priority items
28
 
 
 
 
 
 
 
 
 
 
29
  ### P1-1 Β· Reroll at a fixed seed is a toggle, not a sampler
30
 
31
  - **Evidence:** two sessions days apart, reroll at seed 0 always swaps chosen with `alternatives[6]` (`4459b90d ↔ b86b2f4d`, same z-offset 0.0215100129063485 both times). Seed 1 always picks slot 3 (`7c3383a2…`). Same input β‡’ same pick, forever.
 
26
 
27
  ## 2. Priority items
28
 
29
+ ### P0-0 Β· Context budget: inline geometry can eat the whole chat (added 2026-10-30, after measuring)
30
+
31
+ - A scene page with the 4893-face cat inlines ~30,000 numbers (~15,000 vertex coords in `p` + ~14,700 triangle indices in `i`). Numbers tokenize badly (~2-4 tokens each), so **one tool result is on the order of tens of thousands of tokens β€” potentially most of a chat's context in a single call**. Measured live: three consecutive `translate_to_scene` calls in one session each returned the full 4893-face page. Even the repo's own 400-face budget (once deployed, see P2-5) leaves ~1,900 numbers β‰ˆ several thousand tokens per refine β€” heavy for a multi-turn loop.
32
+ - Fixes, in order of leverage:
33
+ 1. **Ship P2-5** (build parity) β€” cuts pages ~15Γ— with zero new code.
34
+ 2. **Make the page opt-in over MCP:** add `html: "skip" | "url" | "inline"` (default `"url"`) to `translate_to_scene`/`refine_scene`. `pipeline.outputs` already builds the page and `app._html_views` already writes `scene.html`, so returning the file/URL instead of the text is a small change. Iterating models then need only the ledger (~2k tokens) + PNG/GLB URLs β†’ a refine turn costs ~2-3k tokens instead of tens of thousands. URL expiry is tolerable mid-loop: the ledger fully determines the scene and any next refine re-renders. A model requests `"inline"` only when it wants to build on the page.
35
+ 3. **Shrink what's inlined when it is inlined:** round `p` to 2 decimals in `html._geometry` (`np.round(m.vertices, 2)` β€” mm precision is overkill at 0.3 m scale, ~25-30% off the vertex array), and consider a ~150-200-face working budget for iteration with 400 reserved for the final handoff. Primitives-only objects already cost ~12 numbers, which shows the floor.
36
+ - Principle for SPEC.md: **geometry is for the handoff, not the loop** β€” models iterate on ledger + picture; inline the page only when building on it.
37
+
38
  ### P1-1 Β· Reroll at a fixed seed is a toggle, not a sampler
39
 
40
  - **Evidence:** two sessions days apart, reroll at seed 0 always swaps chosen with `alternatives[6]` (`4459b90d ↔ b86b2f4d`, same z-offset 0.0215100129063485 both times). Seed 1 always picks slot 3 (`7c3383a2…`). Same input β‡’ same pick, forever.