Spaces:
Running on Zero
Running on Zero
kwabs22 Claude Opus 5.5 commited on
Commit Β·
e6c37a7
1
Parent(s): b546563
Add the P0-0 context-budget item to the archived QA report
Browse filesCo-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feedback/vli-qa-feedback-2026-10-30.md
CHANGED
|
@@ -26,6 +26,15 @@ Seed-0 baseline reproduced bit-identically from the first session: uid `4459b90d
|
|
| 26 |
|
| 27 |
## 2. Priority items
|
| 28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
### P1-1 Β· Reroll at a fixed seed is a toggle, not a sampler
|
| 30 |
|
| 31 |
- **Evidence:** two sessions days apart, reroll at seed 0 always swaps chosen with `alternatives[6]` (`4459b90d β b86b2f4d`, same z-offset 0.0215100129063485 both times). Seed 1 always picks slot 3 (`7c3383a2β¦`). Same input β same pick, forever.
|
|
|
|
| 26 |
|
| 27 |
## 2. Priority items
|
| 28 |
|
| 29 |
+
### P0-0 Β· Context budget: inline geometry can eat the whole chat (added 2026-10-30, after measuring)
|
| 30 |
+
|
| 31 |
+
- A scene page with the 4893-face cat inlines ~30,000 numbers (~15,000 vertex coords in `p` + ~14,700 triangle indices in `i`). Numbers tokenize badly (~2-4 tokens each), so **one tool result is on the order of tens of thousands of tokens β potentially most of a chat's context in a single call**. Measured live: three consecutive `translate_to_scene` calls in one session each returned the full 4893-face page. Even the repo's own 400-face budget (once deployed, see P2-5) leaves ~1,900 numbers β several thousand tokens per refine β heavy for a multi-turn loop.
|
| 32 |
+
- Fixes, in order of leverage:
|
| 33 |
+
1. **Ship P2-5** (build parity) β cuts pages ~15Γ with zero new code.
|
| 34 |
+
2. **Make the page opt-in over MCP:** add `html: "skip" | "url" | "inline"` (default `"url"`) to `translate_to_scene`/`refine_scene`. `pipeline.outputs` already builds the page and `app._html_views` already writes `scene.html`, so returning the file/URL instead of the text is a small change. Iterating models then need only the ledger (~2k tokens) + PNG/GLB URLs β a refine turn costs ~2-3k tokens instead of tens of thousands. URL expiry is tolerable mid-loop: the ledger fully determines the scene and any next refine re-renders. A model requests `"inline"` only when it wants to build on the page.
|
| 35 |
+
3. **Shrink what's inlined when it is inlined:** round `p` to 2 decimals in `html._geometry` (`np.round(m.vertices, 2)` β mm precision is overkill at 0.3 m scale, ~25-30% off the vertex array), and consider a ~150-200-face working budget for iteration with 400 reserved for the final handoff. Primitives-only objects already cost ~12 numbers, which shows the floor.
|
| 36 |
+
- Principle for SPEC.md: **geometry is for the handoff, not the loop** β models iterate on ledger + picture; inline the page only when building on it.
|
| 37 |
+
|
| 38 |
### P1-1 Β· Reroll at a fixed seed is a toggle, not a sampler
|
| 39 |
|
| 40 |
- **Evidence:** two sessions days apart, reroll at seed 0 always swaps chosen with `alternatives[6]` (`4459b90d β b86b2f4d`, same z-offset 0.0215100129063485 both times). Seed 1 always picks slot 3 (`7c3383a2β¦`). Same input β same pick, forever.
|