arkai2025 Claude Opus 4.8 (1M context) commited on
Commit
88bd014
Β·
1 Parent(s): a5a7e62

docs(wiki): describe the report's two views (Gallery + Matrix)

Browse files

The Reusable tooling note in model-selection-spike predated the Matrix view and
only described the live-context Gallery strategy. Document both views and the
two rendering strategies (Gallery: live WebGL contexts + reconcile; Matrix:
static offscreen-renderer snapshots), and bump the sweep to 20 models.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Files changed (2) hide show
  1. wiki/log.md +10 -0
  2. wiki/model-selection-spike.md +15 -10
wiki/log.md CHANGED
@@ -2,6 +2,16 @@
2
 
3
  ## 2026-06-15
4
 
 
 
 
 
 
 
 
 
 
 
5
  ### model-selection-spike: point at the tracked eval tool
6
 
7
  - Added a **Reusable tooling** section: the spike's 11 throwaway
 
2
 
3
  ## 2026-06-15
4
 
5
+ ### model-selection-spike: two report views (Gallery + Matrix)
6
+
7
+ - Updated the Reusable tooling section: the report now has a **View** dropdown β€”
8
+ Gallery (live rotatable contexts + Theme filter) and Matrix (model Γ— theme
9
+ comparison table). Corrected the rendering-strategy note: Gallery uses live
10
+ WebGL contexts (capped + distance-reconciled); Matrix uses static snapshots
11
+ from one shared offscreen renderer (the app's thumbnail strategy), so all 152
12
+ cells render without hitting the context cap. Sweep is now 20 models
13
+ (MiniCPM4.1-8B added).
14
+
15
  ### model-selection-spike: point at the tracked eval tool
16
 
17
  - Added a **Reusable tooling** section: the spike's 11 throwaway
wiki/model-selection-spike.md CHANGED
@@ -164,9 +164,11 @@ model test does not start from zero:
164
  helpers, no model loading), `models.py` (the editable list β€” add a `(label,
165
  repo)` tuple), `eval.py` (CLI: one model at a time, download β†’ grammar-enforced
166
  test β†’ optional `--delete-after` β†’ merge; `--fewshot` for the exemplar A/B),
167
- and `report.html` + committed `report_data.js` (the 19-model Γ— 8-theme sweep,
168
- so the rotatable 3D report opens populated; the **Theme** dropdown compares
169
- every model on one creature).
 
 
170
  - **`/voxel-model-eval` skill** β€” the run procedure plus the hard rules from this
171
  page (grammar always on; ≀32B/MoE total; latency relative-only; valid-JSON β‰ 
172
  visual quality; never judge from thumbnails).
@@ -174,13 +176,16 @@ model test does not start from zero:
174
  (`PYTHON_BIN=/Users/chenzhikai/Documents/Project/build-a-buddy-hf/.venv/bin/python`);
175
  no deps are added to this repo. Unit tests: `tests/test_voxel_eval_{core,cli}.py`.
176
 
177
- Note: the report viewer renders a **grid of many live WebGL contexts** capped at
178
- 12 and reconciled by distance-to-viewport-center on scroll β€” a different strategy
179
- from the app's renderer (one shared offscreen thumbnail renderer + a single live
180
- stage, see [voxel-rendering](voxel-rendering.md)). The naive "evict the furthest
181
- live context and rely on the IntersectionObserver to re-mount it" blanks
182
- still-on-screen cards permanently (IO does not re-fire for already-intersecting
183
- elements); the viewer reconciles the full visible set instead.
 
 
 
184
 
185
  ## Methodology lessons
186
 
 
164
  helpers, no model loading), `models.py` (the editable list β€” add a `(label,
165
  repo)` tuple), `eval.py` (CLI: one model at a time, download β†’ grammar-enforced
166
  test β†’ optional `--delete-after` β†’ merge; `--fewshot` for the exemplar A/B),
167
+ and `report.html` + committed `report_data.js` (the 20-model Γ— 8-theme sweep,
168
+ so the 3D report opens populated). The report has two views, switched by a
169
+ **View** dropdown: **Gallery** (big rotatable cards + a **Theme** filter to
170
+ compare every model on one creature) and **Matrix** (a model Γ— theme
171
+ comparison table).
172
  - **`/voxel-model-eval` skill** β€” the run procedure plus the hard rules from this
173
  page (grammar always on; ≀32B/MoE total; latency relative-only; valid-JSON β‰ 
174
  visual quality; never judge from thumbnails).
 
176
  (`PYTHON_BIN=/Users/chenzhikai/Documents/Project/build-a-buddy-hf/.venv/bin/python`);
177
  no deps are added to this repo. Unit tests: `tests/test_voxel_eval_{core,cli}.py`.
178
 
179
+ Note: the two views use different rendering strategies, both bounded by the
180
+ browser's ~16 live-WebGL-context cap. **Gallery** mounts a grid of **live
181
+ contexts** (rotatable), capped at 12 and reconciled by distance-to-viewport-center
182
+ on scroll. **Matrix** renders each cell as a **static snapshot** from one shared
183
+ offscreen renderer (the app's thumbnail strategy, see
184
+ [voxel-rendering](voxel-rendering.md)) β€” flat context count, so all 152 cells show
185
+ at once. Gotcha that drove the Gallery design: the naive "evict the furthest live
186
+ context and rely on the IntersectionObserver to re-mount it" blanks still-on-screen
187
+ cards permanently (IO does not re-fire for already-intersecting elements), so the
188
+ Gallery reconciles the full visible set instead.
189
 
190
  ## Methodology lessons
191