leaderboard: fixed 4-sample gallery, auto-size iframe, center modal
Michael Rabinovich+1: Cursor
08eae45
Replace the gallery's interactive fixture picker with a fixed
comparison sheet: two generation + two editing samples, one Medium and
one Hard per task. Difficulty is chosen once from the Claude Opus 4.8
baseline's per-sample CAD scores (median = Medium, 20th percentile =
Hard) and frozen into FIXED_FIXTURES; the GUI no longer recomputes it
as the leaderboard evolves, trading a moving ground truth for a simple,
stable interface. Adds tools/pick_gallery_fixtures.py to regenerate the
constant.
Size the gallery iframe to its content (no more 90vh fixed box / nested
scrollbar) and pin the compare modal to the visible part of the iframe
so it opens centered in view instead of far below the fold.
Co-authored-by: Cursor <cursoragent@cursor.com>
leaderboard: backfill tool to grid-ify already-published reports
Michael Rabinovich+1: Cursor
ba3eefb
Rewrites the summary view of published reports/<id>.html in the submissions
dataset from the old flat table to the thumbnail grid, in place and without
re-evaluating. Parses each report's existing rows (sample/status/CAD) + the
render-bucket base already embedded in the file, classifies editing by the
starting-shape renders (so invalid edits group correctly), and rebuilds the
grid via single_run's shared builders (byte-identical to a fresh report),
pointing editing cards at edit_diff.png and generation cards at output iso.
Injects the shared grid CSS/JS; detail cards, header and download button are
left untouched. Supports --files and --dataset (--dry-run); idempotent.
Co-authored-by: Cursor <cursoragent@cursor.com>
leaderboard: backfill tool for edit_diff.png stills in render bucket
Michael Rabinovich+1: Cursor
c7f83a5
One-time tool to close the gap for submissions evaluated before the eval
pipeline started writing edit_diff.png: lists the render bucket, finds every
<fixture>/edit_diff.webp lacking a sibling edit_diff.png, extracts frame 0
via cadgenbench.common.imaging.first_frame_png (the exact frame the pipeline
now saves), and uploads the still to the same bucket prefix. Idempotent
(skips existing stills); --dry-run lists without a token.
Co-authored-by: Cursor <cursoragent@cursor.com>
leaderboard: bump cadgenbench to 1010043 + legend backfill tool
Michael Rabinovich+1: Cursor
1a24d4a
Pin to the report generator with color-chip interface/edit-diff legends, and
add tools/backfill_report_legends.py to retro-patch the corrected legends into
already-published reports/<id>.html in place (no re-eval).
Co-authored-by: Cursor <cursoragent@cursor.com>
leaderboard: download-zip button in merge-path report + backfill tool
Michael Rabinovich+1: Cursor
c48c18f
Pass the submission-zip URL to generate_html in the merge path so reassembled
reports show the "Download submission ZIP" button, and add a one-off
tools/backfill_report_download_button.py that injects the button into
already-published reports/<id>.html in place (no re-eval). Bump CADGENBENCH_SHA
to 0c7690e.
Co-authored-by: Cursor <cursoragent@cursor.com>
leaderboard: add Tasks tab to browse benchmark fixtures
Michael Rabinovich+1: Cursor
f4924d6
A read-only task browser that mirrors the per-submission report's
summary-table -> detail-card navigation (j/k, Esc, Prev/Next,
deep-linkable) but shows only the prompt + input (drawing for
generation, starting-shape renders for editing) with no scores or
ground truth. Fixtures come from each <fixture>/description.yaml in the
inputs dataset; input images are proxied through the Space's read token
via /task-input and lazy-loaded.
Co-authored-by: Cursor <cursoragent@cursor.com>
Serve rotating WebP turntables + GT generator
Michael Rabinovich+1: Cursor
c1cb5e4
Point the gallery and proxy routes at rotating.webp (submissions and
GT), publish the full per-fixture renders folder, and add a reusable
one-time GT turntable generator that renders from the cached GT mesh.
Co-authored-by: Cursor <cursoragent@cursor.com>
