Commit History
-
leaderboard: mobile uses the same aligned table as desktop
Michael Rabinovich +1: CursorRevert the mobile card layout (it broke the GT-vs-output comparison by putting GT in its own card). On phones now show the same table: sticky sample-column headers + pinned ground-truth row, with each model's four renders aligned in the same columns directly under the GT. To fit phone width the per-model name+score becomes a slim bar above that model's render row (no room for a left identity column plus four renders). The box still caps + scrolls internally so the GT row stays pinned. Co-authored-by: Cursor <cursoragent@cursor.com>0c0ddd8 -
leaderboard: dedicated mobile card layout for the gallery
Michael Rabinovich +1: CursorA horizontal-scroll table is unusable on phones (one render at a time). On narrow screens hide the table and render one card per model instead (plus a ground-truth card), each showing the four renders in a labelled 2x2 grid, so the whole comparison reads top-to-bottom with a single vertical scroll. Co-authored-by: Cursor <cursoragent@cursor.com>a636039 -
leaderboard: fix mobile gallery (GT row wrap + double scroll)
Michael Rabinovich +1: Cursor- The GT row wrapped on mobile because a base rule (.grow.gt-row .rank) re-showed the rank cell only on that row (7 items in a 6-col grid); force-hide rank on mobile so the row lays out correctly. - On phones, drop the vertical cap so the page scrolls once vertically and fixtures are reached by horizontal swipe (frozen Submission column), instead of a tiny capped box plus a confusing outer scroll. Co-authored-by: Cursor <cursoragent@cursor.com>00f76c2 -
leaderboard: reclaim Gradio chrome + responsive mobile gallery
Michael Rabinovich +1: Cursor- Hide the Gradio footer and tighten outer padding (the unused space between Refresh and the footer) and grow the gallery box to use it, so more rows fit in one viewport. - Mobile: the 4-sample matrix no longer clips. On narrow screens the table scrolls horizontally (swipe), the rank column is dropped, the Submission column is frozen to the left, and the score breakdown is hidden -- so the renders are actually visible and you can tell which model each belongs to, with the header + GT row still pinned. Co-authored-by: Cursor <cursoragent@cursor.com>2ec97dd -
leaderboard: squeeze header/caption, grow gallery to show more rows
Michael Rabinovich +1: Cursor- Drop the gallery caption and collapse the Gradio title to a single compact line, and cut the gallery body padding (removes the dead gap between the last row and the Refresh button). - With that freed vertical space, lower the box chrome reserve and raise the cap so the gallery shows as many rows as fit in one viewport while staying a single scroll (harness-verified across screen sizes). Co-authored-by: Cursor <cursoragent@cursor.com>882faa0 -
leaderboard: bigger gallery box + scroll affordance
Michael Rabinovich +1: Cursor- Raise the box height cap (760->900) so larger screens use their vertical space and show more rows, keeping the chrome reserve so it still fits in one viewport (single scroll). - Add a clear scroll affordance: a persistent (styled) scrollbar plus a 'scroll for more models' fade cue at the bottom that hides once the list is scrolled to the end, and a caption note that the GT row stays pinned. Co-authored-by: Cursor <cursoragent@cursor.com>0eccd77 -
leaderboard: widen gallery chrome reserve to kill residual outer scroll
Michael Rabinovich +1: CursorLive verification showed a constant ~56px outer-page overflow on smaller screens (real HF chrome is a bit taller than the harness estimate). Bump CHROME_RESERVE 560->620 and lower the box floor so the whole gallery fits the viewport across screen sizes (single scroll). Co-authored-by: Cursor <cursoragent@cursor.com>5fb80c6 -
leaderboard: gallery scroll box sized off screen.availHeight
Michael Rabinovich +1: CursorVerified the real cause with a headless harness reproducing HF's nested iframes: HF auto-resizes the Space iframe to full content height, so inside the gallery window.innerHeight / parent.innerHeight / vh are all inflated to the content height (live: 1642 vs a 900 viewport) and only the cross-origin top window scrolls -- which is why no inner cap or sticky-against-the-iframe ever worked and the GT row scrolled away. screen.availHeight is the one viewport-ish measure that survives the nesting, so size the gallery's own scroll box from it (availHeight minus a chrome reserve, clamped). The column header + GT row are sticky inside that box, so they stay locked while the submission rows scroll, and the box is the single scroller (harness confirms no outer-page overflow across laptop/desktop/large screens). Co-authored-by: Cursor <cursoragent@cursor.com>a435f45 -
metrics: tighten Metrics page copy
Michael Rabinovich +1: CursorFix the "Editing tasks" anchor landing short (bottom spacer), drop the per-section "Full derivation" links in favor of a single docs/metrics.md pointer at the end, trim wording (Boolean kernel, "(pieces)", "(e.g. through-holes)", "not the mean", the report-overlay note), and remove em-dashes. Co-authored-by: Cursor <cursoragent@cursor.com>2a0d6fb -
deploy: bump cadgenbench to b76a55a + ship metrics assets
Michael Rabinovich +1: CursorPins the Space to cadgenbench b76a55a (report title with submission name, click-to-zoom images, metric pill deep-links) and copies the new assets/ dir into the image so the Metrics page's mating-group WebP is served by the /metrics-assets route. Co-authored-by: Cursor <cursoragent@cursor.com>0c1ec25 -
leaderboard: add Metrics tab/page + report deep-links
Michael Rabinovich +1: CursorNew self-contained Metrics explainer (metrics_page.py) served at /metrics and embedded in a Metrics tab: validity gate, the three axes, editing renormalization, with formulas and the interface mating-group illustration (vendored under assets/metrics/ via LFS, served by the /metrics-assets route). Submit's merge path passes the submission name and the /metrics base URL to the report generator so hosted reports title themselves and their metric pills deep-link to the explainer. Co-authored-by: Cursor <cursoragent@cursor.com>28e0081 -
fix(oauth): use real HF OAuth on this Docker Space instead of mock login
Michael Rabinovich +1: CursorGradio's get_space() only returns truthy when SYSTEM=="spaces", which HF sets on Gradio-SDK Spaces but not on sdk: docker Spaces like this one. Without it, mount_gradio_app wired up Gradio's MOCK OAuth routes: they never contact hf.co and log every visitor in as the container token's owner (our HF_TOKEN account). Because that account is in CADGENBENCH_ADMINS, this leaked the "Logout (michaelr27)" identity to all visitors and granted admin to anyone who completed the (mock) login. Set SYSTEM=spaces when running on a Space (SPACE_ID present) so the real hf_oauth: true flow runs; left unset locally so the local mock login still works for dev. Co-authored-by: Cursor <cursoragent@cursor.com>f17ac64 -
leaderboard: make the gallery the single scroller
Michael Rabinovich +1: CursorSizing the iframe to a fixed 80vh left two scrollbars (the iframe body and the outer Gradio page), which was confusing. Now fitIframe measures the available height from the iframe's own top down to the bottom of the viewport (via the same-origin parent) and sizes the iframe to min(content, available), reserving room for the Refresh button. Short content shrinks the iframe (nothing scrolls); tall content fills the viewport so only the iframe body scrolls and the sticky header + GT row stay locked. The outer page no longer needs its own scrollbar. Co-authored-by: Cursor <cursoragent@cursor.com>5eaf6b8 -
leaderboard: lock GT row via in-iframe scroll capped by max-height
Michael Rabinovich +1: CursorThe previous attempt capped the gallery scroll box from JS using window.parent.innerHeight, but inside HF's nested iframes that is not the browser viewport, so the cap came out huge: the box never scrolled, the iframe grew to full height, and the page scrolled the GT row away. Instead, scroll inside the iframe itself (the column header + GT row are sticky against the iframe viewport, which reliably works) and cap the iframe with `max-height: 80vh` in app.py, where vh is the Gradio doc viewport. fitIframe still shrinks the iframe to content for a compact page when there are few rows; once content exceeds the cap the iframe body scrolls and the GT row stays pinned. Drops the unreliable JS cap and the parent-aware modal positioning (back to plain fixed centering). Co-authored-by: Cursor <cursoragent@cursor.com>a0462d9
-
leaderboard: lock GT row + header in a capped gallery scroll box
Michael Rabinovich +1: CursorMake the gallery a scroll container capped at ~78% of the visible viewport so the column header and ground-truth row stay pinned (sticky) at the top while submissions scroll inside it. The cap is set from the parent viewport in JS and the iframe is sized to the (bounded) content, so the page stays compact and shows no scrollbar when the rows are shorter than the cap. Co-authored-by: Cursor <cursoragent@cursor.com>5dd0c03 -
leaderboard: fixed 4-sample gallery, auto-size iframe, center modal
Michael Rabinovich +1: CursorReplace the gallery's interactive fixture picker with a fixed comparison sheet: two generation + two editing samples, one Medium and one Hard per task. Difficulty is chosen once from the Claude Opus 4.8 baseline's per-sample CAD scores (median = Medium, 20th percentile = Hard) and frozen into FIXED_FIXTURES; the GUI no longer recomputes it as the leaderboard evolves, trading a moving ground truth for a simple, stable interface. Adds tools/pick_gallery_fixtures.py to regenerate the constant. Size the gallery iframe to its content (no more 90vh fixed box / nested scrollbar) and pin the compare modal to the visible part of the iframe so it opens centered in view instead of far below the fold. Co-authored-by: Cursor <cursoragent@cursor.com>08eae45 -
deploy: bump cadgenbench to 72f6e82 (scale-relative edit-diff tol, viz polish)
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>96dcaf6 -
deploy: bump cadgenbench to 850281d (unified viz colours)
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>4bfd457
-
leaderboard: bump cadgenbench to d514d6a (report thumbnail grid)
Michael Rabinovich +1: CursorPins the cadgenbench install to the report-grid commit so newly generated hosted reports render the grouped thumbnail-grid summary view. Excludes the later f9b142b scoring change deliberately. Co-authored-by: Cursor <cursoragent@cursor.com>fd0c525 -
leaderboard: backfill tool to grid-ify already-published reports
Michael Rabinovich +1: CursorRewrites the summary view of published reports/<id>.html in the submissions dataset from the old flat table to the thumbnail grid, in place and without re-evaluating. Parses each report's existing rows (sample/status/CAD) + the render-bucket base already embedded in the file, classifies editing by the starting-shape renders (so invalid edits group correctly), and rebuilds the grid via single_run's shared builders (byte-identical to a fresh report), pointing editing cards at edit_diff.png and generation cards at output iso. Injects the shared grid CSS/JS; detail cards, header and download button are left untouched. Supports --files and --dataset (--dry-run); idempotent. Co-authored-by: Cursor <cursoragent@cursor.com>ba3eefb -
leaderboard: admin rescore (selected + all) reusing the eval pipeline
Michael Rabinovich +1: CursorAdd a maintainer-only rescore that re-evaluates submissions against the current ground truth/data, reusing the submit worker end to end: each target row is reset to the pending regime (scores cleared, submitted_at preserved) and _spawn_worker is re-dispatched, which re-renders the gallery, regenerates reports/<id>.{html,json}, and recomputes the score. - rescore_rows(ids): selected rows; LookupError on unknown id before any write. rescore_all(): every row with a stored zip that isn't pending (skips legacy zip-less seed rows + in-flight evals). - Bulk dispatch is staggered on a background thread so a board-wide rescore doesn't fire N run_job calls at once; fully re-runnable, so an interrupted rescore converges on a retry. - UI in the Admin tab: "Rescore selected" behind a confirm checkbox and "Rescore ALL" behind a type-to-confirm phrase, both admin-gated with a server-side re-check and disarmed after firing / on load. Co-authored-by: Cursor <cursoragent@cursor.com>2893b22 -
leaderboard: backfill tool for edit_diff.png stills in render bucket
Michael Rabinovich +1: CursorOne-time tool to close the gap for submissions evaluated before the eval pipeline started writing edit_diff.png: lists the render bucket, finds every <fixture>/edit_diff.webp lacking a sibling edit_diff.png, extracts frame 0 via cadgenbench.common.imaging.first_frame_png (the exact frame the pipeline now saves), and uploads the still to the same bucket prefix. Idempotent (skips existing stills); --dry-run lists without a token. Co-authored-by: Cursor <cursoragent@cursor.com>c7f83a5 -
leaderboard: bump cadgenbench to 1010043 + legend backfill tool
Michael Rabinovich +1: CursorPin to the report generator with color-chip interface/edit-diff legends, and add tools/backfill_report_legends.py to retro-patch the corrected legends into already-published reports/<id>.html in place (no re-eval). Co-authored-by: Cursor <cursoragent@cursor.com>1a24d4a -
leaderboard: rebuild Tasks tab as grouped thumbnail grid
Michael Rabinovich +1: CursorReplace the flat 84-row Tasks list with a responsive card grid grouped by type (Generation / Editing) with live count badges, a number search, and an All/Generation/Editing segmented filter. Each card shows the task's input geometry thumbnail (generation -> input.png; editing -> renders/iso.png, the iso render of input.step) plus the sample number and type tag. Thumbnails use loading="lazy" + decoding="async" with reserved 4:3 dimensions and CSS-constrained width so the 84 images don't all fetch at once. Card click opens the task's existing detail card (prompt + input), keeping j/k/Esc navigation. System fonts only. Co-authored-by: Cursor <cursoragent@cursor.com>95f3ee8 -
leaderboard: download-zip button in merge-path report + backfill tool
Michael Rabinovich +1: CursorPass the submission-zip URL to generate_html in the merge path so reassembled reports show the "Download submission ZIP" button, and add a one-off tools/backfill_report_download_button.py that injects the button into already-published reports/<id>.html in place (no re-eval). Bump CADGENBENCH_SHA to 0c7690e. Co-authored-by: Cursor <cursoragent@cursor.com>c48c18f -
leaderboard: rename tabs, relabel fixtures as samples, inline gallery row stats
Michael Rabinovich +1: Cursor- Rename the visual "Gallery" tab to "Leaderboard" and the table tab to "Detailed View" (order unchanged); update the smoke test accordingly. - Replace user-facing "fixture" wording with "sample" across the gallery, task browser, submit instructions, and submitter status/error messages. Internal data keys, routes, CSS, and identifiers are unchanged. - Drop the gallery's "more numbers" expander: surface Generation/Editing inline under the aggregate score and move Download ZIP under the name. - Link each gallery submission title to its hosted report (same /reports/<id>.html the Detailed View table uses); drop the date line. Co-authored-by: Cursor <cursoragent@cursor.com>461547b -
leaderboard: serve GT report assets via proxy; link them in hosted report
Michael Rabinovich +1: CursorAdd a /gt/{fixture}/{relpath} proxy route that streams the private GT view PNGs + PDF (token-holding), and pass gt_base_url/input_base_url to generate_html in the merge path so the hosted report links GT/input/overlay (lazy) instead of base64-inlining them. Bump CADGENBENCH_SHA to 25943a0. Co-authored-by: Cursor <cursoragent@cursor.com>5140b0a -
leaderboard: serve renders from the public bucket, not the dataset proxy
Michael Rabinovich +1: CursorCandidate renders (turntables + edit-diff WebP) now live in the public HF Storage Bucket under renders/<id>/, uploaded once by the eval job. The gallery grid and the hosted report reference them by anonymous bucket URL, so the Space is out of the render read path and the binary renders stop bloating the submissions dataset / its commit queue. - leaderboard: HF_RENDER_BUCKET + render URL/path/prefix helpers. - app: render resolvers return public bucket URLs; the two render proxy routes + fetchers are retired. Only the private GT render keeps its token-held proxy. - submit: merge passes render_base_url to generate_html and commits report-only; dispatched job gets CADGENBENCH_RENDER_BUCKET/HF_ENDPOINT. - admin: delete also purges the bucket renders/<id>/ prefix (previously orphaned). - Dockerfile: bump cadgenbench pin to 3d49822 (render_base_url support). Co-authored-by: Cursor <cursoragent@cursor.com>d2161b1
-
leaderboard: show edit-diff turntable in gallery grid for editing fixtures
Michael Rabinovich +1: CursorThe gallery grid tile for an editing fixture now serves the ghost edit-diff WebP (renders/<id>/<fixture>/edit_diff.webp) instead of the plain candidate turntable, so a small edit is legible at a glance. Generation tiles, the GT column, and the cell-click modal are unchanged (the modal keeps the plain aligned output via img; the grid uses a new gridImg). - gallery: per-cell gridImg + gridRenderFor hook; build/render take a diff_resolver. - app: _fetch_render_diff + /render-diff proxy route + resolver. A miss 404s to the dashed cell (no fallback to rotating.webp). Co-authored-by: Cursor <cursoragent@cursor.com>e611f15 -
leaderboard: move Tasks tab after Leaderboard
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>31854f7 -
leaderboard: add Tasks tab to browse benchmark fixtures
Michael Rabinovich +1: CursorA read-only task browser that mirrors the per-submission report's summary-table -> detail-card navigation (j/k, Esc, Prev/Next, deep-linkable) but shows only the prompt + input (drawing for generation, starting-shape renders for editing) with no scores or ground truth. Fixtures come from each <fixture>/description.yaml in the inputs dataset; input images are proxied through the Space's read token via /task-input and lazy-loaded. Co-authored-by: Cursor <cursoragent@cursor.com>f4924d6 -
Serve rotating WebP turntables + GT generator
Michael Rabinovich +1: CursorPoint the gallery and proxy routes at rotating.webp (submissions and GT), publish the full per-fixture renders folder, and add a reusable one-time GT turntable generator that renders from the cached GT mesh. Co-authored-by: Cursor <cursoragent@cursor.com>c1cb5e4 -
stage shards via bucket API instead of volume mount
Michael RabinovichAvoids Space volume-mount issues on an existing Space: shard jobs sync artifacts to the HF bucket via the bucket API, and the Space syncs them back down to merge. Gated on CADGENBENCH_SHARD_BUCKET.920b1b4 -
fix HF Jobs polling namespace
Michael Rabinovichdd284f3 -
leaderboard: open reports in new tab
Michael RabinovichNow that the Space is public, link completed submissions directly to their proxied HTML reports instead of inlining large report payloads through the row-click detail panel.5fb3ebc -
stage shard artifacts through HF bucket
Michael Rabinovich6ffd043 -
test: fix leaderboard smoke test for gallery-default tab + sr-only label
Michael Rabinovich +1: CursorGallery is now the default tab, so the Leaderboard content is hidden on load; click the Leaderboard tab before asserting. Scope the label locator to the visible <p> since the Leaderboard widget also renders a screen-reader-only <caption> with the same text (strict-mode 2-element violation otherwise). Co-authored-by: Cursor <cursoragent@cursor.com>1004adf -
gallery: searchable fixture picker, URL-persisted picks, cached render proxies
Michael Rabinovich +1: CursorReplace the always-open fixture chip grid with the compact reference UI: current picks as removable pills + an "Add fixture" button opening a searchable, grouped (Generation/Editing) dropdown, capped at 3 with a "remove one to add another" note. Persist the 3 picks in ?fixtures= so shared links reproduce the view; URL read/replaceState are try/catch guarded for sandboxed iframes. Drop the external Google Fonts @import for system font stacks so the page never errors in a sandboxed frame. Serve thumbnails as lazy-loaded, hard-cached /render and /gt-render proxy URLs (loading="lazy" + decoding="async" + CSS-constrained width) instead of base64-inlining every tile, so only the ~33 visible thumbnails load and fixture swaps / repeat visits hit cache. Requires the Space to be public. Fix a stale test expecting the old 6-tuple _admin_delete signature. Co-authored-by: Cursor <cursoragent@cursor.com>4a9408a -
admin: stop pending jobs before delete
Michael RabinovichAdd an admin action that cancels matching in-flight HF Jobs before deleting submissions, and keep the admin table refreshed so pending rows stay visible.c4b5d70 -
submit: stream live eval progress to the submitter's status panel
Michael Rabinovich +1: CursorAdds a small in-process progress registry (progress.py) that the eval worker publishes stage notes to (queued -> waiting for GPU -> evaluating -> collecting results -> done/failed, plus an "X of N chunks done" count for sharded runs). handle_submit keeps its generator alive after queuing and streams those notes into the personal status panel until a terminal stage or a backstop deadline. The shared leaderboard table stays coarse; this is the personal-view feedback only, so it adds no write load to the results file. Also fixes the _with_hub_retries tests for the newer huggingface_hub HfHubHTTPError(response=...) signature. Co-authored-by: Cursor <cursoragent@cursor.com>b0f4559 -
submit: publish merged report + gallery in one commit
Michael Rabinovich +1: CursorReplace the per-file gallery-thumbnail uploads (one upload_file commit per fixture, ~80+ for a full fan-out) with a single create_commit that stages reports/<id>.{html,json} plus every renders/<id>/<fixture>.png at once. Collapses finalize from ~86 commits to one publish commit, cutting post-shard finalize from minutes to well under a minute and removing the 429 "concurrency queue" failures that stranded earlier submissions. Co-authored-by: Cursor <cursoragent@cursor.com>07430f9 -
submit: retry Hub commits on 429/5xx + persistent status panel
Michael Rabinovich +1: CursorWrap every Space-side commit (zip upload, results.jsonl RMW, report + gallery uploads, shard cleanup) in a backoff-with-jitter retry that honors Retry-After, so a rate-limited submissions repo delays rather than fails a submit. Make handle_submit a generator streaming a durable status panel (validating -> uploading -> queued, or the rejection reason) instead of relying on transient toasts. Co-authored-by: Cursor <cursoragent@cursor.com>b224eee
-
submit: allow missing fixture outputs
Michael Rabinovichdc8ff2a -
gallery: refresh after admin changes
Michael Rabinovichdaae24c -
pin cadgenbench to e446cca (benchmark-v1 aligner + sidecar meshes)
Michael Rabinovich +1: CursorCo-authored-by: Cursor <cursoragent@cursor.com>30909aa -
admin: refresh table on auth load
Michael Rabinovich02751ff
-
submit: shard large submissions across HF Jobs + merge (UC3)
Michael Rabinovich +1: CursorSubmissions over SHARD_THRESHOLD (12) fixtures now fan out into 12-fixture shards dispatched at once, polled to terminal with ERROR-only retry, then merged into one run via write_run_summary (regenerating the report + gallery from the merged whole) before the row flips. At/under the threshold a submission stays a single job, so the original path is unchanged. Eval is CPU-bound, so more shards is the throughput lever. Co-authored-by: Cursor <cursoragent@cursor.com>293bc3b -
gallery: stop negative-caching render fetches
Michael Rabinovich +1: CursorRenders land after boot (a submission completes, or an existing row is backfilled), so lru_cache was pinning boot-time 404s and keeping thumbnails dashed until the next restart. Drop the memoization on the render byte-fetchers; hf_hub_download keeps its own per-revision disk cache, so re-fetching an unchanged thumbnail stays cheap. Also picks up the 3 baseline rows' just-backfilled renders on this redeploy. Co-authored-by: Cursor <cursoragent@cursor.com>8eb8954 -
add visual Gallery tab (top-10 verified, sticky GT, fixture picker)
Michael Rabinovich +1: CursorVisual-first leaderboard as a new default tab. Self-contained HTML (gallery.py) inlined into an iframe srcdoc with base64 thumbnails, so it works on the private Space where HF's edge 404s in-browser custom-route fetches (same constraint the report viewer's srcdoc path already handles). Top-10 verified only, teal sticky ground-truth row, 3-fixture picker (drop-oldest, global re-render), dashed cell for invalid/missing fixtures, per-row generation/editing/validity expander, and a GT-vs-output compare modal per thumbnail. Existing Leaderboard / Submit / About / Admin tabs unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>01d67e9 -
docker: cache-bust the cadgenbench @main install
Michael Rabinovich +1: CursorADD the GitHub commits endpoint for the tracked ref before the pip install so BuildKit invalidates the cached layer whenever main moves; otherwise @main would stay pinned to the first build's HEAD. No-op when CADGENBENCH_SHA is a fixed commit SHA. Co-authored-by: Cursor <cursoragent@cursor.com>3909559