app+leaderboard: link submission_name to a Space-side report proxy
Michael Rabinovich
77edebf
Polish follow-up #2 on C6. Three user-visible changes plus the
plumbing required to make them work.
1. Drop the `report` and `zip` columns from the rendered table.
`submission_name` becomes the report's deep link (typical HF
leaderboard pattern: the system / model name doubles as the link
target; cf. Open LLM Leaderboard, DABstep, BigCodeBench). Modern
completed rows get `[<name>](/reports/<sid>.html)`; pending,
failed, and legacy seed rows fall back to plain text so there's
no broken click. The hidden `submission_blob_url` and the
computed `report_url` ride along in the DataFrame and surface in
the row-click detail panel only.
2. Rename the agent-URL column from `agent` to
`model details (optional)`. Missing cells render as italic
`_None_` so the optionality is explicit instead of a blank.
"Agent" was confusing; the schema lets `agent_url` be code OR
paper OR a blog post and "agent" reads narrow.
3. Add a Space-side proxy for the per-submission H…
Polish follow-up on C6, three fixes after the first detail-panel
commit landed:
1. Rename display columns from snake_case schema names to clean
short headers. `agent_url` -> `agent`, `submission_blob_url` ->
`zip`, `report_url` -> `report`. The rename happens at projection
time via `df.rename`, so the schema in results.jsonl is
unchanged. Headers now read as plain English ("agent", "zip",
"report") rather than a python dict.
2. The `agent` column shows the URL itself (scheme stripped,
truncated past 40 chars) as link text instead of a fixed "code"
label. The schema says agent_url can be code OR paper; the
shortened URL is the only honest hint about what's behind the
click. Missing values render as italic `_None_` rather than a
blank cell, so the optionality is explicit.
3. The `report` link is now gated on the row's `submission_sha256`
being non-null in addition to status == "completed". Legacy seed
rows pre-date the modern submit pipeline (which …
leaderboard: markdown link columns for agent_url, submission, report
Michael Rabinovich
53de73a
Bundle 1+2 C5. Three new columns on both tables (validated +
unvalidated) carry clickable links rendered via gradio_leaderboard's
`datatype="markdown"`:
- agent_url: link to the agent's source / paper if the row's
meta.json provided one (label "code"; empty cell when null).
- submission_blob_url: link to the uploaded zip on the submissions
dataset (label "zip"; populated on every non-failed row, the
same URL the submit handler computed at upload time).
- report_url: built at read time from `submission_id` plus
HF_SUBMISSIONS_REPO; points at `reports/<id>.html` under
/resolve/main/ so the browser renders the report directly.
Only emitted for `status == "completed"` rows.
leaderboard.py:
- LEADERBOARD_COLS + VALIDATED_LEADERBOARD_COLS grow by three.
- New LEADERBOARD_DATATYPES + VALIDATED_LEADERBOARD_DATATYPES
constants mark the link columns as "markdown", everything else
as "str". Tied to the column lists by a single helper so the
two stay in lockstep.
- New _agent_u…
submit: default validation + hf_username fields on the pending row
Michael Rabinovich
a533dd2
Bundle 1+2 C4. `_build_pending_row` adds three new keys matching the
schema bump from C1, in the schema.md column order:
- validation_status = "unvalidated" (every new row starts in the
unvalidated tier; maintainers promote via the dataset repo after
methodology review).
- validation_method = None (set by the maintainer at promotion time
to one of code | traces | api | manual).
- hf_username = None (placeholder; C10 plumbs through the value
from gradio.OAuthProfile.username once the LoginButton lands).
Pre-existing fields, submitted_at stamping, the cadgenbench version
+ data revision tags, the score-shaped null fields, the blob URL,
and the sha256 are untouched. Dedup gate still pivots on
submission_sha256 exactly as before.
tests/test_submit.py covers the C4 acceptance:
- new-row defaults: validation_status is "unvalidated",
validation_method + hf_username are None;
- sha256 path passthrough: the hash argument lands on the row's
submission_sha256 key intact (existing …
app: split Leaderboard into Validated + Unvalidated tables
Michael Rabinovich
046548a
Bundle 1+2 C3. Swaps the single Leaderboard widget for two stacked
ones (Validated on top, Unvalidated below) and wires the existing
auto-refresh Timer + Refresh button to push fresh dataframes into
both. Initial render reads via the C2 `load_leaderboard_split()`;
existing rows all land in Unvalidated until a maintainer flips
`validation_status` to `"validated"` on a row in the submissions
dataset.
- app.py: replaces `df_view` with `validated_view` + `unvalidated_view`.
Single timer tick now fans out to both widgets (load split runs
once per tick, both halves get pushed). Refresh button same
fan-out shape. Inline comment points at the validation policy
decision doc.
- leaderboard.py: drops the now-unused single-DataFrame
`load_leaderboard()` per the C3 cleanup half. Stale docstring
references that pointed at the dead function got smoothed out
(the split reader's doc is self-contained now).
- tests/test_smoke.py: picks up the C3 acceptance with a new
`test_two_leaderboa…
leaderboard: add load_leaderboard_split() for two-tier viewer
Michael Rabinovich
0dd7215
Bundle 1+2 C2. Additive new function for the C3 two-stacked-table
swap; existing single-DataFrame load_leaderboard() stays put until
C3 flips the caller in app.py + drops it. Keeps the live Space green
between commits.
- VALIDATED_LEADERBOARD_COLS: same shape as LEADERBOARD_COLS plus
`validation_method` (always null on unvalidated rows, so omitted
from that tier rather than rendered).
- load_leaderboard_split() -> (validated_df, unvalidated_df):
reads results.jsonl the same way as load_leaderboard(), splits
rows on `validation_status` (defensive: anything not literally
"validated" lands in unvalidated, which covers legacy + null +
any future-unknown value), runs the same status-aware cell
formatting per tier via the new _project_and_format helper.
- _project_and_format: shared sort + projection + pending/failed
cell tagging so the two tiers produce identically-shaped cells.
Legacy default: rows without `validation_status` get "unvalidated"
via row-level fill. No resul…
First Space-side test harness; sets up the floor for the Phase D
bundle's per-commit unit + UI tests (split leaderboard, pending-row
defaults, OAuth-required submit) without slipping a regression past
a build-only check.
requirements-dev.txt pulls -r requirements.txt then adds pytest +
playwright. One `pip install -r requirements-dev.txt && playwright
install chromium` brings a fresh venv up to speed.
tests/conftest.py exposes a session-scoped `app_url` fixture: boots
app.py in a subprocess on a free port, polls until the HTTP server
answers, yields the URL, tears the process down on teardown. Logs
land in a tmp file so a startup failure surfaces the real Gradio
traceback rather than a bare RuntimeError. Boot-time stuck-pending
sweep in submit.py is disabled via CADGENBENCH_DISABLE_BOOT_SWEEP=1
so the fixture is fast and offline-tolerant.
tests/test_smoke.py launches headless Chromium via playwright's sync
API, navigates to the fixture's URL, and asserts the Leaderboard /
Submit / A…
