AGENTS.md

AGENTS.md

The SelectArena leaderboard: a static Hugging Face Space (sdk: static) that serves files generated in the SelectArena repo. Static Spaces run no compute, so they are free; do not switch it to Gradio or Docker (those need a paid plan).

Files

• index.html: the whole page: header, the four tab panels (Leaderboard, Per dataset, Stability & validity, About), all user-facing text, the citation, <meta name="selectarena-version"> and the version history (<ul id="version-history">, newest first).
• assets/app.js: plain JavaScript, no build step. Fetches data/meta.json, renders the selector chips (task type, downstream model, budget; combinations that are not published are disabled), sets the iframes' src to the generated pages and resizes them when a page posts {saFrameHeight}. Keeps the selection in the URL hash.
• assets/style.css: color tokens on :root (light) and in prefers-color-scheme: dark.
• smoke_test.py: standard-library check of data/ and of the version (see below).

Data contract

data/ is SelectArena's generated_website_artifacts/clean_website_artifacts/website_data/:

• meta.json: counts, models (key -> name), budgets, total_budget, task_types, has_stability, has_validity, validity_noise, validity_tables, generated_at, and subsets (each with tasks, model, budget, path, n_methods, n_datasets, n_comparisons).
• subsets/tasks_<t>/models_<m>/budgets_<b>/: website_leaderboard.csv, leaderboard_table.html, elo_explorer.html.
• per_dataset.csv, per_dataset_table.html (datasets by name), and optionally stability.csv, stability_table.html, validity.csv and one validity_noise_<level>_table.html per noise level (listed in meta.json as validity_tables).
• website_leaderboard.csv columns: Rank, Method, Category (Search / Non-search / Baseline), Subgroup (the paper's taxonomy, e.g. Tree, Backward search), Elo and its interval, the metrics.

The generated pages are self-contained (inline CSS/JS, no network) and post their height to the parent window. subsetDir() in app.js mirrors selectarena.website.subsets.Subset.rel_path; change both together.

Conventions

• Never hand-edit data/: refresh it with the update-leaderboard skill in the SelectArena repo (../SelectArena/.claude/skills/update-leaderboard/), which deletes the old content before copying.
• Every refresh bumps selectarena-version in index.html (patch for a data refresh, minor for site changes) and adds a matching entry at the top of the version history (<li><strong>YYYY/MM/DD · vX.Y.Z</strong>: ...</li>). smoke_test.py fails if they differ.
• No external scripts, fonts or CDNs; everything is served from this repo.
• Verification: python smoke_test.py, then python -m http.server 8000 and a look at http://localhost:8000 (all four tabs, a few selections, light and dark mode).
• Do not commit or push without an explicit request: pushing deploys the Space.