Spaces:
Running
Running
| # AGENTS.md | |
| The SelectArena leaderboard: a static Hugging Face Space (`sdk: static`) that serves files | |
| generated in the [SelectArena](https://github.com/schaeferbasti/SelectArena) repo. Static Spaces | |
| run no compute, so they are free; do not switch it to Gradio or Docker (those need a paid plan). | |
| ## Files | |
| - `index.html`: the whole page: header, the four tab panels (Leaderboard, Per dataset, | |
| Stability & validity, About), all user-facing text, the citation, `<meta name="selectarena-version">` | |
| and the version history (`<ul id="version-history">`, newest first). | |
| - `assets/app.js`: plain JavaScript, no build step. Fetches `data/meta.json`, renders the selector | |
| chips (task type, downstream model, budget; combinations that are not published are disabled), | |
| sets the iframes' `src` to the generated pages and resizes them when a page posts | |
| `{saFrameHeight}`. Keeps the selection in the URL hash. | |
| - `assets/style.css`: color tokens on `:root` (light) and in `prefers-color-scheme: dark`. | |
| - `smoke_test.py`: standard-library check of `data/` and of the version (see below). | |
| ## Data contract | |
| `data/` is SelectArena's `generated_website_artifacts/clean_website_artifacts/website_data/`: | |
| - `meta.json`: counts, `models` (key -> name), `budgets`, `total_budget`, `task_types`, | |
| `has_stability`, `has_validity`, `validity_noise`, `validity_tables`, `generated_at`, and `subsets` (each with | |
| `tasks`, `model`, `budget`, `path`, `n_methods`, `n_datasets`, `n_comparisons`). | |
| - `subsets/tasks_<t>/models_<m>/budgets_<b>/`: `website_leaderboard.csv`, `leaderboard_table.html`, | |
| `elo_explorer.html`. | |
| - `per_dataset.csv`, `per_dataset_table.html` (datasets by name), and optionally `stability.csv`, | |
| `stability_table.html`, `validity.csv` and one `validity_noise_<level>_table.html` per noise level | |
| (listed in `meta.json` as `validity_tables`). | |
| - `website_leaderboard.csv` columns: `Rank`, `Method`, `Category` (Search / Non-search / Baseline), | |
| `Subgroup` (the paper's taxonomy, e.g. Tree, Backward search), Elo and its interval, the metrics. | |
| The generated pages are self-contained (inline CSS/JS, no network) and post their height to the | |
| parent window. `subsetDir()` in `app.js` mirrors `selectarena.website.subsets.Subset.rel_path`; | |
| change both together. | |
| ## Conventions | |
| - Never hand-edit `data/`: refresh it with the `update-leaderboard` skill in the SelectArena repo | |
| (`../SelectArena/.claude/skills/update-leaderboard/`), which deletes the old content before copying. | |
| - Every refresh bumps `selectarena-version` in `index.html` (patch for a data refresh, minor for | |
| site changes) and adds a matching entry at the top of the version history | |
| (`<li><strong>YYYY/MM/DD · vX.Y.Z</strong>: ...</li>`). `smoke_test.py` fails if they differ. | |
| - No external scripts, fonts or CDNs; everything is served from this repo. | |
| - Verification: `python smoke_test.py`, then `python -m http.server 8000` and a look at | |
| http://localhost:8000 (all four tabs, a few selections, light and dark mode). | |
| - Do not commit or push without an explicit request: pushing deploys the Space. | |