# AGENTS.md

The SelectArena leaderboard: a static Hugging Face Space (`sdk: static`) that serves files
generated in the [SelectArena](https://github.com/schaeferbasti/SelectArena) repo. Static Spaces
run no compute, so they are free; do not switch it to Gradio or Docker (those need a paid plan).

## Files

- `index.html`: the whole page: header, the four tab panels (Leaderboard, Per dataset,
  Stability & validity, About), all user-facing text, the citation, `<meta name="selectarena-version">`
  and the version history (`<ul id="version-history">`, newest first).
- `assets/app.js`: plain JavaScript, no build step. Fetches `data/meta.json`, renders the selector
  chips (task type, downstream model, budget; combinations that are not published are disabled),
  sets the iframes' `src` to the generated pages and resizes them when a page posts
  `{saFrameHeight}`. Keeps the selection in the URL hash.
- `assets/style.css`: color tokens on `:root` (light) and in `prefers-color-scheme: dark`.
- `smoke_test.py`: standard-library check of `data/` and of the version (see below).

## Data contract

`data/` is SelectArena's `generated_website_artifacts/clean_website_artifacts/website_data/`:

- `meta.json`: counts, `models` (key -> name), `budgets`, `total_budget`, `task_types`,
  `has_stability`, `has_validity`, `validity_noise`, `validity_tables`, `generated_at`, and `subsets` (each with
  `tasks`, `model`, `budget`, `path`, `n_methods`, `n_datasets`, `n_comparisons`).
- `subsets/tasks_<t>/models_<m>/budgets_<b>/`: `website_leaderboard.csv`, `leaderboard_table.html`,
  `elo_explorer.html`.
- `per_dataset.csv`, `per_dataset_table.html` (datasets by name), and optionally `stability.csv`,
  `stability_table.html`, `validity.csv` and one `validity_noise_<level>_table.html` per noise level
  (listed in `meta.json` as `validity_tables`).
- `website_leaderboard.csv` columns: `Rank`, `Method`, `Category` (Search / Non-search / Baseline),
  `Subgroup` (the paper's taxonomy, e.g. Tree, Backward search), Elo and its interval, the metrics.

The generated pages are self-contained (inline CSS/JS, no network) and post their height to the
parent window. `subsetDir()` in `app.js` mirrors `selectarena.website.subsets.Subset.rel_path`;
change both together.

## Conventions

- Never hand-edit `data/`: refresh it with the `update-leaderboard` skill in the SelectArena repo
  (`../SelectArena/.claude/skills/update-leaderboard/`), which deletes the old content before copying.
- Every refresh bumps `selectarena-version` in `index.html` (patch for a data refresh, minor for
  site changes) and adds a matching entry at the top of the version history
  (`<li><strong>YYYY/MM/DD · vX.Y.Z</strong>: ...</li>`). `smoke_test.py` fails if they differ.
- No external scripts, fonts or CDNs; everything is served from this repo.
- Verification: `python smoke_test.py`, then `python -m http.server 8000` and a look at
  http://localhost:8000 (all four tabs, a few selections, light and dark mode).
- Do not commit or push without an explicit request: pushing deploys the Space.