diff --git a/AGENTS.md b/AGENTS.md index 53f5d916ea0086c1b5f7282a911ffe5d0fcd0026..d2663f654983ecd64642835be8deb634c89258b3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -6,13 +6,15 @@ run no compute, so they are free; do not switch it to Gradio or Docker (those ne ## Files -- `index.html`: the whole page: header, the four tab panels (Leaderboard, Per dataset, - Stability & validity, About), all user-facing text, the citation, `` +- `index.html`: the whole page: header, the five tab panels (Performance, Stability, Validity, + Data overview, About), all user-facing text, the citation, `` and the version history (`
| Dataset | Task | Method | Model | Budget | #Features | Metric | Error | Error std. | Splits | Selection time (s) | Fallback (%) |
|---|
Elo rating of each feature selection method, from worst to best. Elo @@ -74,37 +76,102 @@ injected noise features are rejected); darker is better for both. Hatched bars have no stability or validity result yet.
+Mean rank of each method on each dataset, over splits, downstream models and budgets (1 is best).
-Stability (Nogueira, Sechidis & Brown, JMLR 2018): agreement of the + features a method selects across bootstrap resamples of the data, corrected for chance; 1 means + identical selections, 0 the agreement of random selections. It comes from runs of the selectors + alone, without a downstream model.
+These metrics come from runs of the selectors alone, without a downstream model.
-Validity: share of the budget a method fills with original features + after synthetic noise features are added to the data. The noise level is the number of noise + features per original one; 1 is best. It comes from runs of the selectors alone, without a + downstream model. The leaderboard's Elo figure uses the validity at noise level + 1.0.
The results behind the leaderboard: one row per dataset, feature selection method, + downstream model and budget, averaged over the splits (All features is the downstream + model without feature selection). Filter, sort and download the rows you need, or get all of + them as results.csv.
+Mean rank of each method on each dataset, over splits, downstream models and budgets (1 is best).
+