Spaces:
Running
Running
Commit History
22 days ago
-
Generate leaderboard data from fev main at build time Running
The Dockerfile now runs save_tables.py against the latest autogluon/fev main, so a newly merged model goes live with a Factory rebuild of the Space, without committing to this repo. tables/ and src/vendor/ are no longer committed. The About page shows which fev commit the running Space was built from. Also switches the sidebar control color to a muted blue.88036ff -
Rework leaderboard table and sidebar filters
- Model names link to the model page; type colors and the link column are replaced by a Tags column with pale chips (model type, Zero-shot, Non-commercial). - Sidebar: model-type pills (System and Closed API off by default) and a toggle to exclude non-commercial models. Win rates are recomputed for the selection; Seasonal Naive and Chronos-Bolt always stay in for imputation. - "See details" is split into Performance, Tags and Imputation sections, with tag badges matching the table. - Shorter column headers; narrower pairwise skill score color range (±15%). - save_tables.py registers loaded modules in sys.modules so pydantic models in the fev repo resolve their annotations. - Pin streamlit==1.53.1: MultiselectColumn does not exist in 1.49. - Tables regenerated from autogluon/fev 0c9d174.283c318
23 days ago
-
Read model metadata from the fev repo instead of a local dict
Model attributes (organization, link, type, zero-shot, commercial use) now come from benchmarks/fev_bench/models.yaml in autogluon/fev, where the submitter declares them, so adding a model to the leaderboard is a pure data refresh: `uv run python save_tables.py [commit]`. - save_tables.py fetches one fev commit and takes results, metadata and compute logic from that single checkout (results and compute could previously come from different commits). Records the commit in tables/source.json and vendors the compute module into src/vendor/. - The sidebar lets readers include `system` and `closed-api` models, which are excluded by default. Win rate and skill score depend on the included set, so the app now computes leaderboard/pairwise per (task group, metric, included types) and caches them, replacing the 17 precomputed tables/<group>/ directories. - Models flagged `hidden` in models.yaml are dropped before metrics are computed, matching the paper figures; previously they …abd8d7f
-
Add CITRAS-FM
a396ab4
-
Add TiRex-2 results
fdf3780
-
Add breakdown by task type
daddcee -
Update pivot table
7f23124
-
Update results
5eb256c
-
Update results
86cf9ec
-
Update description
53f3bdd -
Update results per group
169601b -
Separate tables by subset
d4397a6
-
Update strings
393f9c5 -
Add tables
18a3564 -
Add new leaderboard
eb8267f