Spaces:
Running
Running
Commit History
-
Sync from evaleval/eval-cards via hub-sync (part 2)
Verified signature2a41883
-
Make the matrix precompute byte-deterministic across runs
Tie-break the per-cell row pick on metric_summary_id and emit both queries in stable order; three consecutive builds now hash identically. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>414bacb -
Matrix leaderboard: canonical-scale cells from the pipeline's ranked row
The matrix precompute stamped each (eval, model, metric) cell from whichever eval_results_view row happened to arrive last, on the raw source scale — so one column could mix 0-1 fractions with 0-100 percents (Scale SEAL hle vs the AISI collection rows), the verified checkmark rode an arbitrary row, and the frontier plot disagreed with the leaderboard ordering. Pick exactly one row per cell — the pipeline's ranked-best (position, then canonical score) — and emit score_canonical so every cell of a column sits on the metric's registry scale; the verified flag rides the same chosen row. Percent-unit columns whose values are all fractions render ×100 with a % suffix (column-uniform, so old snapshots without canonical scores are never half-converted), and the distribution/ frontier series apply the same presentation so plot and table agree. Also make the model_id→route mapping deterministic (min() instead of first-seen DISTINCT order). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>6a311d7
-
Add validated evaluator badge
478ae6c
-
Live snapshot date, hide empty Updated col, clean slice contamination
Three independent fixes that surfaced together: * navigation.tsx: replace the hard-coded 'Snapshot · Apr 30 2026' banner label with the live generated_at from /api/backend-manifest, formatted in the local locale. The label hides itself if the manifest fetch fails. * eval-detail.tsx: hide the 'Updated' column when no row in the visible leaderboard has a usable evaluation_timestamp — every cell was rendering 'Unknown' on evals where the upstream pipeline hasn't filled the timestamp yet (e.g. HF Open LLM v2 / GPQA). Also adjust the expanded-row colSpan to match. * build-eval-matrices.mjs: the slice picker on HF Open LLM v2 / GPQA was showing 'Artificial Analysis', 'LLM Stats', 'OpenEval GPQA' because fact_results emits a per-source pseudo-slice for each benchmark that lives under multiple composites, and the matrix was joining slices on benchmark_id alone. Index slices on (composite_slug, benchmark_id) so each composite's slices stay in its own lane. Also drop…cb0ce7c
-
Precompute eval matrices for multi-metric + per-slice leaderboards
The warehouse already carries per-(model, metric) values in eval_results_view and per-(model, slice, metric) values in fact_results, but getEvalSummaryById's SQL filters everything down to the primary metric — so evals like Fibble Arena (3 declared metrics) and Global MMLU Lite (19 language slices) reach the frontend with a single column and no leaderboard_rows. The MultiMetricLeaderboard component and SliceSelector consequently never light up. Add scripts/build-eval-matrices.mjs as a build-time step that joins the two views once and emits data/eval-matrices.json — a per-eval map of leaderboard_rows (model_route_id → values{column_key}) and synthetic subtask-scope leaderboard_metrics. The slice column_key is "<metric_id>::<slice_key>" so it slots straight into the existing column_key schema. view-data.ts loads the JSON lazily on first eval-summary call (one disk read per server start) and splices its rows + subtask metrics into the returned summary. Local dev without the precomputed …553b175