Spaces:
Sleeping
Sleeping
Commit History
-
minor fix of the matrix script
Yanan Longdf161c3 -
Live snapshot date, hide empty Updated col, clean slice contamination
Three independent fixes that surfaced together: * navigation.tsx: replace the hard-coded 'Snapshot · Apr 30 2026' banner label with the live generated_at from /api/backend-manifest, formatted in the local locale. The label hides itself if the manifest fetch fails. * eval-detail.tsx: hide the 'Updated' column when no row in the visible leaderboard has a usable evaluation_timestamp — every cell was rendering 'Unknown' on evals where the upstream pipeline hasn't filled the timestamp yet (e.g. HF Open LLM v2 / GPQA). Also adjust the expanded-row colSpan to match. * build-eval-matrices.mjs: the slice picker on HF Open LLM v2 / GPQA was showing 'Artificial Analysis', 'LLM Stats', 'OpenEval GPQA' because fact_results emits a per-source pseudo-slice for each benchmark that lives under multiple composites, and the matrix was joining slices on benchmark_id alone. Index slices on (composite_slug, benchmark_id) so each composite's slices stay in its own lane. Also drop…cb0ce7c
-
Precompute eval matrices for multi-metric + per-slice leaderboards
The warehouse already carries per-(model, metric) values in eval_results_view and per-(model, slice, metric) values in fact_results, but getEvalSummaryById's SQL filters everything down to the primary metric — so evals like Fibble Arena (3 declared metrics) and Global MMLU Lite (19 language slices) reach the frontend with a single column and no leaderboard_rows. The MultiMetricLeaderboard component and SliceSelector consequently never light up. Add scripts/build-eval-matrices.mjs as a build-time step that joins the two views once and emits data/eval-matrices.json — a per-eval map of leaderboard_rows (model_route_id → values{column_key}) and synthetic subtask-scope leaderboard_metrics. The slice column_key is "<metric_id>::<slice_key>" so it slots straight into the existing column_key schema. view-data.ts loads the JSON lazily on first eval-summary call (one disk read per server start) and splices its rows + subtask metrics into the returned summary. Local dev without the precomputed …553b175 -
Restructure model details + extend cleanHierarchy for split families and aggregator dedup
Adds a tri-state view selector (Source / Category / Overlaps) to the model detail page. The Overlaps view is a dedicated table with mean and 95% CI across cross-suite appearances; plotbox whiskers now come from the metric's paired stderr, not from cross-family spread. cleanHierarchy gains: split-family folding (Fibble Arena, AgentHarm, BFCL, CySE2, MATH-MC, MT-Bench, GSM-MC) into single benchmarks with splits; paren-suffix grouping for SWE-PolyBench / Multi-SWE-Bench language variants and HLE accuracy/calibration metrics; flatten-composites for RewardBench; score-equality-based aggregator dedup against comparison-index so llm-stats copies that literally match a canonical source's scores drop out at build time; leaderboard wrapper merge that re-routes eval ids into the canonical home; richness/subset/self-wrapper rules to drop redundant wrapper families (big-bench-hard, global-mmlu-lite, livebench, mmlu-pro). Stats are recomputed post-cleanup. Adds an HTML hierarchy explorer (scripts/…06313c1
-
Swap backend data (#3)
- Integrate with test backend data (7635aee64606c5b9138e680b833ca1383b570887) - Drop input_modalities/output_modalities from MODEL_CARD_COLUMNS (bfce8f214eed3054b820176601eaa0a23e31bee7) - Merge remote-tracking branch 'origin/main' into feat/use-new-backend-data (25ba6d010ff24b92f252f8fe11a6624f68aa6690) - Use model_key as the addressable identifier and wire comparison-index sidecar (0e529dce5eb243708739730f1fcec00d27202d71) Co-authored-by: Jenny Chim <j-chim@users.noreply.huggingface.co>fe99ffa
-
Restore DuckDB-aware build cache logic
The rebase took origin/main wholesale for cache-hf-data.mjs to preserve the new corpus-aggregates.json fetch. That dropped four pieces of the DuckDB build path that the production HF Space deploy needs: - isDuckDBLean mode + ESSENTIAL/JSON_FALLBACK split (skips ~2.7 GB of JSON snapshots that OOM the Space) - HF_DATASET_REPO env var override (so the Dockerfile ARG reaches the cache step instead of being silently ignored) - duckdb/ in CACHE_DIRECTORIES (else the parquet that lib/duckdb-data.ts reads is never copied → static prerender fails) - Binary-safe download via Buffer.from(arrayBuffer()) instead of response.text() (text-mode would corrupt parquet) - HF_TOKEN auth header for private/gated dataset access Re-applies 14c3a2d's intent and adds corpus-aggregates.json to the essential set (essential because the corpus dashboard depends on it in both modes; optional because the prod dataset may not have it populated yet). Co-Authored-By: Claude Opus 4.7 (1M con…154e1d8 -
Sort eval-detail filenames by codepoint for parity
Jenny Chim +1: Claude Opus 4.7 (1M context)`parity_outputs.build_aggregate_eval_summaries` on the Python side iterates inputs sorted by `eval_summary_id` (default string compare = codepoint order). The TS dump was using filesystem iteration order, so `foo_25.json` vs `foo.json` could land in either order. Plain `.sort()` (no compareFn) matches Python's codepoint sort. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>819e7c9 -
Deploy DuckDB-backed frontend to
Jenny Chimtemp_evalcard_frontendda8db3e
-
Add three-tier test infrastructure for migration safety
Jenny Chim +1: Claude Opus 4.7 (1M context)Tier A — pipeline contract tests (tests/pipeline-contract.test.ts, ~14 tests against pinned tests/fixtures/) assert that every field the TS code depends on is present in upstream artifacts. tests/upstream-drift.test.ts runs the same shape against the live .cache/hf-data/ (5830 models, 587 evals); opt-in via `pnpm test:drift` (RUN_DRIFT=1 env gate) so it doesn't flap on every upstream refresh. Tier B — adapter snapshot tests (tests/adapters.test.ts) snapshot the output of hfModelCardToEvaluationCardData, hfEvalDetailToSummary, flattenModelEvaluations, and hfDeveloperDetailToSummary against ~17 hand-curated fixtures. Large outputs use a digest (count + distinct sets + sha256 of full output) so the snapshot stays reviewable. Tier C — full-cache differential audit (scripts/audit-adapters.mjs) runs every adapter against either pinned fixtures or the full live cache, produces a deterministic JSON digest, and supports `--diff` mode for before/after comparison of a code change. Catches distr…d3cbe09 -
Add DuckDB shadow-read backend with source-metadata fix
Jenny Chim +1: Claude Opus 4.7 (1M context)Two-part change behind a feature flag. Backend dispatcher (lib/data-backend.ts) routes 11 API routes + app/page.tsx to either lib/model-data.ts (default JSON path) or the new lib/duckdb-data.ts based on DATA_BACKEND=duckdb. The DuckDB path reads payload_json from local Parquet via @duckdb/node-api; pipeline-side export is gated by EXPORT_EXPERIMENTAL_PARQUET=1 (lives in eval_cards_backend_pipeline). For parity testing, lib/hf-data.ts gains HF_DATA_LOCAL_DIR (override the default .cache/hf-data path) and HF_DATA_OFFLINE=1 (refuse remote fetches so background refreshes can't poison parity tests). scripts/compare-data-backends.mjs diffs the two backends endpoint-by-endpoint. Source-metadata fallback (item #4 in notes/ts-to-pipeline-migration.md) deleted: pipeline now carries source_metadata on every model_result row (verified 86,183/86,183 production hierarchy + 43,859/43,859 eval-detail). Removed getCanonicalSourceMetadata, buildSourceMetadataIndex, and three inline duplicates in lib/…2fcae3f
-
Add interpretive signals, corpus dashboard, and slice browser
Surfaces reproducibility, reporting completeness, provenance, and comparability signals from the backend across eval detail, model compare, eval list cards, and a new /corpus dashboard. Adapts to the upstream 2-level hierarchy (family → leaf), caps the leaderboard at 24 default columns and replaces the slice tabs with a search dialog when a benchmark has more than 5 slices. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>bca888a
-
Aggregate setup aliases and clarify benchmark variants
dd0b4fc -
Improve eval/model UX, lite data paths, and leaderboard clarity
436ada0
-
Use HF dataset's peer-ranks.json instead of local recomputation
The backend already ships peer-ranks.json keyed by eval_summary_id; the local precompute-ranks.mjs was re-deriving the same data from the legacy data/models/*.json dir. Swap the build step to run cache-hf-data.mjs so the dataset copy (including the new comparison-index.json) gets pulled into public/ and .cache/hf-data/ at build time, and delete the redundant script. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>6a6446b -
Add per-benchmark comparison histograms on model detail
Rewires the model detail page to render one histogram per (benchmark, metric) backed by the new comparison-index.json artifact. Each card groups peers from the same family (Fibble1…Fibble5 under Fibble Arena, RewardBench 2's components under one card), shows metric tabs with backend-sorted order, exposes submission-axis metadata (harness / variant / rerun) on multi-submission peers, and surfaces secondary metrics or per-split subtasks in an accordion below the chart. Ranks are read directly from comparison-index by_model, retiring the peer-ranks round-trip on this surface. Also adds `/api/eval-detail` for the deep-dive modal and wires `comparison-index.json` into the build-time cache script. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>415ac43 -
Harden aggregate evals and cache refresh
9d14977
-
Refresh eval cards UI and backend data flow
c1f2130
-
fix bugs
ae1dc39
-
redesigned
3a12290
-
fix data
Avijit Ghoshddfc163 -
Refactor: Update benchmarks with realistic data, fix UI stats, and improve About page
Avijit Ghosh2554366 -
new ux
Avijit Ghosh6978d97
-
added all the new files
509e21e