Spaces:
Running
Running
Commit History
14 days ago
-
Sync from evaleval/eval-cards via hub-sync
Verified signature46d6cd4
-
Sync from evaleval/eval-cards via hub-sync (part 2)
Verified signature109cc95
-
Add Help: intro tour, tutorials, stakeholder guides, citation info, footer
Ankaf737def
-
Update about (#8)
- refactor(model-resolution-rework): drop client-side identity logic, use view-layer (22c2706b763b5675f9c29bd1b1bb5c92acc24100) - docs: rewrite README for the v2 reader architecture (da94178c65ac75f24d6f202a1dacdf6fb27d218a) - chore: make working notes private (untrack notes/ + docs/INTERPRETIVE_SIGNALS.md) (aec0823295ab00dbea1a7c2cf56529fbbd271d19) - Fix model pages, redirects, developer pages, and charts for folded model ids (d1e9d019b1f2bddcd7f077fe15f32fd51d940c07) - Add an opt-in frontend preflight check suite (ffd4efae6f4288be2743039f0e4c6b4d69e0edcb) - Update About page (a64cd6c04efd9b6536e8f996376ddaf600d505a2)599471d
-
Add /api/_diag probe; restore @duckdb/node-api 1.5.3-r.2
The 1.5.2 rollback did not help — the rebuilt Space still fails with "don't know what type:" on a freshly purged + re-downloaded cache, on the old binding. So neither the disk cache nor the binding version is the cause. The Space's pinned snapshot (warehouse/2026-05-29T00-24-44Z) also reads cleanly under linux/amd64 Rosetta emulation locally, which points at a real-hardware (Sapphire Rapids / AVX-512) Parquet-decode path that emulation doesn't exercise. Adds a temporary read-only /api/_diag endpoint to isolate, on the Space hardware itself, which nested column's to_json decode trips the Thrift error, and whether thread count or the local-file vs httpfs read path matters. Restores the binding to the preferred 1.5.3-r.2 (version proven irrelevant to the failure). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>5fe6792 -
Roll back @duckdb/node-api 1.5.3-r.2 -> 1.5.2-r.1 (diagnostic)
Diagnostic push to test whether the older DuckDB binding affects the "don't know what type:" parquet-decode failure on the Space. Note: this push also triggers a Space rebuild, which purges the /data/sidecars disk cache (BUILD_ID change) and re-pulls warehouse/latest — so a recovery here likely reflects the cache purge, not the version change. Local linux/amd64 testing read all 5528 models cleanly on both versions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>527aae9
-
Bump @duckdb/node-api 1.5.2-r.1 → 1.5.3-r.2; suppress nested embed button
The deployed Space surfaced a stream of "Invalid Error: don't know what type:" failures from the DuckDB Node bindings on /api/model- summary and /api/eval-summary after a parquet snapshot bump — a known class of error where the binding hits a column type it can't deserialize. 1.5.3 ships the matching engine release and fixes type coverage for the affected nested-type cases. Local prod build with v2 backend continues to serve both endpoints with the new version. Also: when the histogram plotbox renders inside the histogram embed itself (embedSurface === "histogram"), suppress its inline EmbedButton so the iframe doesn't sprout a button that just links back to itself. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>247b1c5
-
Precompute eval matrices for multi-metric + per-slice leaderboards
The warehouse already carries per-(model, metric) values in eval_results_view and per-(model, slice, metric) values in fact_results, but getEvalSummaryById's SQL filters everything down to the primary metric — so evals like Fibble Arena (3 declared metrics) and Global MMLU Lite (19 language slices) reach the frontend with a single column and no leaderboard_rows. The MultiMetricLeaderboard component and SliceSelector consequently never light up. Add scripts/build-eval-matrices.mjs as a build-time step that joins the two views once and emits data/eval-matrices.json — a per-eval map of leaderboard_rows (model_route_id → values{column_key}) and synthetic subtask-scope leaderboard_metrics. The slice column_key is "<metric_id>::<slice_key>" so it slots straight into the existing column_key schema. view-data.ts loads the JSON lazily on first eval-summary call (one disk read per server start) and splices its rows + subtask metrics into the returned summary. Local dev without the precomputed …553b175
-
Add new component files and align app to EvalEval design system
Adds score-distribution, model-table, eval-node-table, family-table, developer-table, infinite-scroll components and logo assets that were missing from the repo, causing the build to fail. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>dbdd6d1
-
Add DuckDB shadow-read backend with source-metadata fix
Jenny Chim +1: Claude Opus 4.7 (1M context)Two-part change behind a feature flag. Backend dispatcher (lib/data-backend.ts) routes 11 API routes + app/page.tsx to either lib/model-data.ts (default JSON path) or the new lib/duckdb-data.ts based on DATA_BACKEND=duckdb. The DuckDB path reads payload_json from local Parquet via @duckdb/node-api; pipeline-side export is gated by EXPORT_EXPERIMENTAL_PARQUET=1 (lives in eval_cards_backend_pipeline). For parity testing, lib/hf-data.ts gains HF_DATA_LOCAL_DIR (override the default .cache/hf-data path) and HF_DATA_OFFLINE=1 (refuse remote fetches so background refreshes can't poison parity tests). scripts/compare-data-backends.mjs diffs the two backends endpoint-by-endpoint. Source-metadata fallback (item #4 in notes/ts-to-pipeline-migration.md) deleted: pipeline now carries source_metadata on every model_result row (verified 86,183/86,183 production hierarchy + 43,859/43,859 eval-detail). Removed getCanonicalSourceMetadata, buildSourceMetadataIndex, and three inline duplicates in lib/…2fcae3f
-
Use HF dataset's peer-ranks.json instead of local recomputation
The backend already ships peer-ranks.json keyed by eval_summary_id; the local precompute-ranks.mjs was re-deriving the same data from the legacy data/models/*.json dir. Swap the build step to run cache-hf-data.mjs so the dataset copy (including the new comparison-index.json) gets pulled into public/ and .cache/hf-data/ at build time, and delete the redundant script. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>6a6446b
-
Refresh eval cards UI and backend data flow
c1f2130
-
fix bugs
ae1dc39
-
flatten data
f4514c9
-
updated nextjs
29e08d7
-
fixed a lot of bugs, centralized schema
49d5ba7
-
unit tests added
a58dac7 -
added all the new files
509e21e