Add AutoBench Quality, Balanced, and Cost to the Router Benchmark board.

app.py

@@ -524,8 +524,13 @@ def update_leaderboard_data(selected_run_id: str) -> tuple:
 # --- Build Gradio App ---
 def router_quality_cost_figure(board: pd.DataFrame):
     headline = board[board["is_baseline"] == 0].copy()
-    headline["group"] = headline["collapsed"].map(
-        {1: "Collapsed (at least 90% one model)", 0: "Spread"}
+    headline["group"] = headline.apply(
+        lambda row: "AutoBench"
+        if row["vendor"] == "AutoBench"
+        else "Collapsed (at least 90% one model)"
+        if row["collapsed"] == 1
+        else "Spread",
+        axis=1,
     )
     figure = px.scatter(
         headline,
@@ -542,6 +547,7 @@ def router_quality_cost_figure(board: pd.DataFrame):
         },
         log_x=True,
         color_discrete_map={
+            "AutoBench": "#16a34a",
             "Spread": "#155dfc",
             "Collapsed (at least 90% one model)": "#d97706",
         },
@@ -643,14 +649,14 @@ def router_intro_markdown(metadata: Dict) -> str:
 
 {metadata.get("description", "")}
 
-We graded **{facts.get("commercial_routers")} commercial routers** ({facts.get("headline_passes")} passes) on a frozen matrix of {facts.get("questions")} questions and {facts.get("matrix_models")} models ({facts.get("matrix_date")}).
+We graded **{facts.get("commercial_routers")} commercial routers** and AutoBench ({facts.get("headline_passes")} headline passes) on a frozen matrix of {facts.get("questions")} questions and {facts.get("matrix_models")} models ({facts.get("matrix_date")}).
 
 **Accuracy** is the share of questions where the chosen model is statistically as good as the best available answer.
 
 ### Findings
-- The cheap-and-strong region is empty.
+- AutoBench Quality, Balanced, and Cost occupy the cheap-and-strong region. No commercial pass does.
 - {facts.get("collapsed_passes")} of {facts.get("headline_passes")} passes collapsed, sending at least {threshold:.0f}% of questions to one model.
-- Median cross-vendor same-pick is {median:g} across {facts.get("cross_vendor_pairs")} pairs.
+- Median cross-vendor same-pick among the original Wave pairs is {median:g} across {facts.get("cross_vendor_pairs")} pairs. The heatmap stays the 12-pass grid from the report.
 
 The board is ordered by **ARI** (AutoBench Router Index) at β = 0.2, highest first. ARI runs from 0 to 100 and is higher when a pass is both accurate and cheap. A pass no better than random scores 0. Quality is the share of the gap from the random floor to a perfect answer. Cost is a log score between $0.00001 and $1 per question. β = 0.2 is the weight on cost. Each row shows how many questions that pass actually answered (`n`). Read the report: [{metadata.get("report_url", "")}]({metadata.get("report_url", "")}).
 """

runs/router1_2026-10-26/metadata.json

@@ -26,7 +26,7 @@
     "Auto low"
   ],
   "facts": {
-    "headline_passes": 17,
+    "headline_passes": 20,
     "collapsed_passes": 5,
     "collapse_threshold": 0.9,
     "commercial_routers": 7,

runs/router1_2026-10-26/router_board.csv

@@ -22,3 +22,6 @@ or_auto,Auto low,OpenRouter,low,245,65.7,59.2,71.0,0.0005501390004897959,0.00044
 or_auto_medium,Auto medium,OpenRouter,medium,246,73.6,67.1,78.0,0.013130098215934962,0.008043599401597561,0.019080980965126015,0.7817,0,0,20,26.7
 or_auto_high,Auto high,OpenRouter,high,244,79.1,73.4,83.6,0.023005027726557378,0.018076698892231558,0.02858057378719467,0.5794,0,0,21,39.2
 or_auto_max,Auto max,OpenRouter,max,240,78.3,72.9,83.3,0.028915902712999996,0.020327376228814583,0.043068421132057284,0.7817,0,0,22,37.1
+autobench_quality,Quality,AutoBench,quality,245,88.6,83.7,91.8,0.009698,0.00547,0.01602,0.6626984126984127,0,0,23,60.8
+autobench_balanced,Balanced,AutoBench,balanced,245,87.8,83.3,91.8,0.006586,0.00477,0.00881,0.5992063492063492,0,0,24,60.3
+autobench_cost,Cost,AutoBench,cost,242,86.8,83.1,91.3,0.0037,0.00308,0.00438,0.7892561983471075,0,0,25,59.8