Add DeepSeek API + DSH results and independent leaderboard entry

README.md

@@ -6,22 +6,38 @@ colorTo: blue
 sdk: static
 pinned: false
 license: apache-2.0
-short_description: Single-pass agents and DeepAgents on econometric replication
+short_description: Single-pass, DeepAgents and DSH replication results
 ---
 
 # InferenceNet Agent & Harness Leaderboard
 
-The September 2026 research leaderboard displays **six models, twelve groups,
-and 12,000 archived task records**. Switch between a paired comparison, a
-single-pass Agent ranking, and an Agent + Harness ranking. Every group uses
-the same 1,000 Selected_1000 task IDs.
+The September 2026 research leaderboard displays **six requested model aliases,
+thirteen groups, and 13,000 archived task records**. Switch between the original
+baseline / DeepAgents paired comparison, a single-pass Agent ranking, and a
+harness ranking that also includes one independent DSH run. Every group uses
+the same 1,000 Selected_1000 task IDs. An API alias does not establish the
+identity of the underlying model weights.
 
 - **Single-pass Agent:** one model call to generate code, followed by execution.
-- **Agent + Harness:** DeepAgents with up to six model calls and four trial tool
+- **DeepAgents:** up to six model calls and four trial tool
   calls, followed by final execution.
+- **DeepSeek Harness (DSH):** official SDK 0.1.5rc1, requested
+  `deepseek-v4-pro` API alias; displayed separately as
+  **DeepSeek API [v4-pro alias] + DSH**. Its underlying model weights are
+  **unverified**, and it is never used in the baseline / DeepAgents paired lift.
 
 Models: GPT-5.6 Sol, Claude Opus 4.8, Kimi K3, Gemini 3.1 Pro Preview,
-Qwen3.7-Max, and DeepSeek V4 Pro. GPT-5.5 is outside this edition.
+Qwen3.7-Max, and the historical DeepSeek V4 Pro entries. GPT-5.5 is outside this edition.
+
+The DSH [run protocol](./results/deepseek-v4-pro/dsh/README.md) documents its
+fixed selection of 1,000 results: 913 original attempts and 87 technical
+replacement attempts (60 infrastructure failures and 27 input-schema failures).
+The 27 schema recoveries changed the input-schema prompt. All 1,087 attempts
+remain in the cost accounting; selection is not best-of. The final selection
+has 699 scored, 271 failed and 30 uncertain task outcomes. Matching per-attempt
+numeric caps do not establish equal total run budgets: the DSH result includes
+87 additional attempts. Full execution-environment and reasoning-setting
+parity with the prior DeepAgents run has not been verified.
 
 ## Reading the results
 
@@ -43,21 +59,24 @@ See each run's configuration before making cross-model claims.
 
 Kimi retains 5 baseline and 4 harness task-status unknowns. Qwen3.7-Max harness
 retains 6 task-status unknowns and 1 invalid-evidence record (task 453), which is
-not sealed. Historical Sol/Opus interruption counts are shown as originally
+not sealed. DSH retains 30 task-status unknowns, with no invalid-evidence
+records in the fixed selection. Historical Sol/Opus interruption counts are shown as originally
 exported. An archive of 1,000 task IDs does not imply 1,000 definitive outcomes.
 
 ## Data and reproducibility
 
 - [`leaderboard-data.json`](./leaderboard-data.json): displayed data, metric
   profiles, counts, evidence status and source-file SHA-256 hashes.
-- [`leaderboard.csv`](./leaderboard.csv): 60 displayed metric rows; six models ×
-  two arms × five metrics. One metric profile is explicit on every row.
+- [`leaderboard.csv`](./leaderboard.csv): 65 displayed metric rows; thirteen
+  groups × five metrics. One metric profile is explicit on every row.
 - [`results/`](./results): frozen per-task archives, configurations and summaries.
 - [`legacy.html`](./legacy.html): original leaderboard with a historical banner.
   Its original [`results.csv`](./results.csv) is unchanged and is not mixed into
   the new comparison.
 
-Source archive commit: `731111974cf4d1a3c7a4f848a899f0b0708dd12b`.
+The exact source archive commit is recorded as `source_revision` in
+[`leaderboard-data.json`](./leaderboard-data.json); every displayed evidence
+link is pinned to that commit.
 Dataset: [CamoAiLab/InferenceNet](https://huggingface.co/datasets/CamoAiLab/InferenceNet),
 revision `59f9512a38e594528807744214a60ee00367434e`, task list
 `Selected_1000/1000_new.csv`.
@@ -71,10 +90,11 @@ To rebuild the display exports from those archives:
 
 ```sh
 python build_leaderboard.py --source . --output . \
-  --source-revision 731111974cf4d1a3c7a4f848a899f0b0708dd12b
+  --source-revision ARCHIVE_COMMIT_SHA --published-date 2026-09-20
 ```
 
-The builder verifies all task populations and 56 metric counts against archived
+Use the 40-character archive commit SHA from `leaderboard-data.json` above.
+The builder verifies all task populations and 61 metric counts against archived
 flags. The four Sol/Opus local full-replication summaries have no per-task local
 flags in the archive and are preserved from their accepted summaries. This is a
 display refresh, not a fresh inference run, native artifact audit, or rescore.

build_leaderboard.py

@@ -7,6 +7,7 @@ import argparse
 import csv
 import hashlib
 import json
+from datetime import date
 from pathlib import Path
 
 NAMES = {
@@ -18,30 +19,45 @@ NAMES = {
     "deepseek-v4-pro": "DeepSeek V4 Pro",
 }
 METRICS = ("compilation_success", "partial_replication", "coefficient_direction", "significance_level")
+PAIRED_ARMS = ("baseline", "deepagents")
+DSH_ENTRY = ("deepseek-v4-pro", "dsh")
+DSH_NAME = "DeepSeek API [v4-pro alias] + DSH"
+DSH_NOTICE = (
+    "Requested API alias: deepseek-v4-pro. Underlying model weights are unverified. "
+    "This technical recovery result is a separate harness entry, not a paired DeepAgents gain."
+)
 
 
-def build(source, output, revision):
+def build(source, output, revision, published_date=None):
     assert len(revision) == 40 and all(c in "0123456789abcdef" for c in revision)
-    index = json.loads((source / "results/index.json").read_text())
+    index = json.loads((source / "results/index.json").read_text(encoding="utf-8"))
     entries = index["entries"]
-    assert {(e["model"], e["arm"]) for e in entries} == {(m, a) for m in NAMES for a in ("baseline", "deepagents")}
-    assert len(entries) == 12
+    pairs = {(e["model"], e["arm"]) for e in entries}
+    original_pairs = {(m, a) for m in NAMES for a in PAIRED_ARMS}
+    assert original_pairs <= pairs <= original_pairs | {DSH_ENTRY}
+    assert len(entries) == len(pairs), "Duplicate archive entry"
     models = {m: {"id": m, "name": name, "arms": {}} for m, name in NAMES.items()}
     population = None
     dataset_revision = None
     source_files = {}
+    checked_metric_count = accepted_summary_count = records_count = 0
     for entry in entries:
         path = Path(entry["path"])
         assert path == Path("results") / entry["model"] / entry["arm"]
-        summary = json.loads((source / path / "summary.json").read_text())
-        config = json.loads((source / path / "config.json").read_text())
-        records = [json.loads(line) for line in (source / path / "results.jsonl").read_text().splitlines()]
+        summary = json.loads((source / path / "summary.json").read_text(encoding="utf-8"))
+        config = json.loads((source / path / "config.json").read_text(encoding="utf-8"))
+        records = [json.loads(line) for line in (source / path / "results.jsonl").read_text(encoding="utf-8").splitlines()]
         ids = {r["task_id"] for r in records}
         assert len(records) == len(ids) == entry["rows"] == summary["result"]["expected_count"] == 1000
         assert ids == set(config["dataset"]["task_ids"])
         assert all(r["model"] == entry["model"] and r["arm"] == entry["arm"] for r in records)
         assert (summary["model"], summary["arm"]) == (entry["model"], entry["arm"])
         assert summary["official_parity_verified"] is False
+        independent = (entry["model"], entry["arm"]) == DSH_ENTRY
+        if independent:
+            assert config["model_alias"] == "deepseek-v4-pro"
+            assert config["underlying_model_weights_verified"] is False
+            assert summary["evidence_class"] == "technical_recovery_view"
         if population is None:
             population = ids
             dataset_revision = summary["dataset_revision"]
@@ -55,10 +71,15 @@ def build(source, output, revision):
             assert abs(m["rate"] - count / n) < 1e-12
             field = "local_perfect" if local else metric
             has_flags = all(field in r for r in records)
+            if independent:
+                assert has_flags, "Every DSH displayed metric requires an archived task flag"
             if not local or has_flags:
                 assert all(r[field] is None or isinstance(r[field], bool) for r in records)
                 assert sum(r[field] is True for r in records) == count
                 assert sum(r[field] is None for r in records) == unknown
+                checked_metric_count += 1
+            else:
+                accepted_summary_count += 1
             # Sol/Opus local flags were not exported: preserve their accepted
             # summary counts, without claiming a fresh per-task recomputation.
             metrics[metric] = {
@@ -77,29 +98,37 @@ def build(source, output, revision):
             "interruption_count": summary["result"].get("interruption_count"),
             "evidence_class": summary["evidence_class"],
         }
+        if independent:
+            arm.update(display_name=DSH_NAME, identity_notice=DSH_NOTICE, paired_comparison=False,
+                       model_alias=config["model_alias"], underlying_model_weights_verified=False,
+                       comparison_group="independent", protocol_url=(path / "README.md").as_posix())
         models[entry["model"]]["arms"][entry["arm"]] = arm
+        records_count += len(records)
         for name in ("summary.json", "results.jsonl", "config.json"):
             rel = (path / name).as_posix()
             source_files[rel] = hashlib.sha256((source / rel).read_bytes()).hexdigest()
     data = {
         "schema_version": 1, "kind": "research_leaderboard",
-        "published_date": "2026-09-17", "source_repository": "CamoAiLab/InferenceNet-Leaderboard",
+        "published_date": published_date or date.today().isoformat(), "source_repository": "CamoAiLab/InferenceNet-Leaderboard",
         "source_revision": revision, "dataset_revision": dataset_revision,
         "task_list": "Selected_1000/1000_new.csv", "tasks_per_group": 1000,
-        "groups": 12, "records": 12000, "official_parity_verified": False,
+        "groups": len(entries), "records": records_count, "model_aliases": len(models),
+        "paired_groups": len(original_pairs), "independent_groups": len(pairs-original_pairs),
+        "paired_arms": list(PAIRED_ARMS), "official_parity_verified": False,
         "models": list(models.values()), "source_sha256": source_files,
     }
     output.mkdir(parents=True, exist_ok=True)
-    (output / "leaderboard-data.json").write_text(json.dumps(data, indent=2) + "\n")
-    with (output / "leaderboard.csv").open("w", newline="") as f:
+    (output / "leaderboard-data.json").write_text(json.dumps(data, indent=2) + "\n", encoding="utf-8")
+    with (output / "leaderboard.csv").open("w", newline="", encoding="utf-8") as f:
         writer = csv.writer(f)
-        writer.writerow(["model", "arm", "harness", "metric_profile", "metric", "successes", "denominator", "unknown", "score_percent", "official_parity_verified", "source_revision"])
+        writer.writerow(["model", "arm", "harness", "metric_profile", "metric", "successes", "denominator", "unknown", "score_percent", "official_parity_verified", "source_revision", "display_name", "comparison_group", "underlying_model_weights_verified"])
         for model in data["models"]:
             for arm_id, arm in model["arms"].items():
                 for key, metric in arm["metrics"].items():
-                    writer.writerow([model["id"], arm_id, arm["harness"], metric["profile"], key, metric["count"], 1000, metric["unknown"], metric["score"], False, revision])
-    print("Verified: 6 models, 12 arms, 12,000 unique-per-arm records; 60 displayed metric summaries.")
-    print("56 metric summaries checked against task flags; 4 historical local summaries preserved as accepted.")
+                    writer.writerow([model["id"], arm_id, arm["harness"], metric["profile"], key, metric["count"], 1000, metric["unknown"], metric["score"], False, revision, arm.get("display_name", model["name"]), arm.get("comparison_group", "historical_pair"), arm.get("underlying_model_weights_verified", "")])
+    print(f"Verified: {len(models)} requested model aliases, {len(entries)} arms, {records_count:,} records; {len(entries)*5} displayed metric summaries.")
+    print(f"{checked_metric_count} metric summaries checked against task flags; {accepted_summary_count} historical local summaries preserved as accepted.")
+    return data
 
 
 if __name__ == "__main__":
@@ -107,5 +136,6 @@ if __name__ == "__main__":
     parser.add_argument("--source", type=Path, default=Path("."))
     parser.add_argument("--output", type=Path, default=Path("."))
     parser.add_argument("--source-revision", required=True)
+    parser.add_argument("--published-date", help="Publication date, YYYY-MM-DD; defaults to today")
     args = parser.parse_args()
-    build(args.source, args.output, args.source_revision)
+    build(args.source, args.output, args.source_revision, args.published_date)

index.html

@@ -3,7 +3,7 @@
 <head>
   <meta charset="utf-8">
   <meta name="viewport" content="width=device-width, initial-scale=1">
-  <meta name="description" content="InferenceNet research leaderboard: six models compared with single-pass generation and a DeepAgents harness on 1,000 econometric replication tasks.">
+  <meta name="description" content="InferenceNet research leaderboard: single-pass generation, DeepAgents and an independent DeepSeek Harness result on 1,000 econometric replication tasks.">
   <meta name="color-scheme" content="dark">
   <title>InferenceNet · Agent & Harness Leaderboard</title>
   <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='6' fill='%230c1219'/%3E%3Ctext x='5' y='23' font-family='sans-serif' font-size='21' fill='%2378dbc0'%3EiN%3C/text%3E%3C/svg%3E">
@@ -24,30 +24,33 @@
 
     <header class="hero">
       <p class="eyebrow"><span class="dot" aria-hidden="true"></span><span data-i18n="edition">RESEARCH LEADERBOARD · SEPTEMBER 2026</span></p>
-      <h1 data-i18n="title">Same models.<br><span>Two ways to solve.</span></h1>
-      <p class="intro" data-i18n="intro">Compare single-pass agents with a DeepAgents harness on real econometric replication tasks.</p>
+      <h1 data-i18n="title">Models and harnesses.<br><span>Real research, reproduced.</span></h1>
+      <p class="intro" data-i18n="intro">Explore single-pass agents, DeepAgents and a separate DeepSeek Harness result on real econometric replication tasks.</p>
       <div class="facts" aria-label="Dataset coverage">
-        <span><strong id="model-count">—</strong><span data-i18n="models">models</span></span>
+        <span><strong id="model-count">—</strong><span data-i18n="models">requested model aliases</span></span>
         <span><strong id="group-count">—</strong><span data-i18n="groups">experiment groups</span></span>
         <span><strong id="task-count">—</strong><span data-i18n="tasks">tasks per group</span></span>
-        <span class="update" data-i18n="updated">Published 17 Sep 2026</span>
+        <span><strong id="record-count">—</strong><span data-i18n="records">task records</span></span>
+        <span class="update" id="published-date"></span>
       </div>
     </header>
 
     <section class="protocols" aria-label="Experimental settings">
       <div><p class="protocol-tag baseline"><span aria-hidden="true">A</span><strong data-i18n="single">Single-pass Agent</strong></p><p data-i18n="singleDesc">One code-generation call, followed by execution. No iterative agent loop.</p></div>
-      <div><p class="protocol-tag harness"><span aria-hidden="true">B</span><strong data-i18n="harness">Agent + Harness</strong></p><p data-i18n="harnessDesc">DeepAgents plans, uses tools and revises code. Up to 6 model calls, plus final execution.</p></div>
+      <div><p class="protocol-tag harness"><span aria-hidden="true">B</span><strong data-i18n="deepagents">DeepAgents</strong></p><p data-i18n="harnessDesc">DeepAgents plans, uses tools and revises code. Up to 6 model calls, plus final execution.</p></div>
+      <div><p class="protocol-tag independent"><span aria-hidden="true">C</span><strong data-i18n="dsh">DeepSeek Harness · separate run</strong></p><p data-i18n="dshDesc">Official DSH SDK, requested v4-pro API alias. A fixed technical recovery selection; underlying model weights are unverified.</p></div>
     </section>
 
     <section id="rankings" aria-labelledby="rankings-title">
       <div class="section-head"><div><p class="eyebrow" data-i18n="resultsLabel">01 / RESULTS</p><h2 id="rankings-title" data-i18n="rankings">Agent & Harness leaderboard</h2></div><span class="badge" data-i18n="research">Research results · provisional</span></div>
       <p class="scope-note" data-i18n="scope">Locally scored, archived results. Official scorer parity is not yet verified. Different budgets and run protocols mean the lift is descriptive, not an equal-cost comparison.</p>
+      <aside id="independent-notice" class="independent-notice" hidden><strong data-i18n="dshNoticeTitle">DSH is an independent result.</strong><p data-i18n="dshNotice">The request used deepseek-v4-pro; the underlying model weights were not verified. The fixed 1,000-task selection includes 87 technical replacement attempts (27 with changed input-schema prompts), with 30 task outcomes still unknown. It is excluded from the paired DeepAgents lift.</p><a id="dsh-protocol" data-i18n="dshProtocol" target="_blank" rel="noopener noreferrer">Read run protocol & evidence ↗</a></aside>
 
       <div class="toolbar">
         <div class="view-switch" role="group" aria-label="Leaderboard view">
-          <button type="button" data-view="compare" aria-pressed="true" data-i18n="compare">Side by side</button>
+          <button type="button" data-view="compare" aria-pressed="false" data-i18n="compare">Baseline / DeepAgents pairs</button>
           <button type="button" data-view="baseline" aria-pressed="false" data-i18n="single">Single-pass Agent</button>
-          <button type="button" data-view="deepagents" aria-pressed="false" data-i18n="harness">Agent + Harness</button>
+          <button type="button" data-view="harness" aria-pressed="true" data-i18n="harness">All harness runs</button>
         </div>
         <a class="download" href="./leaderboard.csv" download data-i18n="download">Download CSV ↓</a>
       </div>
@@ -63,7 +66,7 @@
             <option value="significance_level" data-i18n="significance_level">Significance level · HF-style</option>
           </optgroup>
         </select></label>
-        <label id="sort-control"><span data-i18n="sort">Sort by</span><select id="sort"><option value="deepagents" data-i18n="harnessScore">Harness score</option><option value="baseline" data-i18n="singleScore">Single-pass score</option><option value="gain" data-i18n="lift">Lift</option></select></label>
+        <label id="sort-control"><span data-i18n="sort">Sort by</span><select id="sort"><option value="deepagents" data-i18n="harnessScore">DeepAgents score</option><option value="baseline" data-i18n="singleScore">Single-pass score</option><option value="gain" data-i18n="lift">Lift</option></select></label>
         <label class="search-control"><span data-i18n="search">Find a model</span><input id="search" type="search" placeholder="Search models…" autocomplete="off"></label>
       </div>
       <p id="metric-description" class="metric-description"></p>
@@ -81,13 +84,13 @@
       <div class="method-grid">
         <article><h3 data-i18n="fixedTasks">Fixed task set</h3><p data-i18n="fixedDesc">The same Selected_1000 task IDs and dataset revision are used for every group. Each score is successes ÷ 1,000; failed or unknown tasks stay in the denominator.</p></article>
         <article><h3 data-i18n="separateMetrics">Two scoring profiles</h3><p data-i18n="profilesDesc">Full replication is the local paper metric. The four HF-style metrics reproduce the published descriptions locally; they are not yet verified against the official scorer.</p></article>
-        <article><h3 data-i18n="budgetTitle">Read lift with the budgets</h3><p data-i18n="budgetDesc">Baseline: 1 model call. DeepAgents: up to 6 model calls and 4 trial tool calls. Recovery amendments, time limits and run protocols differ; these are not equal-cost causal estimates.</p></article>
+        <article><h3 data-i18n="budgetTitle">Read lift with the budgets</h3><p data-i18n="budgetDesc">Baseline: 1 model call. DeepAgents and DSH: up to 6 model calls and 4 trial tool calls per attempt. DSH adds 87 recovery attempts; full environment and reasoning parity is unverified. DSH has no paired lift; these are not equal-cost causal estimates.</p></article>
       </div>
       <details class="evidence"><summary data-i18n="evidence">Evidence status & metric definitions</summary>
         <p data-i18n="unknownDesc">“Metric unknown” means the selected metric cannot be assessed, including failed execution or unavailable outputs. It is different from a task whose final status is unknown. Neither is dropped or counted as a success.</p>
         <ul id="definitions"></ul>
-        <div class="evidence-table-wrap"><table class="evidence-table"><caption data-i18n="evidenceCaption">Status from each accepted archive</caption><thead><tr><th data-i18n="model">Model</th><th data-i18n="single">Single-pass Agent</th><th data-i18n="harness">Agent + Harness</th></tr></thead><tbody id="evidence-body"></tbody></table></div>
-        <p data-i18n="verification">HF-style counts are checked against archived task flags. Full-replication counts for Sol and Opus come from accepted summaries; the other four models also have per-task local flags. This refresh does not rerun or rescore any experiment.</p>
+        <div class="evidence-table-wrap"><table class="evidence-table"><caption data-i18n="evidenceCaption">Status from each accepted archive</caption><thead><tr><th data-i18n="model">Model / API alias</th><th data-i18n="protocol">Protocol</th><th data-i18n="evidenceCol">Evidence</th></tr></thead><tbody id="evidence-body"></tbody></table></div>
+        <p data-i18n="verification">HF-style counts are checked against archived task flags. Full-replication counts for Sol and Opus come from accepted summaries; all other displayed runs, including DSH, have per-task local flags. This refresh does not rerun or rescore any experiment.</p>
         <p data-i18n="archiveNote">Per-run archive files remain frozen, including their original publication metadata. This page is a new display derived from those archives. GPT-5.5 and unarchived experiments are not included in this edition.</p>
       </details>
     </section>

leaderboard-data.json

@@ -1,785 +1,860 @@
-{
-  "schema_version": 1,
-  "kind": "research_leaderboard",
-  "published_date": "2026-09-17",
-  "source_repository": "CamoAiLab/InferenceNet-Leaderboard",
-  "source_revision": "731111974cf4d1a3c7a4f848a899f0b0708dd12b",
-  "dataset_revision": "59f9512a38e594528807744214a60ee00367434e",
-  "task_list": "Selected_1000/1000_new.csv",
-  "tasks_per_group": 1000,
-  "groups": 12,
-  "records": 12000,
-  "official_parity_verified": false,
-  "models": [
-    {
-      "id": "gpt-5.6-sol",
-      "name": "GPT-5.6 Sol",
-      "arms": {
-        "baseline": {
-          "path": "results/gpt-5.6-sol/baseline",
-          "metrics": {
-            "perfect": {
-              "count": 338,
-              "denominator": 1000,
-              "unknown": 445,
-              "failure": 217,
-              "score": 33.8,
-              "profile": "local-paper-v1",
-              "verification": "accepted_summary"
-            },
-            "compilation_success": {
-              "count": 557,
-              "denominator": 1000,
-              "unknown": 3,
-              "failure": 440,
-              "score": 55.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 430,
-              "denominator": 1000,
-              "unknown": 444,
-              "failure": 126,
-              "score": 43.0,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 537,
-              "denominator": 1000,
-              "unknown": 444,
-              "failure": 19,
-              "score": 53.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 485,
-              "denominator": 1000,
-              "unknown": 444,
-              "failure": 71,
-              "score": 48.5,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-13T17:49:16.537733+00:00",
-          "complete": true,
-          "harness": "none",
-          "model_call_cap": 1,
-          "task_unknown_count": null,
-          "invalid_evidence_count": 0,
-          "interruption_count": 3,
-          "evidence_class": "amended_comparison"
-        },
-        "deepagents": {
-          "path": "results/gpt-5.6-sol/deepagents",
-          "metrics": {
-            "perfect": {
-              "count": 518,
-              "denominator": 1000,
-              "unknown": 95,
-              "failure": 387,
-              "score": 51.8,
-              "profile": "local-paper-v1",
-              "verification": "accepted_summary"
-            },
-            "compilation_success": {
-              "count": 910,
-              "denominator": 1000,
-              "unknown": 10,
-              "failure": 80,
-              "score": 91.0,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 699,
-              "denominator": 1000,
-              "unknown": 92,
-              "failure": 209,
-              "score": 69.9,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 879,
-              "denominator": 1000,
-              "unknown": 92,
-              "failure": 29,
-              "score": 87.9,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 787,
-              "denominator": 1000,
-              "unknown": 93,
-              "failure": 120,
-              "score": 78.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-13T17:49:16.537733+00:00",
-          "complete": true,
-          "harness": "DeepAgents",
-          "model_call_cap": 6,
-          "task_unknown_count": null,
-          "invalid_evidence_count": 0,
-          "interruption_count": 10,
-          "evidence_class": "amended_comparison"
-        }
-      }
-    },
-    {
-      "id": "claude-opus-4-8",
-      "name": "Claude Opus 4.8",
-      "arms": {
-        "baseline": {
-          "path": "results/claude-opus-4-8/baseline",
-          "metrics": {
-            "perfect": {
-              "count": 184,
-              "denominator": 1000,
-              "unknown": 538,
-              "failure": 278,
-              "score": 18.4,
-              "profile": "local-paper-v1",
-              "verification": "accepted_summary"
-            },
-            "compilation_success": {
-              "count": 467,
-              "denominator": 1000,
-              "unknown": 0,
-              "failure": 533,
-              "score": 46.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 305,
-              "denominator": 1000,
-              "unknown": 537,
-              "failure": 158,
-              "score": 30.5,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 437,
-              "denominator": 1000,
-              "unknown": 537,
-              "failure": 26,
-              "score": 43.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 370,
-              "denominator": 1000,
-              "unknown": 537,
-              "failure": 93,
-              "score": 37.0,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-13T17:50:43.870921+00:00",
-          "complete": true,
-          "harness": "none",
-          "model_call_cap": 1,
-          "task_unknown_count": null,
-          "invalid_evidence_count": 0,
-          "interruption_count": 0,
-          "evidence_class": "amended_comparison"
-        },
-        "deepagents": {
-          "path": "results/claude-opus-4-8/deepagents",
-          "metrics": {
-            "perfect": {
-              "count": 392,
-              "denominator": 1000,
-              "unknown": 174,
-              "failure": 434,
-              "score": 39.2,
-              "profile": "local-paper-v1",
-              "verification": "accepted_summary"
-            },
-            "compilation_success": {
-              "count": 834,
-              "denominator": 1000,
-              "unknown": 4,
-              "failure": 162,
-              "score": 83.4,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 621,
-              "denominator": 1000,
-              "unknown": 170,
-              "failure": 209,
-              "score": 62.1,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 793,
-              "denominator": 1000,
-              "unknown": 170,
-              "failure": 37,
-              "score": 79.3,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 696,
-              "denominator": 1000,
-              "unknown": 172,
-              "failure": 132,
-              "score": 69.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-13T17:50:43.870921+00:00",
-          "complete": true,
-          "harness": "DeepAgents",
-          "model_call_cap": 6,
-          "task_unknown_count": null,
-          "invalid_evidence_count": 0,
-          "interruption_count": 4,
-          "evidence_class": "amended_comparison"
-        }
-      }
-    },
-    {
-      "id": "kimi-k3",
-      "name": "Kimi K3",
-      "arms": {
-        "baseline": {
-          "path": "results/kimi-k3/baseline",
-          "metrics": {
-            "perfect": {
-              "count": 230,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 289,
-              "score": 23.0,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 526,
-              "denominator": 1000,
-              "unknown": 5,
-              "failure": 469,
-              "score": 52.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 366,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 153,
-              "score": 36.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 494,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 25,
-              "score": 49.4,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 431,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 88,
-              "score": 43.1,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T15:28:00.953066+00:00",
-          "complete": false,
-          "harness": "none",
-          "model_call_cap": 1,
-          "task_unknown_count": 5,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        },
-        "deepagents": {
-          "path": "results/kimi-k3/deepagents",
-          "metrics": {
-            "perfect": {
-              "count": 390,
-              "denominator": 1000,
-              "unknown": 159,
-              "failure": 451,
-              "score": 39.0,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 867,
-              "denominator": 1000,
-              "unknown": 4,
-              "failure": 129,
-              "score": 86.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 635,
-              "denominator": 1000,
-              "unknown": 156,
-              "failure": 209,
-              "score": 63.5,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 804,
-              "denominator": 1000,
-              "unknown": 156,
-              "failure": 40,
-              "score": 80.4,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 709,
-              "denominator": 1000,
-              "unknown": 157,
-              "failure": 134,
-              "score": 70.9,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T15:28:00.953066+00:00",
-          "complete": false,
-          "harness": "DeepAgents 0.7.13",
-          "model_call_cap": 6,
-          "task_unknown_count": 4,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        }
-      }
-    },
-    {
-      "id": "gemini-3.1-pro-preview",
-      "name": "Gemini 3.1 Pro Preview",
-      "arms": {
-        "baseline": {
-          "path": "results/gemini-3.1-pro-preview/baseline",
-          "metrics": {
-            "perfect": {
-              "count": 246,
-              "denominator": 1000,
-              "unknown": 462,
-              "failure": 292,
-              "score": 24.6,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 543,
-              "denominator": 1000,
-              "unknown": 0,
-              "failure": 457,
-              "score": 54.3,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 366,
-              "denominator": 1000,
-              "unknown": 460,
-              "failure": 174,
-              "score": 36.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 506,
-              "denominator": 1000,
-              "unknown": 460,
-              "failure": 34,
-              "score": 50.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 437,
-              "denominator": 1000,
-              "unknown": 460,
-              "failure": 103,
-              "score": 43.7,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T15:28:00.953066+00:00",
-          "complete": true,
-          "harness": "none",
-          "model_call_cap": 1,
-          "task_unknown_count": 0,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        },
-        "deepagents": {
-          "path": "results/gemini-3.1-pro-preview/deepagents",
-          "metrics": {
-            "perfect": {
-              "count": 391,
-              "denominator": 1000,
-              "unknown": 155,
-              "failure": 454,
-              "score": 39.1,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 850,
-              "denominator": 1000,
-              "unknown": 0,
-              "failure": 150,
-              "score": 85.0,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 632,
-              "denominator": 1000,
-              "unknown": 153,
-              "failure": 215,
-              "score": 63.2,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 811,
-              "denominator": 1000,
-              "unknown": 153,
-              "failure": 36,
-              "score": 81.1,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 718,
-              "denominator": 1000,
-              "unknown": 154,
-              "failure": 128,
-              "score": 71.8,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T15:28:00.953066+00:00",
-          "complete": true,
-          "harness": "DeepAgents 0.7.13",
-          "model_call_cap": 6,
-          "task_unknown_count": 0,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        }
-      }
-    },
-    {
-      "id": "qwen3.7-max",
-      "name": "Qwen3.7-Max",
-      "arms": {
-        "baseline": {
-          "path": "results/qwen3.7-max/baseline",
-          "metrics": {
-            "perfect": {
-              "count": 198,
-              "denominator": 1000,
-              "unknown": 519,
-              "failure": 283,
-              "score": 19.8,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 486,
-              "denominator": 1000,
-              "unknown": 0,
-              "failure": 514,
-              "score": 48.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 315,
-              "denominator": 1000,
-              "unknown": 518,
-              "failure": 167,
-              "score": 31.5,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 454,
-              "denominator": 1000,
-              "unknown": 518,
-              "failure": 28,
-              "score": 45.4,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 396,
-              "denominator": 1000,
-              "unknown": 518,
-              "failure": 86,
-              "score": 39.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T17:18:47.373930+00:00",
-          "complete": true,
-          "harness": "none",
-          "model_call_cap": 1,
-          "task_unknown_count": 0,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        },
-        "deepagents": {
-          "path": "results/qwen3.7-max/deepagents",
-          "metrics": {
-            "perfect": {
-              "count": 248,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 271,
-              "score": 24.8,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 523,
-              "denominator": 1000,
-              "unknown": 7,
-              "failure": 470,
-              "score": 52.3,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 379,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 140,
-              "score": 37.9,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 488,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 31,
-              "score": 48.8,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 433,
-              "denominator": 1000,
-              "unknown": 481,
-              "failure": 86,
-              "score": 43.3,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T17:18:47.373930+00:00",
-          "complete": false,
-          "harness": "DeepAgents 0.7.13",
-          "model_call_cap": 6,
-          "task_unknown_count": 6,
-          "invalid_evidence_count": 1,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        }
-      }
-    },
-    {
-      "id": "deepseek-v4-pro",
-      "name": "DeepSeek V4 Pro",
-      "arms": {
-        "baseline": {
-          "path": "results/deepseek-v4-pro/baseline",
-          "metrics": {
-            "perfect": {
-              "count": 129,
-              "denominator": 1000,
-              "unknown": 723,
-              "failure": 148,
-              "score": 12.9,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 281,
-              "denominator": 1000,
-              "unknown": 0,
-              "failure": 719,
-              "score": 28.1,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 195,
-              "denominator": 1000,
-              "unknown": 723,
-              "failure": 82,
-              "score": 19.5,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 260,
-              "denominator": 1000,
-              "unknown": 723,
-              "failure": 17,
-              "score": 26.0,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 236,
-              "denominator": 1000,
-              "unknown": 723,
-              "failure": 41,
-              "score": 23.6,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T17:18:47.373930+00:00",
-          "complete": true,
-          "harness": "none",
-          "model_call_cap": 1,
-          "task_unknown_count": 0,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        },
-        "deepagents": {
-          "path": "results/deepseek-v4-pro/deepagents",
-          "metrics": {
-            "perfect": {
-              "count": 183,
-              "denominator": 1000,
-              "unknown": 648,
-              "failure": 169,
-              "score": 18.3,
-              "profile": "local-paper-v1",
-              "verification": "archived_task_flags"
-            },
-            "compilation_success": {
-              "count": 352,
-              "denominator": 1000,
-              "unknown": 0,
-              "failure": 648,
-              "score": 35.2,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "partial_replication": {
-              "count": 270,
-              "denominator": 1000,
-              "unknown": 648,
-              "failure": 82,
-              "score": 27.0,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "coefficient_direction": {
-              "count": 338,
-              "denominator": 1000,
-              "unknown": 648,
-              "failure": 14,
-              "score": 33.8,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            },
-            "significance_level": {
-              "count": 308,
-              "denominator": 1000,
-              "unknown": 648,
-              "failure": 44,
-              "score": 30.8,
-              "profile": "hf-leaderboard-v1",
-              "verification": "archived_task_flags"
-            }
-          },
-          "generated_at": "2026-09-16T17:18:47.373930+00:00",
-          "complete": true,
-          "harness": "DeepAgents 0.7.13",
-          "model_call_cap": 6,
-          "task_unknown_count": 0,
-          "invalid_evidence_count": 0,
-          "interruption_count": null,
-          "evidence_class": "amended_comparison"
-        }
-      }
-    }
-  ],
-  "source_sha256": {
-    "results/gpt-5.6-sol/baseline/summary.json": "6d6ab1aaffe754e08f9656dcb5f51fcd5c772e60660dc34d86cfc21f3c5c03d7",
-    "results/gpt-5.6-sol/baseline/results.jsonl": "3b12160382043e27531ff5cb3e23bccc9a8f7807e4121a82ffa13df28f65a859",
-    "results/gpt-5.6-sol/baseline/config.json": "90ec4e0781b57498fef52de31eacf5c8ec9424549643865a7140a1ccdb04c1d4",
-    "results/gpt-5.6-sol/deepagents/summary.json": "821ca6e55a66006c7fb958951a7859a9dccde29d87de8415d5c3f07f523563c5",
-    "results/gpt-5.6-sol/deepagents/results.jsonl": "378cdec8e7a880f091ba442649d1c034997acefdc623898b60a063baad29d7be",
-    "results/gpt-5.6-sol/deepagents/config.json": "a3e24175f2395cffbbcd6d9d3c89689bd3a3b22dd4ce61bb78d5da33c6b6cc1b",
-    "results/claude-opus-4-8/baseline/summary.json": "7b5b06bd9b3d7b47952b1dc424a01fcdf04ddccca0b3f64d04132358d6380cdc",
-    "results/claude-opus-4-8/baseline/results.jsonl": "b9d6bd51c9eb81a634e78e422c86da16f655d97da87d3f5afa4c9aa0f96809f4",
-    "results/claude-opus-4-8/baseline/config.json": "8058afcd14c34d666277c0f32ac5962398b10618df5b1aa0e13cd9862344dc56",
-    "results/claude-opus-4-8/deepagents/summary.json": "f86da9c76f8becb75b4970c486160013209a6aa4239923018b217e55f1b0a614",
-    "results/claude-opus-4-8/deepagents/results.jsonl": "3bef3b9850f24b9cb0b5806efd8425c4e4cc650735a6b28b45befdd55a6ec28a",
-    "results/claude-opus-4-8/deepagents/config.json": "12215dfcba5d2d692268a305364514a03ff7579094b2d4af12f493383b8e0f21",
-    "results/kimi-k3/baseline/summary.json": "2edd2e96f3b448ceceb6189edb7f23c737b245c00c9820728c69f5237feb34b7",
-    "results/kimi-k3/baseline/results.jsonl": "0502ae8892dd6c66ee92890a83c678527cff6c3396aa4c13faaadcf41e1857c1",
-    "results/kimi-k3/baseline/config.json": "7191c406e8695a5cb760ee8fa99c4fb0dd71bca59347acc4d6d33fe0630be081",
-    "results/kimi-k3/deepagents/summary.json": "81c8cabaa58acc65ef6afba792241b70951b666ab90c3f7c25194b61b0d9a228",
-    "results/kimi-k3/deepagents/results.jsonl": "bdab99f8f1d81885f74326cf9c8c8e57b9bb8a8d616bfbb1b5a2f6bf32b737ca",
-    "results/kimi-k3/deepagents/config.json": "312c9260c01d5988878e4f2b88da55a16362ee994e8217c63b15ba772da3857c",
-    "results/gemini-3.1-pro-preview/baseline/summary.json": "a8ba5100a28f1f0aeae6949eb3669c7717c844c4e2eee6fa42ded351aef6b5c7",
-    "results/gemini-3.1-pro-preview/baseline/results.jsonl": "2f1e2019157d105efee7ba311a49eba99abe4a7b84f0a4b4e0ac174e2d56cdba",
-    "results/gemini-3.1-pro-preview/baseline/config.json": "ff9884d2095695a1f40f5fc573982b9a49fd53d41dca2a6fc36d04dc81f10998",
-    "results/gemini-3.1-pro-preview/deepagents/summary.json": "bf9ce721c94103f91b0ca576f717f4f5dca7cc9965601658a65fc6d2a15b7045",
-    "results/gemini-3.1-pro-preview/deepagents/results.jsonl": "c2824f8277b283a6dd60118b6e6a0cb0419e4c8b9193de15a7bcf7aaaad24181",
-    "results/gemini-3.1-pro-preview/deepagents/config.json": "f09d9d99d26328ae2f54e9173b22f69d68f7d7bb50c3cc5e352c996b98dd57f9",
-    "results/qwen3.7-max/baseline/summary.json": "e506814d33789673fcd6b8e6869cdcbd52371cd3d0455ce53ae27e1098a52214",
-    "results/qwen3.7-max/baseline/results.jsonl": "4924e929d67956f9deb6c319605042d09f40b12f7bd2f18baa0464b8f7c9851a",
-    "results/qwen3.7-max/baseline/config.json": "9f553440e741863e82f32fec11470affe6f0dbffa03c668c647bd63d8f73227f",
-    "results/qwen3.7-max/deepagents/summary.json": "a5dcb2297ccc31a921e3f4eeb025ed81a764e16556a12068733cfe3bfe0bdbe3",
-    "results/qwen3.7-max/deepagents/results.jsonl": "e009f8c5c88b05896836330822afbda33af463c1f264f9adb9bbf5b094e61dfa",
-    "results/qwen3.7-max/deepagents/config.json": "3862d438d381ca3e57b1bf68921ef01a8c38727902152ada2c6e1fcf7b70cc77",
-    "results/deepseek-v4-pro/baseline/summary.json": "c51e4f1de78eba8f30e4a59276fb7821132cd103f0ae3f9d04c75c98344f2e34",
-    "results/deepseek-v4-pro/baseline/results.jsonl": "7827218f6f18f39988e8c439f0c93cab611c04d158d8704aa50c9274cd438837",
-    "results/deepseek-v4-pro/baseline/config.json": "4cef524f37a325029ea4ad325b163f72f9cf725f507f20cd76424d06e71e24ad",
-    "results/deepseek-v4-pro/deepagents/summary.json": "231e4264b0403e12788136d01af465f86fe9ba3c3ab992510925a7ef4c2a40e9",
-    "results/deepseek-v4-pro/deepagents/results.jsonl": "a006c8b0b80064295faeb3aa7395a2408f0208677853448816b361bf67e39bb6",
-    "results/deepseek-v4-pro/deepagents/config.json": "400c28e4008d5241bf50bc71c2674861846d6b6d2e7da21f2a7602a60ecc0ae9"
-  }
-}
+{
+  "schema_version": 1,
+  "kind": "research_leaderboard",
+  "published_date": "2026-09-20",
+  "source_repository": "CamoAiLab/InferenceNet-Leaderboard",
+  "source_revision": "6ea7c5684664bfe5c6e3d282a5e28f68244acd9e",
+  "dataset_revision": "59f9512a38e594528807744214a60ee00367434e",
+  "task_list": "Selected_1000/1000_new.csv",
+  "tasks_per_group": 1000,
+  "groups": 13,
+  "records": 13000,
+  "model_aliases": 6,
+  "paired_groups": 12,
+  "independent_groups": 1,
+  "paired_arms": [
+    "baseline",
+    "deepagents"
+  ],
+  "official_parity_verified": false,
+  "models": [
+    {
+      "id": "gpt-5.6-sol",
+      "name": "GPT-5.6 Sol",
+      "arms": {
+        "baseline": {
+          "path": "results/gpt-5.6-sol/baseline",
+          "metrics": {
+            "perfect": {
+              "count": 338,
+              "denominator": 1000,
+              "unknown": 445,
+              "failure": 217,
+              "score": 33.8,
+              "profile": "local-paper-v1",
+              "verification": "accepted_summary"
+            },
+            "compilation_success": {
+              "count": 557,
+              "denominator": 1000,
+              "unknown": 3,
+              "failure": 440,
+              "score": 55.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 430,
+              "denominator": 1000,
+              "unknown": 444,
+              "failure": 126,
+              "score": 43.0,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 537,
+              "denominator": 1000,
+              "unknown": 444,
+              "failure": 19,
+              "score": 53.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 485,
+              "denominator": 1000,
+              "unknown": 444,
+              "failure": 71,
+              "score": 48.5,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-13T17:49:16.537733+00:00",
+          "complete": true,
+          "harness": "none",
+          "model_call_cap": 1,
+          "task_unknown_count": null,
+          "invalid_evidence_count": 0,
+          "interruption_count": 3,
+          "evidence_class": "amended_comparison"
+        },
+        "deepagents": {
+          "path": "results/gpt-5.6-sol/deepagents",
+          "metrics": {
+            "perfect": {
+              "count": 518,
+              "denominator": 1000,
+              "unknown": 95,
+              "failure": 387,
+              "score": 51.8,
+              "profile": "local-paper-v1",
+              "verification": "accepted_summary"
+            },
+            "compilation_success": {
+              "count": 910,
+              "denominator": 1000,
+              "unknown": 10,
+              "failure": 80,
+              "score": 91.0,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 699,
+              "denominator": 1000,
+              "unknown": 92,
+              "failure": 209,
+              "score": 69.9,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 879,
+              "denominator": 1000,
+              "unknown": 92,
+              "failure": 29,
+              "score": 87.9,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 787,
+              "denominator": 1000,
+              "unknown": 93,
+              "failure": 120,
+              "score": 78.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-13T17:49:16.537733+00:00",
+          "complete": true,
+          "harness": "DeepAgents",
+          "model_call_cap": 6,
+          "task_unknown_count": null,
+          "invalid_evidence_count": 0,
+          "interruption_count": 10,
+          "evidence_class": "amended_comparison"
+        }
+      }
+    },
+    {
+      "id": "claude-opus-4-8",
+      "name": "Claude Opus 4.8",
+      "arms": {
+        "baseline": {
+          "path": "results/claude-opus-4-8/baseline",
+          "metrics": {
+            "perfect": {
+              "count": 184,
+              "denominator": 1000,
+              "unknown": 538,
+              "failure": 278,
+              "score": 18.4,
+              "profile": "local-paper-v1",
+              "verification": "accepted_summary"
+            },
+            "compilation_success": {
+              "count": 467,
+              "denominator": 1000,
+              "unknown": 0,
+              "failure": 533,
+              "score": 46.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 305,
+              "denominator": 1000,
+              "unknown": 537,
+              "failure": 158,
+              "score": 30.5,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 437,
+              "denominator": 1000,
+              "unknown": 537,
+              "failure": 26,
+              "score": 43.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 370,
+              "denominator": 1000,
+              "unknown": 537,
+              "failure": 93,
+              "score": 37.0,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-13T17:50:43.870921+00:00",
+          "complete": true,
+          "harness": "none",
+          "model_call_cap": 1,
+          "task_unknown_count": null,
+          "invalid_evidence_count": 0,
+          "interruption_count": 0,
+          "evidence_class": "amended_comparison"
+        },
+        "deepagents": {
+          "path": "results/claude-opus-4-8/deepagents",
+          "metrics": {
+            "perfect": {
+              "count": 392,
+              "denominator": 1000,
+              "unknown": 174,
+              "failure": 434,
+              "score": 39.2,
+              "profile": "local-paper-v1",
+              "verification": "accepted_summary"
+            },
+            "compilation_success": {
+              "count": 834,
+              "denominator": 1000,
+              "unknown": 4,
+              "failure": 162,
+              "score": 83.4,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 621,
+              "denominator": 1000,
+              "unknown": 170,
+              "failure": 209,
+              "score": 62.1,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 793,
+              "denominator": 1000,
+              "unknown": 170,
+              "failure": 37,
+              "score": 79.3,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 696,
+              "denominator": 1000,
+              "unknown": 172,
+              "failure": 132,
+              "score": 69.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-13T17:50:43.870921+00:00",
+          "complete": true,
+          "harness": "DeepAgents",
+          "model_call_cap": 6,
+          "task_unknown_count": null,
+          "invalid_evidence_count": 0,
+          "interruption_count": 4,
+          "evidence_class": "amended_comparison"
+        }
+      }
+    },
+    {
+      "id": "kimi-k3",
+      "name": "Kimi K3",
+      "arms": {
+        "baseline": {
+          "path": "results/kimi-k3/baseline",
+          "metrics": {
+            "perfect": {
+              "count": 230,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 289,
+              "score": 23.0,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 526,
+              "denominator": 1000,
+              "unknown": 5,
+              "failure": 469,
+              "score": 52.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 366,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 153,
+              "score": 36.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 494,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 25,
+              "score": 49.4,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 431,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 88,
+              "score": 43.1,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T15:28:00.953066+00:00",
+          "complete": false,
+          "harness": "none",
+          "model_call_cap": 1,
+          "task_unknown_count": 5,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        },
+        "deepagents": {
+          "path": "results/kimi-k3/deepagents",
+          "metrics": {
+            "perfect": {
+              "count": 390,
+              "denominator": 1000,
+              "unknown": 159,
+              "failure": 451,
+              "score": 39.0,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 867,
+              "denominator": 1000,
+              "unknown": 4,
+              "failure": 129,
+              "score": 86.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 635,
+              "denominator": 1000,
+              "unknown": 156,
+              "failure": 209,
+              "score": 63.5,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 804,
+              "denominator": 1000,
+              "unknown": 156,
+              "failure": 40,
+              "score": 80.4,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 709,
+              "denominator": 1000,
+              "unknown": 157,
+              "failure": 134,
+              "score": 70.9,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T15:28:00.953066+00:00",
+          "complete": false,
+          "harness": "DeepAgents 0.7.13",
+          "model_call_cap": 6,
+          "task_unknown_count": 4,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        }
+      }
+    },
+    {
+      "id": "gemini-3.1-pro-preview",
+      "name": "Gemini 3.1 Pro Preview",
+      "arms": {
+        "baseline": {
+          "path": "results/gemini-3.1-pro-preview/baseline",
+          "metrics": {
+            "perfect": {
+              "count": 246,
+              "denominator": 1000,
+              "unknown": 462,
+              "failure": 292,
+              "score": 24.6,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 543,
+              "denominator": 1000,
+              "unknown": 0,
+              "failure": 457,
+              "score": 54.3,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 366,
+              "denominator": 1000,
+              "unknown": 460,
+              "failure": 174,
+              "score": 36.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 506,
+              "denominator": 1000,
+              "unknown": 460,
+              "failure": 34,
+              "score": 50.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 437,
+              "denominator": 1000,
+              "unknown": 460,
+              "failure": 103,
+              "score": 43.7,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T15:28:00.953066+00:00",
+          "complete": true,
+          "harness": "none",
+          "model_call_cap": 1,
+          "task_unknown_count": 0,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        },
+        "deepagents": {
+          "path": "results/gemini-3.1-pro-preview/deepagents",
+          "metrics": {
+            "perfect": {
+              "count": 391,
+              "denominator": 1000,
+              "unknown": 155,
+              "failure": 454,
+              "score": 39.1,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 850,
+              "denominator": 1000,
+              "unknown": 0,
+              "failure": 150,
+              "score": 85.0,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 632,
+              "denominator": 1000,
+              "unknown": 153,
+              "failure": 215,
+              "score": 63.2,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 811,
+              "denominator": 1000,
+              "unknown": 153,
+              "failure": 36,
+              "score": 81.1,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 718,
+              "denominator": 1000,
+              "unknown": 154,
+              "failure": 128,
+              "score": 71.8,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T15:28:00.953066+00:00",
+          "complete": true,
+          "harness": "DeepAgents 0.7.13",
+          "model_call_cap": 6,
+          "task_unknown_count": 0,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        }
+      }
+    },
+    {
+      "id": "qwen3.7-max",
+      "name": "Qwen3.7-Max",
+      "arms": {
+        "baseline": {
+          "path": "results/qwen3.7-max/baseline",
+          "metrics": {
+            "perfect": {
+              "count": 198,
+              "denominator": 1000,
+              "unknown": 519,
+              "failure": 283,
+              "score": 19.8,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 486,
+              "denominator": 1000,
+              "unknown": 0,
+              "failure": 514,
+              "score": 48.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 315,
+              "denominator": 1000,
+              "unknown": 518,
+              "failure": 167,
+              "score": 31.5,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 454,
+              "denominator": 1000,
+              "unknown": 518,
+              "failure": 28,
+              "score": 45.4,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 396,
+              "denominator": 1000,
+              "unknown": 518,
+              "failure": 86,
+              "score": 39.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T17:18:47.373930+00:00",
+          "complete": true,
+          "harness": "none",
+          "model_call_cap": 1,
+          "task_unknown_count": 0,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        },
+        "deepagents": {
+          "path": "results/qwen3.7-max/deepagents",
+          "metrics": {
+            "perfect": {
+              "count": 248,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 271,
+              "score": 24.8,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 523,
+              "denominator": 1000,
+              "unknown": 7,
+              "failure": 470,
+              "score": 52.3,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 379,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 140,
+              "score": 37.9,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 488,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 31,
+              "score": 48.8,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 433,
+              "denominator": 1000,
+              "unknown": 481,
+              "failure": 86,
+              "score": 43.3,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T17:18:47.373930+00:00",
+          "complete": false,
+          "harness": "DeepAgents 0.7.13",
+          "model_call_cap": 6,
+          "task_unknown_count": 6,
+          "invalid_evidence_count": 1,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        }
+      }
+    },
+    {
+      "id": "deepseek-v4-pro",
+      "name": "DeepSeek V4 Pro",
+      "arms": {
+        "baseline": {
+          "path": "results/deepseek-v4-pro/baseline",
+          "metrics": {
+            "perfect": {
+              "count": 129,
+              "denominator": 1000,
+              "unknown": 723,
+              "failure": 148,
+              "score": 12.9,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 281,
+              "denominator": 1000,
+              "unknown": 0,
+              "failure": 719,
+              "score": 28.1,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 195,
+              "denominator": 1000,
+              "unknown": 723,
+              "failure": 82,
+              "score": 19.5,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 260,
+              "denominator": 1000,
+              "unknown": 723,
+              "failure": 17,
+              "score": 26.0,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 236,
+              "denominator": 1000,
+              "unknown": 723,
+              "failure": 41,
+              "score": 23.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T17:18:47.373930+00:00",
+          "complete": true,
+          "harness": "none",
+          "model_call_cap": 1,
+          "task_unknown_count": 0,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        },
+        "deepagents": {
+          "path": "results/deepseek-v4-pro/deepagents",
+          "metrics": {
+            "perfect": {
+              "count": 183,
+              "denominator": 1000,
+              "unknown": 648,
+              "failure": 169,
+              "score": 18.3,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 352,
+              "denominator": 1000,
+              "unknown": 0,
+              "failure": 648,
+              "score": 35.2,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 270,
+              "denominator": 1000,
+              "unknown": 648,
+              "failure": 82,
+              "score": 27.0,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 338,
+              "denominator": 1000,
+              "unknown": 648,
+              "failure": 14,
+              "score": 33.8,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 308,
+              "denominator": 1000,
+              "unknown": 648,
+              "failure": 44,
+              "score": 30.8,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-16T17:18:47.373930+00:00",
+          "complete": true,
+          "harness": "DeepAgents 0.7.13",
+          "model_call_cap": 6,
+          "task_unknown_count": 0,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "amended_comparison"
+        },
+        "dsh": {
+          "path": "results/deepseek-v4-pro/dsh",
+          "metrics": {
+            "perfect": {
+              "count": 372,
+              "denominator": 1000,
+              "unknown": 305,
+              "failure": 323,
+              "score": 37.2,
+              "profile": "local-paper-v1",
+              "verification": "archived_task_flags"
+            },
+            "compilation_success": {
+              "count": 699,
+              "denominator": 1000,
+              "unknown": 30,
+              "failure": 271,
+              "score": 69.9,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "partial_replication": {
+              "count": 556,
+              "denominator": 1000,
+              "unknown": 302,
+              "failure": 142,
+              "score": 55.6,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "coefficient_direction": {
+              "count": 683,
+              "denominator": 1000,
+              "unknown": 302,
+              "failure": 15,
+              "score": 68.3,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            },
+            "significance_level": {
+              "count": 598,
+              "denominator": 1000,
+              "unknown": 304,
+              "failure": 98,
+              "score": 59.8,
+              "profile": "hf-leaderboard-v1",
+              "verification": "archived_task_flags"
+            }
+          },
+          "generated_at": "2026-09-19T18:18:48.070003+00:00",
+          "complete": false,
+          "harness": "DeepSeek Harness SDK 0.1.5rc1",
+          "model_call_cap": 6,
+          "task_unknown_count": 30,
+          "invalid_evidence_count": 0,
+          "interruption_count": null,
+          "evidence_class": "technical_recovery_view",
+          "display_name": "DeepSeek API [v4-pro alias] + DSH",
+          "identity_notice": "Requested API alias: deepseek-v4-pro. Underlying model weights are unverified. This technical recovery result is a separate harness entry, not a paired DeepAgents gain.",
+          "paired_comparison": false,
+          "model_alias": "deepseek-v4-pro",
+          "underlying_model_weights_verified": false,
+          "comparison_group": "independent",
+          "protocol_url": "results/deepseek-v4-pro/dsh/README.md"
+        }
+      }
+    }
+  ],
+  "source_sha256": {
+    "results/gpt-5.6-sol/baseline/summary.json": "6d6ab1aaffe754e08f9656dcb5f51fcd5c772e60660dc34d86cfc21f3c5c03d7",
+    "results/gpt-5.6-sol/baseline/results.jsonl": "3b12160382043e27531ff5cb3e23bccc9a8f7807e4121a82ffa13df28f65a859",
+    "results/gpt-5.6-sol/baseline/config.json": "90ec4e0781b57498fef52de31eacf5c8ec9424549643865a7140a1ccdb04c1d4",
+    "results/gpt-5.6-sol/deepagents/summary.json": "821ca6e55a66006c7fb958951a7859a9dccde29d87de8415d5c3f07f523563c5",
+    "results/gpt-5.6-sol/deepagents/results.jsonl": "378cdec8e7a880f091ba442649d1c034997acefdc623898b60a063baad29d7be",
+    "results/gpt-5.6-sol/deepagents/config.json": "a3e24175f2395cffbbcd6d9d3c89689bd3a3b22dd4ce61bb78d5da33c6b6cc1b",
+    "results/claude-opus-4-8/baseline/summary.json": "7b5b06bd9b3d7b47952b1dc424a01fcdf04ddccca0b3f64d04132358d6380cdc",
+    "results/claude-opus-4-8/baseline/results.jsonl": "b9d6bd51c9eb81a634e78e422c86da16f655d97da87d3f5afa4c9aa0f96809f4",
+    "results/claude-opus-4-8/baseline/config.json": "8058afcd14c34d666277c0f32ac5962398b10618df5b1aa0e13cd9862344dc56",
+    "results/claude-opus-4-8/deepagents/summary.json": "f86da9c76f8becb75b4970c486160013209a6aa4239923018b217e55f1b0a614",
+    "results/claude-opus-4-8/deepagents/results.jsonl": "3bef3b9850f24b9cb0b5806efd8425c4e4cc650735a6b28b45befdd55a6ec28a",
+    "results/claude-opus-4-8/deepagents/config.json": "12215dfcba5d2d692268a305364514a03ff7579094b2d4af12f493383b8e0f21",
+    "results/kimi-k3/baseline/summary.json": "2edd2e96f3b448ceceb6189edb7f23c737b245c00c9820728c69f5237feb34b7",
+    "results/kimi-k3/baseline/results.jsonl": "0502ae8892dd6c66ee92890a83c678527cff6c3396aa4c13faaadcf41e1857c1",
+    "results/kimi-k3/baseline/config.json": "7191c406e8695a5cb760ee8fa99c4fb0dd71bca59347acc4d6d33fe0630be081",
+    "results/kimi-k3/deepagents/summary.json": "81c8cabaa58acc65ef6afba792241b70951b666ab90c3f7c25194b61b0d9a228",
+    "results/kimi-k3/deepagents/results.jsonl": "bdab99f8f1d81885f74326cf9c8c8e57b9bb8a8d616bfbb1b5a2f6bf32b737ca",
+    "results/kimi-k3/deepagents/config.json": "312c9260c01d5988878e4f2b88da55a16362ee994e8217c63b15ba772da3857c",
+    "results/gemini-3.1-pro-preview/baseline/summary.json": "a8ba5100a28f1f0aeae6949eb3669c7717c844c4e2eee6fa42ded351aef6b5c7",
+    "results/gemini-3.1-pro-preview/baseline/results.jsonl": "2f1e2019157d105efee7ba311a49eba99abe4a7b84f0a4b4e0ac174e2d56cdba",
+    "results/gemini-3.1-pro-preview/baseline/config.json": "ff9884d2095695a1f40f5fc573982b9a49fd53d41dca2a6fc36d04dc81f10998",
+    "results/gemini-3.1-pro-preview/deepagents/summary.json": "bf9ce721c94103f91b0ca576f717f4f5dca7cc9965601658a65fc6d2a15b7045",
+    "results/gemini-3.1-pro-preview/deepagents/results.jsonl": "c2824f8277b283a6dd60118b6e6a0cb0419e4c8b9193de15a7bcf7aaaad24181",
+    "results/gemini-3.1-pro-preview/deepagents/config.json": "f09d9d99d26328ae2f54e9173b22f69d68f7d7bb50c3cc5e352c996b98dd57f9",
+    "results/qwen3.7-max/baseline/summary.json": "e506814d33789673fcd6b8e6869cdcbd52371cd3d0455ce53ae27e1098a52214",
+    "results/qwen3.7-max/baseline/results.jsonl": "4924e929d67956f9deb6c319605042d09f40b12f7bd2f18baa0464b8f7c9851a",
+    "results/qwen3.7-max/baseline/config.json": "9f553440e741863e82f32fec11470affe6f0dbffa03c668c647bd63d8f73227f",
+    "results/qwen3.7-max/deepagents/summary.json": "a5dcb2297ccc31a921e3f4eeb025ed81a764e16556a12068733cfe3bfe0bdbe3",
+    "results/qwen3.7-max/deepagents/results.jsonl": "e009f8c5c88b05896836330822afbda33af463c1f264f9adb9bbf5b094e61dfa",
+    "results/qwen3.7-max/deepagents/config.json": "3862d438d381ca3e57b1bf68921ef01a8c38727902152ada2c6e1fcf7b70cc77",
+    "results/deepseek-v4-pro/baseline/summary.json": "c51e4f1de78eba8f30e4a59276fb7821132cd103f0ae3f9d04c75c98344f2e34",
+    "results/deepseek-v4-pro/baseline/results.jsonl": "7827218f6f18f39988e8c439f0c93cab611c04d158d8704aa50c9274cd438837",
+    "results/deepseek-v4-pro/baseline/config.json": "4cef524f37a325029ea4ad325b163f72f9cf725f507f20cd76424d06e71e24ad",
+    "results/deepseek-v4-pro/deepagents/summary.json": "231e4264b0403e12788136d01af465f86fe9ba3c3ab992510925a7ef4c2a40e9",
+    "results/deepseek-v4-pro/deepagents/results.jsonl": "a006c8b0b80064295faeb3aa7395a2408f0208677853448816b361bf67e39bb6",
+    "results/deepseek-v4-pro/deepagents/config.json": "400c28e4008d5241bf50bc71c2674861846d6b6d2e7da21f2a7602a60ecc0ae9",
+    "results/deepseek-v4-pro/dsh/summary.json": "62936f76b1ca0de5e761d1ea1f894343695503ce934bf0213dc9fde1ea72a782",
+    "results/deepseek-v4-pro/dsh/results.jsonl": "b12427d5f4476351e571e4b29f61e0463540a1701bec51e642753d717dbc1df2",
+    "results/deepseek-v4-pro/dsh/config.json": "df322f13761e06dc78cb553c5dac347ce4361599937af4017a902c56a7df9d77"
+  }
+}

leaderboard.css

@@ -3,3 +3,5 @@
 @media(max-width:760px){.page{padding:0 20px}.topbar{min-height:76px}.nav-links{gap:12px}.nav-links a{font-size:11px}.brand{font-size:18px}.brand-mark{display:none}.hero{padding-top:34px}h1{letter-spacing:-1.3px}.intro{font-size:15px}.facts{gap:18px}.facts .update{width:100%;margin-left:0}.protocols{gap:18px}.protocols>div+div{padding-left:18px}.protocols p+p{font-size:12px}.protocol-tag{font-size:12px}h2{font-size:22px}.section-head{align-items:flex-start;flex-direction:column;gap:12px}.toolbar{align-items:flex-start;flex-direction:column}.view-switch{width:100%}.view-switch button{flex:1;font-size:11px;padding:8px}.filters{flex-wrap:wrap;gap:12px}.metric-control{width:100%;min-width:0!important}.filters label:not(.metric-control){flex:1}.search-control{margin-left:0;width:auto}.metric-description{padding-bottom:16px}.method-grid{grid-template-columns:1fr;gap:20px}.method{padding-top:36px}.table-foot{flex-direction:column;gap:5px}.table-foot>span:last-child{text-align:left}.table-wrap td{padding:17px 14px}.method .section-head{gap:14px}footer{flex-direction:column;gap:12px}}
 tbody th[scope=row]{background:transparent;color:var(--text);border-bottom-color:#23313b}tbody tr:last-child th{border-bottom:none}
 @media(prefers-reduced-motion:no-preference){button,a{transition:background .12s,color .12s}}
+.protocols{grid-template-columns:repeat(3,1fr);gap:24px}.protocols>div+div{padding-left:24px}.independent{color:var(--amber)}.independent .fill{background:var(--amber)}.independent-notice{margin-top:18px;padding:16px 20px;border:1px solid #4b422e;border-left:3px solid var(--amber);border-radius:6px;background:#1c1e1b;font-size:12px;color:var(--muted)}.independent-notice strong{color:var(--amber)}.independent-notice p{margin:5px 0 7px;max-width:960px}.independent-notice a{color:var(--amber)}.run-protocol{display:block;font-size:10px;margin-top:5px}.model-name{white-space:normal;max-width:355px}.model-meta{flex-wrap:wrap}.facts{gap:24px}.facts .update{font-size:11px}
+@media(max-width:760px){.protocols{grid-template-columns:1fr;gap:18px}.protocols>div+div{border-left:0;border-top:1px solid var(--line);padding:18px 0 0}.facts{gap:14px 24px}.independent-notice{padding:14px}.view-switch button{flex-basis:100%}.model-cell{min-width:225px}.model-name{max-width:275px}}

leaderboard.csv

@@ -1,61 +1,66 @@
-model,arm,harness,metric_profile,metric,successes,denominator,unknown,score_percent,official_parity_verified,source_revision
-gpt-5.6-sol,baseline,none,local-paper-v1,perfect,338,1000,445,33.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,baseline,none,hf-leaderboard-v1,compilation_success,557,1000,3,55.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,baseline,none,hf-leaderboard-v1,partial_replication,430,1000,444,43.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,baseline,none,hf-leaderboard-v1,coefficient_direction,537,1000,444,53.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,baseline,none,hf-leaderboard-v1,significance_level,485,1000,444,48.5,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,deepagents,DeepAgents,local-paper-v1,perfect,518,1000,95,51.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,compilation_success,910,1000,10,91.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,partial_replication,699,1000,92,69.9,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,coefficient_direction,879,1000,92,87.9,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,significance_level,787,1000,93,78.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,baseline,none,local-paper-v1,perfect,184,1000,538,18.4,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,baseline,none,hf-leaderboard-v1,compilation_success,467,1000,0,46.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,baseline,none,hf-leaderboard-v1,partial_replication,305,1000,537,30.5,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,baseline,none,hf-leaderboard-v1,coefficient_direction,437,1000,537,43.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,baseline,none,hf-leaderboard-v1,significance_level,370,1000,537,37.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,deepagents,DeepAgents,local-paper-v1,perfect,392,1000,174,39.2,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,compilation_success,834,1000,4,83.4,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,partial_replication,621,1000,170,62.1,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,coefficient_direction,793,1000,170,79.3,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,significance_level,696,1000,172,69.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,baseline,none,local-paper-v1,perfect,230,1000,481,23.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,baseline,none,hf-leaderboard-v1,compilation_success,526,1000,5,52.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,baseline,none,hf-leaderboard-v1,partial_replication,366,1000,481,36.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,baseline,none,hf-leaderboard-v1,coefficient_direction,494,1000,481,49.4,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,baseline,none,hf-leaderboard-v1,significance_level,431,1000,481,43.1,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,390,1000,159,39.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,867,1000,4,86.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,635,1000,156,63.5,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,804,1000,156,80.4,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,709,1000,157,70.9,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,baseline,none,local-paper-v1,perfect,246,1000,462,24.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,compilation_success,543,1000,0,54.3,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,partial_replication,366,1000,460,36.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,coefficient_direction,506,1000,460,50.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,significance_level,437,1000,460,43.7,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,391,1000,155,39.1,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,850,1000,0,85.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,632,1000,153,63.2,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,811,1000,153,81.1,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,718,1000,154,71.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,baseline,none,local-paper-v1,perfect,198,1000,519,19.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,baseline,none,hf-leaderboard-v1,compilation_success,486,1000,0,48.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,baseline,none,hf-leaderboard-v1,partial_replication,315,1000,518,31.5,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,baseline,none,hf-leaderboard-v1,coefficient_direction,454,1000,518,45.4,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,baseline,none,hf-leaderboard-v1,significance_level,396,1000,518,39.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,248,1000,481,24.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,523,1000,7,52.3,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,379,1000,481,37.9,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,488,1000,481,48.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,433,1000,481,43.3,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,baseline,none,local-paper-v1,perfect,129,1000,723,12.9,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,baseline,none,hf-leaderboard-v1,compilation_success,281,1000,0,28.1,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,baseline,none,hf-leaderboard-v1,partial_replication,195,1000,723,19.5,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,baseline,none,hf-leaderboard-v1,coefficient_direction,260,1000,723,26.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,baseline,none,hf-leaderboard-v1,significance_level,236,1000,723,23.6,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,183,1000,648,18.3,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,352,1000,0,35.2,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,270,1000,648,27.0,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,338,1000,648,33.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
-deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,308,1000,648,30.8,False,731111974cf4d1a3c7a4f848a899f0b0708dd12b
+model,arm,harness,metric_profile,metric,successes,denominator,unknown,score_percent,official_parity_verified,source_revision,display_name,comparison_group,underlying_model_weights_verified
+gpt-5.6-sol,baseline,none,local-paper-v1,perfect,338,1000,445,33.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,baseline,none,hf-leaderboard-v1,compilation_success,557,1000,3,55.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,baseline,none,hf-leaderboard-v1,partial_replication,430,1000,444,43.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,baseline,none,hf-leaderboard-v1,coefficient_direction,537,1000,444,53.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,baseline,none,hf-leaderboard-v1,significance_level,485,1000,444,48.5,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,deepagents,DeepAgents,local-paper-v1,perfect,518,1000,95,51.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,compilation_success,910,1000,10,91.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,partial_replication,699,1000,92,69.9,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,coefficient_direction,879,1000,92,87.9,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+gpt-5.6-sol,deepagents,DeepAgents,hf-leaderboard-v1,significance_level,787,1000,93,78.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,GPT-5.6 Sol,historical_pair,
+claude-opus-4-8,baseline,none,local-paper-v1,perfect,184,1000,538,18.4,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,baseline,none,hf-leaderboard-v1,compilation_success,467,1000,0,46.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,baseline,none,hf-leaderboard-v1,partial_replication,305,1000,537,30.5,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,baseline,none,hf-leaderboard-v1,coefficient_direction,437,1000,537,43.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,baseline,none,hf-leaderboard-v1,significance_level,370,1000,537,37.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,deepagents,DeepAgents,local-paper-v1,perfect,392,1000,174,39.2,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,compilation_success,834,1000,4,83.4,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,partial_replication,621,1000,170,62.1,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,coefficient_direction,793,1000,170,79.3,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+claude-opus-4-8,deepagents,DeepAgents,hf-leaderboard-v1,significance_level,696,1000,172,69.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Claude Opus 4.8,historical_pair,
+kimi-k3,baseline,none,local-paper-v1,perfect,230,1000,481,23.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,baseline,none,hf-leaderboard-v1,compilation_success,526,1000,5,52.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,baseline,none,hf-leaderboard-v1,partial_replication,366,1000,481,36.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,baseline,none,hf-leaderboard-v1,coefficient_direction,494,1000,481,49.4,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,baseline,none,hf-leaderboard-v1,significance_level,431,1000,481,43.1,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,390,1000,159,39.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,867,1000,4,86.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,635,1000,156,63.5,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,804,1000,156,80.4,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+kimi-k3,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,709,1000,157,70.9,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Kimi K3,historical_pair,
+gemini-3.1-pro-preview,baseline,none,local-paper-v1,perfect,246,1000,462,24.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,compilation_success,543,1000,0,54.3,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,partial_replication,366,1000,460,36.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,coefficient_direction,506,1000,460,50.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,baseline,none,hf-leaderboard-v1,significance_level,437,1000,460,43.7,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,391,1000,155,39.1,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,850,1000,0,85.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,632,1000,153,63.2,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,811,1000,153,81.1,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+gemini-3.1-pro-preview,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,718,1000,154,71.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Gemini 3.1 Pro Preview,historical_pair,
+qwen3.7-max,baseline,none,local-paper-v1,perfect,198,1000,519,19.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,baseline,none,hf-leaderboard-v1,compilation_success,486,1000,0,48.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,baseline,none,hf-leaderboard-v1,partial_replication,315,1000,518,31.5,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,baseline,none,hf-leaderboard-v1,coefficient_direction,454,1000,518,45.4,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,baseline,none,hf-leaderboard-v1,significance_level,396,1000,518,39.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,248,1000,481,24.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,523,1000,7,52.3,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,379,1000,481,37.9,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,488,1000,481,48.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+qwen3.7-max,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,433,1000,481,43.3,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,Qwen3.7-Max,historical_pair,
+deepseek-v4-pro,baseline,none,local-paper-v1,perfect,129,1000,723,12.9,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,baseline,none,hf-leaderboard-v1,compilation_success,281,1000,0,28.1,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,baseline,none,hf-leaderboard-v1,partial_replication,195,1000,723,19.5,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,baseline,none,hf-leaderboard-v1,coefficient_direction,260,1000,723,26.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,baseline,none,hf-leaderboard-v1,significance_level,236,1000,723,23.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,deepagents,DeepAgents 0.7.13,local-paper-v1,perfect,183,1000,648,18.3,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,compilation_success,352,1000,0,35.2,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,partial_replication,270,1000,648,27.0,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,coefficient_direction,338,1000,648,33.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,deepagents,DeepAgents 0.7.13,hf-leaderboard-v1,significance_level,308,1000,648,30.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek V4 Pro,historical_pair,
+deepseek-v4-pro,dsh,DeepSeek Harness SDK 0.1.5rc1,local-paper-v1,perfect,372,1000,305,37.2,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek API [v4-pro alias] + DSH,independent,False
+deepseek-v4-pro,dsh,DeepSeek Harness SDK 0.1.5rc1,hf-leaderboard-v1,compilation_success,699,1000,30,69.9,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek API [v4-pro alias] + DSH,independent,False
+deepseek-v4-pro,dsh,DeepSeek Harness SDK 0.1.5rc1,hf-leaderboard-v1,partial_replication,556,1000,302,55.6,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek API [v4-pro alias] + DSH,independent,False
+deepseek-v4-pro,dsh,DeepSeek Harness SDK 0.1.5rc1,hf-leaderboard-v1,coefficient_direction,683,1000,302,68.3,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek API [v4-pro alias] + DSH,independent,False
+deepseek-v4-pro,dsh,DeepSeek Harness SDK 0.1.5rc1,hf-leaderboard-v1,significance_level,598,1000,304,59.8,False,6ea7c5684664bfe5c6e3d282a5e28f68244acd9e,DeepSeek API [v4-pro alias] + DSH,independent,False

leaderboard.js

@@ -3,7 +3,7 @@ const $ = id => document.getElementById(id);
 const metricKeys = ['perfect', 'partial_replication', 'compilation_success', 'coefficient_direction', 'significance_level'];
 const en = Object.fromEntries([...document.querySelectorAll('[data-i18n]')].map(el => [el.dataset.i18n, el.innerHTML]));
 Object.assign(en, {
-  model: 'Model', rank: '#', score: 'Score', evidenceCol: 'Evidence', pp: 'pp',
+  model: 'Model / API alias', rank: '#', score: 'Score', evidenceCol: 'Evidence', pp: 'pp',
   passed: 'successes', metricUnknown: 'metric unknown', review: 'Evidence gaps',
   config: 'Config', recordsLink: 'Results', allSealed: 'Archived',
   taskUnknown: 'task status unknown', invalid: 'invalid evidence', interruptions: 'interruptions',
@@ -11,6 +11,8 @@ Object.assign(en, {
   groupLocal: 'Local paper profile', groupHF: 'HF-style profile · parity unverified',
   loading: 'Loading archived results…', searchPlaceholder: 'Search models…',
   matched: '{n} of {total} models', localProfile: 'local-paper-v1', hfProfile: 'hf-leaderboard-v1 · parity unverified',
+  matchedRuns: '{n} of {total} runs', protocol: 'Protocol', independentRun: 'Independent · alias unverified',
+  published: 'Published {date}',
   perfectDesc: 'Coefficient and standard-error relative errors ≤ 1%, and p-value absolute error ≤ 0.01, all at once.',
   partial_replicationDesc: 'Coefficient relative error ≤ 5%. No standard-error or p-value condition.',
   compilation_successDesc: 'Verified code execution completed successfully; a valid prediction JSON is not required.',
@@ -19,35 +21,41 @@ Object.assign(en, {
 });
 const zh = {
   dataset: '数据集 ↗', archive: '结果存档 ↗', edition: '研究榜单 · 2026 年 9 月',
-  title: '同样的模型，<br><span>两种执行方式。</span>',
-  intro: '在真实计量经济学复现任务上，对比单次 Agent 与加入 DeepAgents Harness 后的表现。',
-  models: '个模型', groups: '组实验', tasks: '题 / 组', updated: '发布于 2026 年 9 月 17 日',
-  single: '单次 Agent', harness: 'Agent + Harness',
+  title: '模型与 Harness，<br><span>复现真实研究。</span>',
+  intro: '查看单次 Agent、DeepAgents 和独立 DeepSeek Harness 实验在真实计量经济学复现任务上的表现。',
+  models: '种请求模型别名', groups: '组实验', tasks: '题 / 组', records: '条任务记录', published: '发布于 {date}',
+  single: '单次 Agent', harness: '全部 Harness 实验', deepagents: 'DeepAgents',
+  dsh: 'DeepSeek Harness · 独立实验',
+  dshDesc: '使用官方 DSH SDK，请求 v4-pro API 别名。按固定规则纳入技术补测，底层模型权重未经核实。',
+  dshNoticeTitle: 'DSH 作为独立实验展示。',
+  dshNotice: '请求使用 deepseek-v4-pro 别名，底层模型权重未经核实。固定的 1,000 题结果包含 87 次技术补测替换（其中 27 题调整了输入 schema 提示），仍有 30 题状态未知；不纳入 DeepAgents 配对提升计算。',
+  dshProtocol: '查看运行协议与证据 ↗',
   singleDesc: '一次模型调用生成代码，然后执行；不进行多轮 Agent 循环。',
   harnessDesc: 'DeepAgents 进行规划、工具调用与代码修订；最多 6 次模型调用，最后执行。',
   resultsLabel: '01 / 评测结果', rankings: 'Agent 与 Harness 对照榜单', research: '研究结果 · 阶段版',
   scope: '基于已存档结果的本地评分，尚未验证与官方评分器完全一致。两组预算及不同批次协议存在差异，提升值用于描述观察结果，不代表等成本对比。',
-  compare: '双组对照', download: '下载 CSV ↓', metric: '评分指标',
+  compare: 'Baseline / DeepAgents 配对', download: '下载 CSV ↓', metric: '评分指标',
   perfect: '完整复现率 · 本地口径', partial_replication: '部分复现率 · HF 口径',
   compilation_success: '执行成功率 · HF 口径', coefficient_direction: '系数方向正确率 · HF 口径',
   significance_level: '显著性水平正确率 · HF 口径',
-  sort: '排序方式', harnessScore: 'Harness 组成绩', singleScore: '单次组成绩', lift: '提升', search: '查找模型',
+  sort: '排序方式', harnessScore: 'DeepAgents 组成绩', singleScore: '单次组成绩', lift: '提升', search: '查找模型',
   error: '结果加载失败，请重试或打开结果存档。', retry: '重试', empty: '没有匹配的模型。',
   denominator: '所有指标固定以 1,000 题为分母，未知和无效记录不计为成功。',
   methodLabel: '02 / 如何解读', methodTitle: '每个成绩都有据可查。', historical: '查看历史榜单 ↗',
   fixedTasks: '固定的题目集合', fixedDesc: '所有组使用相同的 Selected_1000 题号和数据集版本。成绩 = 成功数 ÷ 1,000，失败和未知记录始终保留在分母中。',
   separateMetrics: '两套评分口径', profilesDesc: '完整复现率采用本地论文口径。其余四项 HF 指标由本地实现复现网页定义，尚未验证与官方评分器一致。',
-  budgetTitle: '结合预算看提升', budgetDesc: '单次组使用 1 次模型调用；DeepAgents 最多使用 6 次模型调用和 4 次试运行工具调用。历史恢复、时限和运行协议有差异，不能作为等成本的因果估计。',
+  budgetTitle: '结合预算看提升', budgetDesc: '单次组使用 1 次模型调用；DeepAgents 和 DSH 每次尝试最多 6 次模型调用、4 次试运行工具调用。DSH 额外进行了 87 次技术补测，完整环境和推理设定是否一致尚未核实；不计算 DSH 配对提升，也不能据此判断等成本的因果效果。',
   evidence: '证据状态与指标定义',
   unknownDesc: '“指标未知”表示该指标无法判断，可能源于执行失败或输出缺失；它与“任务最终状态未知”不同。两者均不剔除，也不计为成功。',
-  evidenceCaption: '各组已接收存档中的状态', model: '模型',
-  verification: '四项 HF 指标已与逐题存档标记核对。Sol 和 Opus 的完整复现率来自已接收汇总；其余四个模型还可逐题核对本地评分标记。本次网页更新没有重新运行或重新评分实验。',
+  evidenceCaption: '各组已接收存档中的状态', model: '模型 / API 别名', protocol: '执行方式',
+  verification: '四项 HF 指标已与逐题存档标记核对。Sol 和 Opus 的完整复现率来自已接收汇总；其余实验（含 DSH）均可逐题核对本地评分标记。本次网页更新没有重新运行或重新评分实验。',
   archiveNote: '各组原始存档及当时的发布元数据保持不变，本页是基于存档的新展示。本版不包含 GPT-5.5 和尚未存档的实验。',
   footer: '衡量计量经济学复现能力。', source: '来源快照 ↗', json: '数据 JSON ↗', project: '项目页 ↗',
   rank: '#', score: '成绩', evidenceCol: '证据', pp: '百分点', passed: '题成功', metricUnknown: '指标未知', review: '存在证据缺口',
   config: '配置', recordsLink: '逐题结果', allSealed: '已存档', taskUnknown: '任务状态未知', invalid: '无效证据', interruptions: '中断记录',
   historicalStatus: '历史存档，未导出任务未知总数', groupLocal: '本地论文口径', groupHF: 'HF 网页口径 · 一致性待验证',
   loading: '正在加载存档结果…', searchPlaceholder: '搜索模型…', matched: '显示 {n} / {total} 个模型',
+  matchedRuns: '显示 {n} / {total} 组实验', independentRun: '独立实验 · 别名权重未核实',
   localProfile: 'local-paper-v1 · 本地口径', hfProfile: 'hf-leaderboard-v1 · 一致性待验证',
   perfectDesc: '系数与标准误的相对误差均 ≤ 1%，同时 p 值绝对误差 ≤ 0.01。',
   partial_replicationDesc: '系数相对误差 ≤ 5%，不要求标准误或 p 值达标。',
@@ -58,7 +66,7 @@ const zh = {
 let language = 'en';
 try { language = localStorage.getItem('inferencenet-language') === 'zh' ? 'zh' : 'en'; } catch (_) {}
 let data = null;
-let view = 'compare';
+let view = 'harness';
 let sortBy = 'deepagents';
 const t = key => (language === 'zh' ? zh[key] : en[key]) ?? en[key] ?? key;
 const esc = value => String(value).replace(/[&<>"']/g, char => ({'&':'&amp;', '<':'&lt;', '>':'&gt;', '"':'&quot;', "'":'&#39;'}[char]));
@@ -79,23 +87,40 @@ function applyLanguage() {
 }
 
 function validate(candidate) {
-  if (candidate.schema_version !== 1 || candidate.models?.length !== 6 || candidate.groups !== 12 || candidate.records !== 12000 || candidate.tasks_per_group !== 1000 || candidate.official_parity_verified !== false || !/^[a-f0-9]{40}$/.test(candidate.source_revision)) throw new Error('Invalid leaderboard snapshot');
+  if (candidate.schema_version !== 1 || candidate.models?.length !== 6 || ![12, 13].includes(candidate.groups) || candidate.records !== candidate.groups * 1000 || candidate.tasks_per_group !== 1000 || candidate.official_parity_verified !== false || !/^[a-f0-9]{40}$/.test(candidate.source_revision) || !/^\d{4}-\d{2}-\d{2}$/.test(candidate.published_date)) throw new Error('Invalid leaderboard snapshot');
   const ids = new Set();
+  let groups = 0;
   for (const model of candidate.models) {
     if (!/^[a-z0-9.-]+$/.test(model.id) || ids.has(model.id) || typeof model.name !== 'string') throw new Error('Invalid model');
     ids.add(model.id);
-    for (const armId of ['baseline', 'deepagents']) {
+    if (!model.arms?.baseline || !model.arms?.deepagents) throw new Error('Missing paired arm');
+    for (const armId of Object.keys(model.arms)) {
+      if (!['baseline', 'deepagents'].includes(armId) && !(model.id === 'deepseek-v4-pro' && armId === 'dsh')) throw new Error('Unexpected arm');
       const arm = model.arms?.[armId];
       if (arm?.path !== `results/${model.id}/${armId}` || typeof arm.complete !== 'boolean') throw new Error('Invalid arm');
+      if (armId === 'dsh' && (arm.paired_comparison !== false || arm.comparison_group !== 'independent' || arm.underlying_model_weights_verified !== false || arm.model_alias !== 'deepseek-v4-pro' || typeof arm.display_name !== 'string' || arm.evidence_class !== 'technical_recovery_view' || arm.protocol_url !== arm.path + '/README.md')) throw new Error('Invalid independent result');
+      groups += 1;
       for (const key of metricKeys) {
         const metric = arm.metrics?.[key];
         if (!metric || metric.denominator !== 1000 || !Number.isInteger(metric.count) || !Number.isInteger(metric.unknown) || metric.count < 0 || metric.unknown < 0 || metric.count + metric.unknown > 1000 || !Number.isFinite(metric.score) || Math.abs(metric.score - metric.count / 10) > 1e-8 || metric.profile !== (key === 'perfect' ? 'local-paper-v1' : 'hf-leaderboard-v1')) throw new Error('Invalid metric');
       }
     }
   }
+  if (groups !== candidate.groups) throw new Error('Inconsistent group count');
   return candidate;
 }
 
+// Pairing is intentionally limited to the two historical arms. Independent
+// harness entries have their own rank and can never enter a paired lift.
+function rankedEntries(snapshot, selectedView, key, selectedSort, query) {
+  const entries = selectedView === 'harness'
+    ? snapshot.models.flatMap(model => Object.entries(model.arms).filter(([id]) => id !== 'baseline').map(([armId, arm]) => ({model, armId, arm, name: arm.display_name || model.name, value: arm.metrics[key].count})))
+    : snapshot.models.map(model => ({model, armId: selectedView === 'compare' ? null : 'baseline', arm: model.arms.baseline, name: model.name,
+      value: selectedView === 'baseline' ? model.arms.baseline.metrics[key].count : selectedSort === 'gain' ? model.arms.deepagents.metrics[key].count - model.arms.baseline.metrics[key].count : model.arms[selectedSort].metrics[key].count}));
+  const search = query.toLowerCase().trim();
+  return {total: entries.length, entries: entries.filter(entry => `${entry.name} ${entry.model.id} ${entry.armId || ''}`.toLowerCase().includes(search)).sort((a, b) => b.value - a.value || a.name.localeCompare(b.name))};
+}
+
 function scoreCell(arm, key, style) {
   const m = arm.metrics[key];
   return `<td class="score-cell ${style}"><span class="score">${m.score.toFixed(1)}<small>%</small></span><div class="track" aria-hidden="true"><div class="fill" style="width:${m.score}%"></div></div><span class="counts">${m.count} / 1,000 ${esc(t('passed'))}</span><span class="unknown">${esc(t('metricUnknown'))}: ${m.unknown}</span></td>`;
@@ -112,38 +137,43 @@ function render() {
   const key = $('metric').value;
   $('metric-description').innerHTML = `<span class="profile">${esc(t(key === 'perfect' ? 'localProfile' : 'hfProfile'))}</span>${esc(t(key + 'Desc'))}`;
   if (!data) return;
-  const search = $('search').value.toLowerCase().trim();
   const count = (m, arm) => m.arms[arm].metrics[key].count;
-  const value = m => view !== 'compare' ? count(m, view) : sortBy === 'gain' ? count(m, 'deepagents') - count(m, 'baseline') : count(m, sortBy);
-  const models = data.models.filter(m => `${m.name} ${m.id}`.toLowerCase().includes(search)).sort((a, b) => value(b) - value(a) || a.name.localeCompare(b.name));
+  const {total, entries} = rankedEntries(data, view, key, sortBy, $('search').value);
+  $('published-date').textContent = t('published').replace('{date}', data.published_date);
   $('sort-control').hidden = view !== 'compare';
-  $('table-wrap').hidden = models.length === 0;
-  $('empty').hidden = models.length !== 0;
-  $('result-count').textContent = t('matched').replace('{n}', models.length).replace('{total}', data.models.length);
+  $('table-wrap').hidden = entries.length === 0;
+  $('empty').hidden = entries.length !== 0;
+  $('result-count').textContent = t(view === 'compare' ? 'matched' : 'matchedRuns').replace('{n}', entries.length).replace('{total}', total);
   $('caption').textContent = `${t(view === 'compare' ? 'compare' : view === 'baseline' ? 'single' : 'harness')} — ${t(key)}`;
   const th = (text, cls = '') => `<th scope="col" class="${cls}">${esc(text)}</th>`;
-  $('table-head').innerHTML = '<tr>' + th(t('rank'), 'rank') + th(t('model')) + (view === 'compare' ? th(t('single'), 'baseline') + th(t('harness'), 'harness') + th(t('lift')) : th(t('score'), view === 'baseline' ? 'baseline' : 'harness')) + th(t('evidenceCol')) + '</tr>';
+  $('table-head').innerHTML = '<tr>' + th(t('rank'), 'rank') + th(t('model')) + (view === 'compare' ? th(t('single'), 'baseline') + th(t('deepagents'), 'harness') + th(t('lift')) : th(t('score'), view === 'baseline' ? 'baseline' : 'harness')) + th(t('evidenceCol')) + '</tr>';
   let lastValue = null, rank = 0;
-  $('table-body').innerHTML = models.map((model, index) => {
-    const currentValue = value(model);
+  $('table-body').innerHTML = entries.map((entry, index) => {
+    const {model, arm, armId, name, value: currentValue} = entry;
     if (currentValue !== lastValue) rank = index + 1;
     lastValue = currentValue;
     const arms = model.arms;
-    const gap = view === 'compare' ? !arms.baseline.complete || !arms.deepagents.complete : !arms[view].complete;
+    const gap = view === 'compare' ? !arms.baseline.complete || !arms.deepagents.complete : !arm.complete;
     const badge = gap ? `<span class="tag">${esc(t('review'))}</span>` : '';
-    const identity = `<td class="rank">${String(rank).padStart(2, '0')}</td><th scope="row" class="model-cell"><div class="model-name">${esc(model.name)}</div><div class="model-meta">1,000 ${language === 'zh' ? '题 / 组' : 'tasks / group'} ${badge}</div></th>`;
+    const independent = armId === 'dsh';
+    const protocol = view === 'harness' ? `<span class="run-protocol ${independent ? 'independent' : 'harness'}">${esc(independent ? t('independentRun') : 'DeepAgents')}</span>` : '';
+    const identity = `<td class="rank">${String(rank).padStart(2, '0')}</td><th scope="row" class="model-cell"><div class="model-name">${esc(name)}</div>${protocol}<div class="model-meta">1,000 ${language === 'zh' ? '题 / 组' : 'tasks / group'} ${badge}</div></th>`;
     let scores = '', sources = '';
     if (view === 'compare') {
       const gain = (count(model, 'deepagents') - count(model, 'baseline')) / 10;
       scores = scoreCell(arms.baseline, key, 'baseline') + scoreCell(arms.deepagents, key, 'harness') + `<td class="gain ${gain < 0 ? 'negative' : ''}">${gain > 0 ? '+' : ''}${gain.toFixed(1)}<small>${esc(t('pp'))}</small></td>`;
       sources = external(sourceURL(arms.baseline.path, true), 'A') + external(sourceURL(arms.deepagents.path, true), 'B');
     } else {
-      scores = scoreCell(arms[view], key, view === 'baseline' ? 'baseline' : 'harness');
-      sources = external(sourceURL(arms[view].path + '/results.jsonl'), t('recordsLink')) + external(sourceURL(arms[view].path + '/config.json'), t('config'));
+      scores = scoreCell(arm, key, view === 'baseline' ? 'baseline' : independent ? 'independent' : 'harness');
+      sources = external(sourceURL(arm.path + '/results.jsonl'), t('recordsLink')) + external(sourceURL(arm.path + '/config.json'), t('config'));
+      if (independent) sources += external(sourceURL(arm.protocol_url), t('protocol'));
     }
-    return `<tr data-model="${esc(model.id)}">${identity}${scores}<td class="source-cell">${sources}</td></tr>`;
+    return `<tr data-model="${esc(model.id)}" data-arm="${esc(armId || 'paired')}">${identity}${scores}<td class="source-cell">${sources}</td></tr>`;
   }).join('');
-  $('evidence-body').innerHTML = data.models.map(m => `<tr><th scope="row">${esc(m.name)}</th>${evidenceState(m.arms.baseline)}${evidenceState(m.arms.deepagents)}</tr>`).join('');
+  $('evidence-body').innerHTML = data.models.flatMap(m => Object.entries(m.arms).map(([id, arm]) => `<tr><th scope="row">${esc(arm.display_name || m.name)}</th><td>${esc(id === 'baseline' ? t('single') : id === 'dsh' ? t('dsh') : 'DeepAgents')}</td>${evidenceState(arm)}</tr>`)).join('');
+  const dsh = data.models.find(m => m.id === 'deepseek-v4-pro')?.arms.dsh;
+  $('independent-notice').hidden = !dsh;
+  if (dsh) $('dsh-protocol').href = sourceURL(dsh.protocol_url);
 }
 
 async function loadData() {
@@ -155,7 +185,8 @@ async function loadData() {
   $('empty').hidden = true;
   $('result-count').textContent = '';
   $('evidence-body').replaceChildren();
-  for (const id of ['model-count', 'group-count', 'task-count']) $(id).textContent = '—';
+  $('independent-notice').hidden = true;
+  for (const id of ['model-count', 'group-count', 'task-count', 'record-count']) $(id).textContent = '—';
   try {
     const response = await fetch('./leaderboard-data.json', {cache: 'no-cache'});
     if (!response.ok) throw new Error(`HTTP ${response.status}`);
@@ -163,6 +194,7 @@ async function loadData() {
     $('model-count').textContent = data.models.length;
     $('group-count').textContent = data.groups;
     $('task-count').textContent = data.tasks_per_group.toLocaleString('en-US');
+    $('record-count').textContent = data.records.toLocaleString('en-US');
     $('revision-link').href = sourceURL('results', true);
     render();
   } catch (error) {

results/README.md

@@ -1,3 +1,27 @@
+# Current archive addition: DeepSeek API + DeepSeek Harness
+
+The archive now includes a separate [DSH group](deepseek-v4-pro/dsh/README.md)
+alongside all twelve existing baseline / DeepAgents groups: 13 groups and 13,000
+task records. The six existing baseline / DeepAgents comparisons are unchanged.
+
+This new group requested `deepseek-v4-pro` through the DeepSeek API; the actual
+served weights are unconfirmed because the provider's routing announcement and
+model table conflict. It is not a verified same-model, equal-cost comparison
+against the historical DeepSeek DeepAgents group.
+
+The DSH archive fixes 1,000 selected task results after 87 preselected technical
+supplements, retains 30 uncertain tasks, and accounts for all 1,087 attempts.
+Full replication is 37.2%; the four separate HF-style scores are 69.9%, 55.6%,
+68.3% and 59.8%. See its README and configuration for recovery, input-profile
+amendments, unknown usage and source identities. No new model calls or rescoring
+were performed for publication.
+
+The current display is defined by the root `leaderboard-data.json` and root
+README. Publication flags and page references in the historical archive text
+below describe those earlier uploads, not necessarily the current display.
+
+## Preserved historical archive notes
+
 # InferenceNet 实验结果归档
 
 目录按 `results/<model>/<baseline|deepagents>/` 排列，不增加日期层。实验批次和时间记录在各目录的 config.json；更新历史由 Space 的 Git commit 保留。

results/deepseek-v4-pro/dsh/README.md

@@ -0,0 +1,38 @@
+# DeepSeek API (deepseek-v4-pro alias) + DSH
+
+This is a **technical-recovery research view**, not an equal-cost controlled estimate of a harness effect. The API alias is `deepseek-v4-pro`; **underlying model weights are not verified**. Conflicting official routing/model descriptions are linked in `config.json`.
+
+| Metric | Successes / 1000 | Score | Unknown |
+|---|---:|---:|---:|
+| Execution success | 699 | 69.9% | 30 |
+| Coefficient-only partial replication | 556 | 55.6% | 302 |
+| Coefficient direction | 683 | 68.3% | 302 |
+| Significance category | 598 | 59.8% | 304 |
+| Full replication — local-paper-v1 | 372 | 37.2% | 305 |
+
+The selected 1000 tasks contain **699 scored, 271 failed, and 30 uncertain** outcomes. Every rate retains all 1000 tasks. Four HF-style metrics and five local-paper-v1 metrics have separate profiles; unknowns are never dropped. Local full replication requires coefficient and standard-error relative errors <= 1%, and p-value absolute error <= 0.01. Official scorer parity remains unverified.
+
+## Attempt selection and amendments
+
+The original 1000 attempts remain unchanged. Eligibility for **one extra attempt for 87 tasks** was frozen before their supplementary outcomes: 60 infrastructure-interrupted tasks and 27 pre-model input-metadata failures. The final view selects the second attempt for all 87, regardless of outcome, and the original attempt for the other 913. No best-of selection or fallback to a better result is used.
+
+The 27 schema tasks explicitly change `raw-role-schema` to `raw-role`, allowing the existing Python tool to inspect input headers. No tools or per-attempt budgets are added, but exact prompt parity is not claimed. Controller transport, cancellation and writer-drain repairs are retained in source hashes; the original historical attempts are not relabelled as if those repairs had already applied.
+
+Each attempt permits 6 model calls, 4 Python tool calls, 32768 cumulative output tokens and 1800 seconds; execution timeout is 90 seconds. Execution uses bwrap and a frozen rootfs, not Docker. The historical baseline/DeepAgents configurations and complete environment/prompt parity are not fully established. This DSH entry does not replace either historical group.
+
+## All-attempt accounting
+
+All **1087 attempts** are charged: **5944 model calls, 3999 tool calls**. There are **38 attempts with unknown usage**, so full input/output/total token sums and monetary cost remain unknown. The **60580246-token subtotal** covers only the 1049 attempts with complete known usage and is not a complete invoice. Task wall-time sums are cumulative worker time, not elapsed campaign duration.
+
+Task 0907 retains its original sealed record plus a reviewed late-timeout receipt; effective usage is unknown. `all-attempts.jsonl` preserves original recorded budgets alongside effective accounting rather than rewriting history.
+
+## Files and evidence boundary
+
+- `results.jsonl` / `results.csv`: the fixed final selection, five local flags plus four HF-style flags, result/score SHA256, source commit, status and usage.
+- `summary.json` / `config.json`: metrics, fixed task IDs, runtime/package pins, per-attempt protocol and disclosures.
+- `first-pass-results.jsonl` / `first-pass-summary.json`: unchanged first-pass four-metric view, independently labelled.
+- `all-attempts.jsonl` / `accounting.json`: all original and supplementary costs, including unknown usage.
+- `provenance.json` / `export-verification.json` / `manifest.sha256`: source bindings and export checks.
+- `diagnostics.json`: failure categories and unknown IDs, without raw error messages.
+
+`resolved=true` means sealed, not necessarily successful or fully known. A later-stage pending/failed status may follow an earlier failure. The selected-results export includes 699 verified prediction triplets and explicitly marks 301 records without a selected final prediction. Prediction fields in the separate first-pass export are omitted (`prediction_export_status=public_export_omission`); this does **not** imply that scored first-pass attempts lacked predictions. Hidden references, inputs, raw prompts/responses, credentials and machine paths are not included. The export copies verified flags and does not call a model, run a program or rescore an attempt.

results/deepseek-v4-pro/dsh/accounting.json

@@ -0,0 +1,76 @@
+{
+  "accounting_note": "Known subtotals are not complete token totals or invoices.",
+  "all_attempts": {
+    "attempt_count": 1087,
+    "known_usage_subtotal": {
+      "input_tokens": 42956609,
+      "output_tokens": 17623637,
+      "total_tokens": 60580246
+    },
+    "known_usage_task_count": 1049,
+    "model_calls": 5944,
+    "task_wall_seconds_sum": 273576.778,
+    "tool_calls": 3999,
+    "unknown_usage_task_count": 38,
+    "usage": {
+      "input_tokens": null,
+      "output_tokens": null,
+      "total_tokens": null
+    }
+  },
+  "infrastructure": {
+    "attempt_count": 60,
+    "known_usage_subtotal": {
+      "input_tokens": 2161547,
+      "output_tokens": 804095,
+      "total_tokens": 2965642
+    },
+    "known_usage_task_count": 49,
+    "model_calls": 310,
+    "task_wall_seconds_sum": 28521.539,
+    "tool_calls": 207,
+    "unknown_usage_task_count": 11,
+    "usage": {
+      "input_tokens": null,
+      "output_tokens": null,
+      "total_tokens": null
+    }
+  },
+  "monetary_cost": null,
+  "primary": {
+    "attempt_count": 1000,
+    "known_usage_subtotal": {
+      "input_tokens": 39691223,
+      "output_tokens": 16146935,
+      "total_tokens": 55838158
+    },
+    "known_usage_task_count": 974,
+    "model_calls": 5476,
+    "task_wall_seconds_sum": 236238.034,
+    "tool_calls": 3687,
+    "unknown_usage_task_count": 26,
+    "usage": {
+      "input_tokens": null,
+      "output_tokens": null,
+      "total_tokens": null
+    }
+  },
+  "schema": {
+    "attempt_count": 27,
+    "known_usage_subtotal": {
+      "input_tokens": 1103839,
+      "output_tokens": 672607,
+      "total_tokens": 1776446
+    },
+    "known_usage_task_count": 26,
+    "model_calls": 158,
+    "task_wall_seconds_sum": 8817.205,
+    "tool_calls": 105,
+    "unknown_usage_task_count": 1,
+    "usage": {
+      "input_tokens": null,
+      "output_tokens": null,
+      "total_tokens": null
+    }
+  }
+}

results/deepseek-v4-pro/dsh/config.json

@@ -0,0 +1,1205 @@
+{
+  "arm": "dsh",
+  "campaign": {
+    "execution_workers": 6,
+    "max_attempts": 1,
+    "workers": 12
+  },
+  "comparison": {
+    "historical_parity_verified": false,
+    "unknown_historical_settings": [
+      "complete_system_prompt_and_tool_schema",
+      "provider_endpoint_and_deployment_version",
+      "effective_thinking_policy",
+      "full_rootfs_and_dependency_lock",
+      "controller_source_and_retry_semantics"
+    ]
+  },
+  "dataset": {
+    "expected_tasks": 1000,
+    "prior_results_imported": false,
+    "repo_id": "CamoAiLab/InferenceNet",
+    "revision": "59f9512a38e594528807744214a60ee00367434e",
+    "task_directory": "Selected_1000",
+    "task_ids": [
+      1,
+      2,
+      3,
+      4,
+      5,
+      6,
+      7,
+      8,
+      9,
+      10,
+      11,
+      12,
+      13,
+      14,
+      15,
+      16,
+      17,
+      18,
+      19,
+      20,
+      21,
+      22,
+      23,
+      24,
+      25,
+      26,
+      27,
+      28,
+      29,
+      30,
+      31,
+      32,
+      33,
+      34,
+      35,
+      36,
+      37,
+      38,
+      39,
+      40,
+      41,
+      42,
+      43,
+      44,
+      45,
+      46,
+      47,
+      48,
+      49,
+      50,
+      51,
+      52,
+      53,
+      54,
+      55,
+      56,
+      57,
+      58,
+      59,
+      60,
+      61,
+      62,
+      63,
+      64,
+      65,
+      66,
+      67,
+      68,
+      69,
+      70,
+      71,
+      72,
+      73,
+      74,
+      75,
+      76,
+      77,
+      78,
+      79,
+      80,
+      81,
+      82,
+      83,
+      84,
+      85,
+      86,
+      87,
+      88,
+      89,
+      90,
+      91,
+      92,
+      93,
+      94,
+      95,
+      96,
+      97,
+      98,
+      100,
+      101,
+      102,
+      103,
+      104,
+      105,
+      106,
+      107,
+      108,
+      109,
+      110,
+      111,
+      112,
+      113,
+      114,
+      115,
+      116,
+      117,
+      118,
+      119,
+      120,
+      121,
+      122,
+      123,
+      124,
+      125,
+      126,
+      127,
+      128,
+      129,
+      130,
+      131,
+      132,
+      133,
+      134,
+      135,
+      136,
+      137,
+      138,
+      139,
+      140,
+      141,
+      142,
+      143,
+      144,
+      145,
+      146,
+      147,
+      148,
+      149,
+      150,
+      151,
+      152,
+      153,
+      154,
+      155,
+      156,
+      157,
+      158,
+      159,
+      160,
+      161,
+      162,
+      163,
+      164,
+      165,
+      166,
+      167,
+      168,
+      169,
+      170,
+      171,
+      172,
+      173,
+      174,
+      175,
+      176,
+      177,
+      178,
+      179,
+      180,
+      181,
+      182,
+      183,
+      184,
+      185,
+      186,
+      187,
+      188,
+      189,
+      190,
+      191,
+      192,
+      193,
+      194,
+      195,
+      196,
+      197,
+      198,
+      199,
+      200,
+      201,
+      202,
+      203,
+      204,
+      205,
+      206,
+      207,
+      208,
+      209,
+      210,
+      211,
+      212,
+      213,
+      214,
+      215,
+      216,
+      217,
+      218,
+      219,
+      220,
+      221,
+      222,
+      223,
+      224,
+      225,
+      226,
+      227,
+      228,
+      229,
+      230,
+      231,
+      232,
+      233,
+      234,
+      235,
+      236,
+      237,
+      238,
+      239,
+      240,
+      241,
+      242,
+      248,
+      249,
+      250,
+      251,
+      252,
+      253,
+      254,
+      255,
+      256,
+      257,
+      258,
+      259,
+      261,
+      262,
+      263,
+      264,
+      265,
+      266,
+      290,
+      291,
+      292,
+      293,
+      294,
+      295,
+      302,
+      303,
+      304,
+      305,
+      306,
+      307,
+      308,
+      309,
+      310,
+      311,
+      312,
+      313,
+      314,
+      315,
+      316,
+      317,
+      318,
+      319,
+      320,
+      321,
+      322,
+      323,
+      324,
+      325,
+      326,
+      327,
+      328,
+      329,
+      330,
+      331,
+      332,
+      333,
+      334,
+      335,
+      336,
+      337,
+      338,
+      339,
+      340,
+      341,
+      342,
+      343,
+      344,
+      345,
+      346,
+      347,
+      348,
+      349,
+      350,
+      351,
+      352,
+      353,
+      354,
+      355,
+      356,
+      357,
+      358,
+      359,
+      360,
+      361,
+      362,
+      363,
+      364,
+      365,
+      366,
+      367,
+      368,
+      369,
+      370,
+      371,
+      372,
+      373,
+      374,
+      375,
+      376,
+      377,
+      378,
+      379,
+      380,
+      381,
+      382,
+      383,
+      384,
+      385,
+      386,
+      387,
+      388,
+      389,
+      390,
+      391,
+      392,
+      393,
+      394,
+      395,
+      396,
+      397,
+      398,
+      399,
+      400,
+      401,
+      402,
+      403,
+      404,
+      405,
+      406,
+      407,
+      408,
+      409,
+      410,
+      411,
+      412,
+      413,
+      414,
+      415,
+      416,
+      417,
+      418,
+      419,
+      420,
+      421,
+      422,
+      423,
+      424,
+      425,
+      426,
+      427,
+      428,
+      429,
+      430,
+      431,
+      432,
+      433,
+      434,
+      435,
+      436,
+      437,
+      438,
+      439,
+      440,
+      441,
+      442,
+      443,
+      444,
+      445,
+      446,
+      447,
+      448,
+      449,
+      450,
+      451,
+      452,
+      453,
+      454,
+      455,
+      456,
+      457,
+      458,
+      459,
+      460,
+      461,
+      462,
+      463,
+      464,
+      465,
+      466,
+      467,
+      468,
+      469,
+      470,
+      471,
+      472,
+      473,
+      474,
+      475,
+      476,
+      477,
+      478,
+      479,
+      480,
+      481,
+      482,
+      483,
+      484,
+      485,
+      486,
+      487,
+      488,
+      489,
+      490,
+      491,
+      492,
+      493,
+      494,
+      495,
+      496,
+      497,
+      498,
+      499,
+      500,
+      501,
+      502,
+      503,
+      504,
+      505,
+      506,
+      507,
+      508,
+      509,
+      510,
+      511,
+      512,
+      513,
+      514,
+      515,
+      516,
+      517,
+      518,
+      519,
+      520,
+      521,
+      522,
+      523,
+      524,
+      525,
+      526,
+      527,
+      528,
+      529,
+      530,
+      531,
+      532,
+      533,
+      534,
+      535,
+      536,
+      537,
+      538,
+      539,
+      540,
+      541,
+      542,
+      543,
+      544,
+      545,
+      546,
+      547,
+      548,
+      549,
+      550,
+      551,
+      552,
+      553,
+      554,
+      555,
+      556,
+      557,
+      558,
+      559,
+      560,
+      561,
+      562,
+      563,
+      565,
+      566,
+      567,
+      568,
+      570,
+      571,
+      572,
+      573,
+      574,
+      575,
+      576,
+      577,
+      578,
+      579,
+      580,
+      581,
+      582,
+      583,
+      584,
+      585,
+      586,
+      587,
+      588,
+      589,
+      590,
+      591,
+      592,
+      593,
+      594,
+      595,
+      596,
+      597,
+      598,
+      601,
+      602,
+      603,
+      604,
+      605,
+      606,
+      607,
+      608,
+      609,
+      610,
+      611,
+      612,
+      613,
+      614,
+      615,
+      616,
+      617,
+      618,
+      619,
+      620,
+      621,
+      622,
+      623,
+      624,
+      664,
+      665,
+      666,
+      667,
+      668,
+      669,
+      670,
+      671,
+      672,
+      673,
+      674,
+      675,
+      676,
+      677,
+      678,
+      679,
+      680,
+      681,
+      682,
+      683,
+      684,
+      685,
+      686,
+      687,
+      688,
+      689,
+      690,
+      691,
+      692,
+      693,
+      694,
+      695,
+      696,
+      697,
+      698,
+      699,
+      700,
+      701,
+      702,
+      703,
+      704,
+      705,
+      706,
+      707,
+      708,
+      709,
+      710,
+      711,
+      712,
+      713,
+      714,
+      715,
+      716,
+      717,
+      718,
+      719,
+      720,
+      721,
+      722,
+      723,
+      724,
+      725,
+      726,
+      727,
+      728,
+      729,
+      730,
+      731,
+      732,
+      733,
+      734,
+      735,
+      736,
+      737,
+      739,
+      740,
+      741,
+      742,
+      743,
+      744,
+      745,
+      746,
+      747,
+      748,
+      749,
+      750,
+      751,
+      752,
+      753,
+      754,
+      755,
+      756,
+      757,
+      758,
+      759,
+      760,
+      761,
+      762,
+      763,
+      764,
+      765,
+      766,
+      767,
+      768,
+      772,
+      773,
+      774,
+      775,
+      776,
+      777,
+      778,
+      779,
+      780,
+      781,
+      782,
+      783,
+      784,
+      785,
+      786,
+      787,
+      788,
+      789,
+      790,
+      791,
+      793,
+      794,
+      795,
+      796,
+      797,
+      798,
+      799,
+      800,
+      801,
+      802,
+      803,
+      804,
+      805,
+      806,
+      807,
+      808,
+      825,
+      826,
+      827,
+      828,
+      829,
+      830,
+      831,
+      832,
+      833,
+      834,
+      835,
+      836,
+      837,
+      838,
+      839,
+      840,
+      841,
+      842,
+      843,
+      844,
+      845,
+      846,
+      847,
+      848,
+      849,
+      850,
+      851,
+      852,
+      853,
+      854,
+      855,
+      856,
+      857,
+      858,
+      859,
+      860,
+      861,
+      862,
+      863,
+      864,
+      865,
+      866,
+      867,
+      868,
+      869,
+      870,
+      871,
+      872,
+      873,
+      874,
+      875,
+      876,
+      877,
+      878,
+      879,
+      880,
+      881,
+      882,
+      883,
+      884,
+      885,
+      886,
+      887,
+      888,
+      889,
+      890,
+      891,
+      892,
+      893,
+      894,
+      895,
+      896,
+      897,
+      898,
+      899,
+      900,
+      901,
+      902,
+      903,
+      906,
+      907,
+      913,
+      914,
+      915,
+      917,
+      920,
+      921,
+      922,
+      923,
+      924,
+      925,
+      926,
+      927,
+      928,
+      929,
+      930,
+      931,
+      932,
+      933,
+      934,
+      935,
+      936,
+      937,
+      938,
+      939,
+      940,
+      941,
+      942,
+      943,
+      944,
+      945,
+      946,
+      947,
+      948,
+      949,
+      950,
+      951,
+      952,
+      953,
+      954,
+      955,
+      956,
+      957,
+      958,
+      959,
+      960,
+      961,
+      962,
+      963,
+      964,
+      965,
+      966,
+      967,
+      968,
+      969,
+      970,
+      971,
+      972,
+      973,
+      974,
+      975,
+      976,
+      977,
+      978,
+      979,
+      980,
+      981,
+      982,
+      983,
+      984,
+      985,
+      1001,
+      1002,
+      1003,
+      1004,
+      1005,
+      1006,
+      1007,
+      1008,
+      1009,
+      1010,
+      1011,
+      1012,
+      1013,
+      1014,
+      1015,
+      1016,
+      1017,
+      1018,
+      1019,
+      1020,
+      1021,
+      1022,
+      1023,
+      1024,
+      1025,
+      1026,
+      1027,
+      1028,
+      1029,
+      1030,
+      1031,
+      1032,
+      1033,
+      1034,
+      1035,
+      1036,
+      1037,
+      1038,
+      1039,
+      1040,
+      1041,
+      1042,
+      1043,
+      1044,
+      1045,
+      1046,
+      1047,
+      1048,
+      1049,
+      1050,
+      1051,
+      1052,
+      1053,
+      1054,
+      1055,
+      1056,
+      1057,
+      1058,
+      1059,
+      1060,
+      1061,
+      1062,
+      1063,
+      1064,
+      1065,
+      1066,
+      1067,
+      1068,
+      1069,
+      1070,
+      1071,
+      1072,
+      1073,
+      1074,
+      1075,
+      1076,
+      1077,
+      1078,
+      1079,
+      1080,
+      1081,
+      1082,
+      1083,
+      1084,
+      1085,
+      1086,
+      1087,
+      1088,
+      1089,
+      1090,
+      1091,
+      1092,
+      1093,
+      1094,
+      1095,
+      1096,
+      1097,
+      1098,
+      1099,
+      1100,
+      1101,
+      1102,
+      1103,
+      1104,
+      1105,
+      1106,
+      1107,
+      1108,
+      1109,
+      1110,
+      1111,
+      1112,
+      1113,
+      1114,
+      1115,
+      1116,
+      1117,
+      1118,
+      1119,
+      1120,
+      1121,
+      1122,
+      1123,
+      1124,
+      1125
+    ],
+    "task_list": "Selected_1000/1000_new.csv",
+    "task_set_sha256": "57e33e1883cc6e1a9e35d068614e28832c87b7cb1c22258cd5d9b3ad1e5ff901"
+  },
+  "display_name": "DeepSeek API (deepseek-v4-pro alias) + DSH",
+  "effective_model_call_cap": 6,
+  "evidence_class": "technical_recovery_view",
+  "execution": {
+    "backend": "bwrap",
+    "bwrap_use_rootfs_loader": true,
+    "cpus": 2.0,
+    "memory": "6g",
+    "proc_mode": "empty",
+    "rootfs_inventory_sha256": "5cdd4e51297f5a092a5b53cbe18e5d0ffd4cd9a9f219451bb2561289487848dd",
+    "timeout": 90,
+    "tmpfs": "256m"
+  },
+  "generation": {
+    "max_output_tokens": 32768,
+    "model": "deepseek-v4-pro",
+    "prompt_profile": "raw-role-schema",
+    "provider": "chat-completions",
+    "reasoning_effort": "high",
+    "schema_context_chars": 24000,
+    "stream": true,
+    "timeout": 1800
+  },
+  "harness": "DeepSeek Harness SDK 0.1.5rc1",
+  "limits": {
+    "model_calls": 6,
+    "output_tokens": 32768,
+    "tool_calls": 4,
+    "tool_timeout": 90,
+    "wall_seconds": 1800
+  },
+  "model": "deepseek-v4-pro",
+  "model_alias": "deepseek-v4-pro",
+  "model_identity": {
+    "api_request_alias": "deepseek-v4-pro",
+    "conflicting_official_descriptions": [
+      {
+        "claim": "Announcement says alias routes to V4.1 Flash from 2026-09-14 12:00 Beijing time until V4.1 Pro launches.",
+        "url": "https://api-docs.deepseek.com/zh-cn/news/news260910/"
+      },
+      {
+        "claim": "Model table lists alias as DeepSeek-V4-Pro-0813.",
+        "url": "https://api-docs.deepseek.com/quick_start/pricing/"
+      }
+    ],
+    "interpretation": "Conflicting official descriptions; neither underlying weight version is independently established.",
+    "successful_response_model_label": "deepseek-v4-pro",
+    "underlying_model_weights_verified": false
+  },
+  "notes": [
+    "Cumulative per-attempt output cap includes provider-reported reasoning tokens; no over-limit response is admitted for scoring.",
+    "Execution uses bwrap with a frozen rootfs, not Docker; historical rootfs equivalence is unverified.",
+    "27 supplementary tasks use raw-role instead of raw-role-schema. These are not exact-prompt-parity results.",
+    "API alias is established; underlying weights are unverified. Do not label as confirmed original V4-Pro-0813.",
+    "Selected prediction triplets and sealed local flags are allowlisted from original evidence; hidden references and raw model responses are excluded.",
+    "No independent same-protocol baseline is supplied; preserve the existing historical baseline and DeepAgents groups."
+  ],
+  "official_parity_verified": false,
+  "recovery": {
+    "best_of_attempts": false,
+    "infrastructure_tasks": 60,
+    "maximum_supplementary_attempts_per_task": 1,
+    "original_attempts": 1000,
+    "original_results_modified": false,
+    "policy": "Freeze eligibility before supplementary outcomes; select attempt 2 regardless of its score or status. Never best-of or fall back to a better attempt.",
+    "schema_amendment": {
+      "exact_prompt_parity": false,
+      "generation.prompt_profile": {
+        "from": "raw-role-schema",
+        "to": "raw-role"
+      },
+      "tools_and_per_attempt_budgets_changed": false
+    },
+    "schema_tasks": 27,
+    "selected_primary_attempts": 913,
+    "selected_supplementary_attempts": 87,
+    "supplementary_attempts": 87,
+    "total_attempts_charged": 1087
+  },
+  "request_policy": {
+    "reasoning_effort": "high",
+    "thinking": "enabled"
+  },
+  "runtime": {
+    "binary_sha256": "6f68ce88d98307533ee8fa58a8125de4dc019ab16fac8b512cec141a2d1961f8",
+    "packages": {
+      "annotated-types": {
+        "installed_wheel_sha256": "f072f4d804ea359e4eaf198b1af7a8b0943881a87f31bb764f8bf219bb9419e0",
+        "record_verified": true,
+        "version": "0.8.0"
+      },
+      "deepseek-harness-runtime-bin": {
+        "installed_wheel_sha256": "ad877237382baff4628969a9ebdbfe38486d971b7987c032d0e29775759b5229",
+        "record_verified": true,
+        "version": "0.1.5rc1"
+      },
+      "deepseek-harness-sdk": {
+        "installed_wheel_sha256": "8d9897e97e8a95fd738cb2c3e2c563951d824e43847ec6f3b0fb172c0729b4d8",
+        "record_verified": true,
+        "version": "0.1.5rc1"
+      },
+      "pydantic": {
+        "installed_wheel_sha256": "346a034f080da3755d8e9cb5e00e8b07de1d39e4f6e2c87d8ab7cafa0b269a73",
+        "record_verified": true,
+        "version": "2.13.5"
+      },
+      "pydantic-core": {
+        "installed_wheel_sha256": "0fc5be0abd4a407e200d844b404e33639a554e7bd0d448e7b9ae181be4789ac2",
+        "record_verified": true,
+        "version": "2.46.5"
+      },
+      "typing-extensions": {
+        "installed_wheel_sha256": "481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8",
+        "record_verified": true,
+        "version": "4.16.0"
+      },
+      "typing-inspection": {
+        "installed_wheel_sha256": "65b8397ba37ccbce054456aaccddfc91e6e3083c92824df348d96ca832f3f147",
+        "record_verified": true,
+        "version": "0.4.4"
+      }
+    },
+    "profile_files": {
+      "__init__.py": "8e24d82c627405e0a8f9de89ebdbf1ce23db63337ef44aacd4a789cea9697908",
+      "profile.patch.yml": "81a92895c5f3c4c85bb602e3349a509262eb144c63a566331a43fa73081e2731",
+      "requirements-controller.txt": "aee6d7d23cbeecc3a101ec56687af9cc7aecbd6e96443295042a5c36307587d4",
+      "requirements.lock.txt": "ed57fde608bf1cdabaa1fc5d23e5b254983606652be2f732ce4ed2f2c69ea7a1",
+      "run-python.mjs": "3681bcbdfc6fa10cbb0742fadcc8f3f94a745b428662c94214b370b01bbea67e",
+      "runtime.lock.json": "e241e17af63e1f2b5ebaab3845a40c8f35c8643c3b5622b778541663428aba44",
+      "runtime.py": "2223a5573a3bb6c7fec46e264920f5cacf3dedaffb48f6a732e7f4371380c5e7"
+    },
+    "python_version": "3.12.12",
+    "research_source_commit": "0d1f50007f9bca3f52b06e1c3074fa14d5fb0720",
+    "research_source_matches_installed_build_verified": false
+  },
+  "schedule_seed": 20260918,
+  "schema_version": 3,
+  "scoring": {
+    "profile": "local-paper-v1"
+  },
+  "source_identity": {
+    "controller_files": {
+      "dsh/__init__.py": "8e24d82c627405e0a8f9de89ebdbf1ce23db63337ef44aacd4a789cea9697908",
+      "dsh/profile.patch.yml": "81a92895c5f3c4c85bb602e3349a509262eb144c63a566331a43fa73081e2731",
+      "dsh/requirements-controller.txt": "aee6d7d23cbeecc3a101ec56687af9cc7aecbd6e96443295042a5c36307587d4",
+      "dsh/requirements.lock.txt": "ed57fde608bf1cdabaa1fc5d23e5b254983606652be2f732ce4ed2f2c69ea7a1",
+      "dsh/run-python.mjs": "3681bcbdfc6fa10cbb0742fadcc8f3f94a745b428662c94214b370b01bbea67e",
+      "dsh/runtime.lock.json": "e241e17af63e1f2b5ebaab3845a40c8f35c8643c3b5622b778541663428aba44",
+      "dsh/runtime.py": "2223a5573a3bb6c7fec46e264920f5cacf3dedaffb48f6a732e7f4371380c5e7",
+      "ntu_gifts/dsh/__init__.py": "11855c8b3a9276249180fa9ddf0f4d967d21140d57fff4bc4a638642864483ff",
+      "ntu_gifts/dsh/__main__.py": "b54ebc80f16c01be4b84a44474b3567e61acdf1e694813764869060ed002a762",
+      "ntu_gifts/dsh/adapter.py": "c0184cbecaba44c58d2779ac4972e33912b4c5235b00d52b20ae0eacb1aa93c9",
+      "ntu_gifts/dsh/bridge.py": "f56ef6509cd831ec7de2c61b5331a7496771cf0754fcfaf0b29aeb88fa9be0f9",
+      "ntu_gifts/dsh/bwrap_payload.py": "02b1eb6067e4ab4d283d98148bd6b260bcc9a0ac465dfd490acb93f7ab79542e",
+      "ntu_gifts/dsh/dependencies.py": "e0618ffbb448082e1b5ee5eb6a3ca384375f0ddbf042be72726ee26053227254",
+      "ntu_gifts/dsh/evidence.py": "bb0f5be46a80c4fcaf993e8071b22160fc04d5d871a9e446340a4041a9ef7591",
+      "ntu_gifts/dsh/execution_bwrap.py": "f37c3818c3b38bad1de8885c02f1c644a2e9abf7abe9b31a2823ee4ae941b632",
+      "ntu_gifts/dsh/failure_policy.py": "50f39ce6228e1eaca24278dd46774d13a39ecd2e7773a5c36e952ee70451083f",
+      "ntu_gifts/dsh/late_evidence.py": "5f6231a93f7668107f6d15501177013a64e27af42a904b8a8acba502edcab197",
+      "ntu_gifts/dsh/protocol.py": "9467ea64de8a60f60988c2dd2ddd804efea53a9bedab8e6c1dfa28ef8b963a5d",
+      "ntu_gifts/dsh/reporting.py": "823067b7a99f03cc682fb448aa26992b06b75ee15096567d93d85b31e715e9f9",
+      "ntu_gifts/dsh/runner.py": "082f3878ee2c24ecf15b0b5229e5feb4eab801dbca78fa1fc1c14aa09c8ed42e",
+      "ntu_gifts/dsh/study.py": "57c3495a97f12baae8aa2bb74c2289bfc5a7cbad0cc70205f34e66c734c28008",
+      "ntu_gifts/dsh/worker.py": "4d5ed66f60161f19e197bf702d4acee3aee37a9bfb99ef0a52a448b0de76810f",
+      "ntu_gifts/harness/budget.py": "84e27094f08676328c0899c2abd14097238efde43a5f38c110f28d32bcb2c385",
+      "ntu_gifts/harness/study.py": "7264540b065df45f76d4d6705bf8485f4a9d778b900035efbcfe39a7520abc6c",
+      "ntu_gifts/workspace.py": "2b5295ade967f7b586ec8b44d4f55a31efb50a27cf75290ada0ee00e9c756346",
+      "scripts/setup_dsh.py": "fe835598584191d1a10c680afd769201efc6a7cf689967f681d8bc64c0b914e4"
+    },
+    "evaluator_revision": "92e56c82d1c91b1297d08dd331e449362eca30f8",
+    "latest_controller_commit": "3ced22b417aea99b0956b504794561964dbf7b33"
+  },
+  "tools": [
+    "run_python"
+  ],
+  "underlying_model_weights_verified": false
+}

results/deepseek-v4-pro/dsh/diagnostics.json

@@ -0,0 +1,1303 @@
+{
+  "categories": {
+    "failed": {
+      "compilation_evidence": {
+        "confirmed_final_execution_error": 43,
+        "confirmed_final_execution_timeout": 3,
+        "confirmed_model_failure_without_final_execution": 225
+      },
+      "count": 271,
+      "final_exception_types": {
+        "AttributeError": 1,
+        "IndexError": 3,
+        "KeyError": 4,
+        "RuntimeError": 1,
+        "TypeError": 5,
+        "ValueError": 26,
+        "none": 3,
+        "numpy._core._exceptions._ArrayMemoryError": 1,
+        "numpy.linalg.LinAlgError": 1,
+        "statsmodels.tools.sm_exceptions.MissingDataError": 1
+      },
+      "incomplete_response_finish_reasons": {
+        "['length']": 182
+      },
+      "reasons": {
+        "Expected one fenced Python program": 1,
+        "final_execution_failed": 46,
+        "model_calls_exhausted": 42,
+        "model_response_not_completed": 182
+      }
+    },
+    "uncertain": {
+      "compilation_evidence": {
+        "pending_or_uncertain_task": 30
+      },
+      "count": 30,
+      "final_exception_types": {},
+      "incomplete_response_finish_reasons": {},
+      "reasons": {
+        "bridge_work_cleanup_unconfirmed": 1,
+        "model_budget_or_deadline_exceeded_after_dispatch": 18,
+        "model_outcome_unknown": 6,
+        "provider_exceeded_output_limit": 1,
+        "study_stopped": 4
+      }
+    }
+  },
+  "chosen_task_count": 1000,
+  "failed_task_ids": [
+    1,
+    5,
+    8,
+    23,
+    35,
+    40,
+    41,
+    55,
+    61,
+    69,
+    82,
+    86,
+    87,
+    88,
+    89,
+    93,
+    94,
+    95,
+    96,
+    97,
+    100,
+    101,
+    102,
+    107,
+    115,
+    122,
+    124,
+    126,
+    131,
+    132,
+    133,
+    134,
+    135,
+    136,
+    137,
+    138,
+    139,
+    140,
+    145,
+    147,
+    148,
+    149,
+    150,
+    156,
+    157,
+    159,
+    160,
+    161,
+    165,
+    166,
+    167,
+    168,
+    170,
+    171,
+    172,
+    182,
+    183,
+    186,
+    187,
+    189,
+    191,
+    192,
+    198,
+    199,
+    211,
+    216,
+    236,
+    238,
+    249,
+    251,
+    252,
+    253,
+    254,
+    255,
+    256,
+    257,
+    258,
+    308,
+    320,
+    321,
+    323,
+    324,
+    325,
+    327,
+    331,
+    332,
+    334,
+    336,
+    337,
+    338,
+    339,
+    341,
+    345,
+    349,
+    352,
+    362,
+    365,
+    366,
+    373,
+    374,
+    376,
+    379,
+    380,
+    384,
+    387,
+    388,
+    393,
+    413,
+    414,
+    419,
+    422,
+    426,
+    427,
+    428,
+    431,
+    434,
+    435,
+    438,
+    443,
+    445,
+    446,
+    447,
+    453,
+    454,
+    455,
+    456,
+    458,
+    462,
+    466,
+    467,
+    473,
+    474,
+    475,
+    492,
+    495,
+    496,
+    498,
+    507,
+    509,
+    519,
+    527,
+    529,
+    532,
+    542,
+    543,
+    553,
+    556,
+    559,
+    568,
+    570,
+    571,
+    572,
+    581,
+    592,
+    601,
+    602,
+    603,
+    604,
+    607,
+    608,
+    609,
+    616,
+    618,
+    621,
+    622,
+    623,
+    667,
+    668,
+    675,
+    676,
+    678,
+    718,
+    732,
+    734,
+    735,
+    739,
+    740,
+    743,
+    744,
+    745,
+    754,
+    760,
+    763,
+    766,
+    767,
+    778,
+    779,
+    780,
+    781,
+    785,
+    789,
+    791,
+    794,
+    798,
+    800,
+    801,
+    802,
+    803,
+    804,
+    806,
+    807,
+    834,
+    842,
+    848,
+    852,
+    854,
+    855,
+    859,
+    862,
+    876,
+    878,
+    880,
+    881,
+    882,
+    885,
+    886,
+    887,
+    892,
+    898,
+    900,
+    906,
+    922,
+    923,
+    924,
+    925,
+    926,
+    931,
+    932,
+    934,
+    943,
+    945,
+    967,
+    968,
+    970,
+    971,
+    973,
+    974,
+    975,
+    976,
+    977,
+    979,
+    980,
+    981,
+    982,
+    983,
+    984,
+    985,
+    1001,
+    1008,
+    1009,
+    1010,
+    1016,
+    1019,
+    1033,
+    1040,
+    1042,
+    1060,
+    1064,
+    1081,
+    1082,
+    1084,
+    1091,
+    1094,
+    1099,
+    1109,
+    1112,
+    1116,
+    1118,
+    1119,
+    1122,
+    1123
+  ],
+  "scope": "Audited categories only. Raw error text, prompts, model responses, prediction/reference structures and machine paths are excluded.",
+  "uncertain_task_ids": [
+    57,
+    64,
+    103,
+    105,
+    146,
+    163,
+    250,
+    294,
+    353,
+    378,
+    424,
+    448,
+    560,
+    577,
+    579,
+    590,
+    598,
+    606,
+    611,
+    787,
+    835,
+    883,
+    897,
+    907,
+    913,
+    921,
+    928,
+    952,
+    962,
+    1004
+  ],
+  "unknown_metric_task_ids": {
+    "coefficient_direction": [
+      1,
+      5,
+      8,
+      23,
+      35,
+      40,
+      41,
+      55,
+      57,
+      61,
+      64,
+      69,
+      82,
+      86,
+      87,
+      88,
+      89,
+      93,
+      94,
+      95,
+      96,
+      97,
+      100,
+      101,
+      102,
+      103,
+      105,
+      107,
+      115,
+      122,
+      124,
+      126,
+      131,
+      132,
+      133,
+      134,
+      135,
+      136,
+      137,
+      138,
+      139,
+      140,
+      145,
+      146,
+      147,
+      148,
+      149,
+      150,
+      156,
+      157,
+      159,
+      160,
+      161,
+      163,
+      165,
+      166,
+      167,
+      168,
+      170,
+      171,
+      172,
+      182,
+      183,
+      186,
+      187,
+      189,
+      191,
+      192,
+      198,
+      199,
+      211,
+      216,
+      236,
+      238,
+      249,
+      250,
+      251,
+      252,
+      253,
+      254,
+      255,
+      256,
+      257,
+      258,
+      294,
+      308,
+      320,
+      321,
+      323,
+      324,
+      325,
+      327,
+      331,
+      332,
+      334,
+      336,
+      337,
+      338,
+      339,
+      341,
+      345,
+      349,
+      352,
+      353,
+      362,
+      365,
+      366,
+      373,
+      374,
+      376,
+      378,
+      379,
+      380,
+      384,
+      387,
+      388,
+      393,
+      413,
+      414,
+      419,
+      422,
+      424,
+      426,
+      427,
+      428,
+      431,
+      434,
+      435,
+      438,
+      443,
+      445,
+      446,
+      447,
+      448,
+      453,
+      454,
+      455,
+      456,
+      458,
+      462,
+      466,
+      467,
+      473,
+      474,
+      475,
+      492,
+      495,
+      496,
+      498,
+      507,
+      509,
+      519,
+      527,
+      529,
+      532,
+      542,
+      543,
+      553,
+      556,
+      559,
+      560,
+      568,
+      570,
+      571,
+      572,
+      577,
+      579,
+      581,
+      590,
+      592,
+      598,
+      601,
+      602,
+      603,
+      604,
+      606,
+      607,
+      608,
+      609,
+      611,
+      616,
+      618,
+      621,
+      622,
+      623,
+      667,
+      668,
+      675,
+      676,
+      678,
+      718,
+      732,
+      734,
+      735,
+      739,
+      740,
+      743,
+      744,
+      745,
+      754,
+      760,
+      763,
+      766,
+      767,
+      778,
+      779,
+      780,
+      781,
+      785,
+      787,
+      789,
+      791,
+      794,
+      798,
+      800,
+      801,
+      802,
+      803,
+      804,
+      806,
+      807,
+      834,
+      835,
+      842,
+      848,
+      852,
+      854,
+      855,
+      859,
+      862,
+      876,
+      878,
+      880,
+      881,
+      882,
+      883,
+      885,
+      886,
+      887,
+      892,
+      897,
+      898,
+      900,
+      906,
+      907,
+      913,
+      921,
+      922,
+      923,
+      924,
+      925,
+      926,
+      928,
+      931,
+      932,
+      934,
+      943,
+      945,
+      952,
+      962,
+      967,
+      968,
+      970,
+      971,
+      973,
+      974,
+      975,
+      976,
+      977,
+      979,
+      980,
+      981,
+      982,
+      983,
+      984,
+      985,
+      1001,
+      1004,
+      1008,
+      1009,
+      1010,
+      1016,
+      1019,
+      1033,
+      1040,
+      1042,
+      1050,
+      1060,
+      1064,
+      1081,
+      1082,
+      1084,
+      1091,
+      1094,
+      1099,
+      1109,
+      1112,
+      1116,
+      1118,
+      1119,
+      1122,
+      1123
+    ],
+    "compilation_success": [
+      57,
+      64,
+      103,
+      105,
+      146,
+      163,
+      250,
+      294,
+      353,
+      378,
+      424,
+      448,
+      560,
+      577,
+      579,
+      590,
+      598,
+      606,
+      611,
+      787,
+      835,
+      883,
+      897,
+      907,
+      913,
+      921,
+      928,
+      952,
+      962,
+      1004
+    ],
+    "partial_replication": [
+      1,
+      5,
+      8,
+      23,
+      35,
+      40,
+      41,
+      55,
+      57,
+      61,
+      64,
+      69,
+      82,
+      86,
+      87,
+      88,
+      89,
+      93,
+      94,
+      95,
+      96,
+      97,
+      100,
+      101,
+      102,
+      103,
+      105,
+      107,
+      115,
+      122,
+      124,
+      126,
+      131,
+      132,
+      133,
+      134,
+      135,
+      136,
+      137,
+      138,
+      139,
+      140,
+      145,
+      146,
+      147,
+      148,
+      149,
+      150,
+      156,
+      157,
+      159,
+      160,
+      161,
+      163,
+      165,
+      166,
+      167,
+      168,
+      170,
+      171,
+      172,
+      182,
+      183,
+      186,
+      187,
+      189,
+      191,
+      192,
+      198,
+      199,
+      211,
+      216,
+      236,
+      238,
+      249,
+      250,
+      251,
+      252,
+      253,
+      254,
+      255,
+      256,
+      257,
+      258,
+      294,
+      308,
+      320,
+      321,
+      323,
+      324,
+      325,
+      327,
+      331,
+      332,
+      334,
+      336,
+      337,
+      338,
+      339,
+      341,
+      345,
+      349,
+      352,
+      353,
+      362,
+      365,
+      366,
+      373,
+      374,
+      376,
+      378,
+      379,
+      380,
+      384,
+      387,
+      388,
+      393,
+      413,
+      414,
+      419,
+      422,
+      424,
+      426,
+      427,
+      428,
+      431,
+      434,
+      435,
+      438,
+      443,
+      445,
+      446,
+      447,
+      448,
+      453,
+      454,
+      455,
+      456,
+      458,
+      462,
+      466,
+      467,
+      473,
+      474,
+      475,
+      492,
+      495,
+      496,
+      498,
+      507,
+      509,
+      519,
+      527,
+      529,
+      532,
+      542,
+      543,
+      553,
+      556,
+      559,
+      560,
+      568,
+      570,
+      571,
+      572,
+      577,
+      579,
+      581,
+      590,
+      592,
+      598,
+      601,
+      602,
+      603,
+      604,
+      606,
+      607,
+      608,
+      609,
+      611,
+      616,
+      618,
+      621,
+      622,
+      623,
+      667,
+      668,
+      675,
+      676,
+      678,
+      718,
+      732,
+      734,
+      735,
+      739,
+      740,
+      743,
+      744,
+      745,
+      754,
+      760,
+      763,
+      766,
+      767,
+      778,
+      779,
+      780,
+      781,
+      785,
+      787,
+      789,
+      791,
+      794,
+      798,
+      800,
+      801,
+      802,
+      803,
+      804,
+      806,
+      807,
+      834,
+      835,
+      842,
+      848,
+      852,
+      854,
+      855,
+      859,
+      862,
+      876,
+      878,
+      880,
+      881,
+      882,
+      883,
+      885,
+      886,
+      887,
+      892,
+      897,
+      898,
+      900,
+      906,
+      907,
+      913,
+      921,
+      922,
+      923,
+      924,
+      925,
+      926,
+      928,
+      931,
+      932,
+      934,
+      943,
+      945,
+      952,
+      962,
+      967,
+      968,
+      970,
+      971,
+      973,
+      974,
+      975,
+      976,
+      977,
+      979,
+      980,
+      981,
+      982,
+      983,
+      984,
+      985,
+      1001,
+      1004,
+      1008,
+      1009,
+      1010,
+      1016,
+      1019,
+      1033,
+      1040,
+      1042,
+      1050,
+      1060,
+      1064,
+      1081,
+      1082,
+      1084,
+      1091,
+      1094,
+      1099,
+      1109,
+      1112,
+      1116,
+      1118,
+      1119,
+      1122,
+      1123
+    ],
+    "significance_level": [
+      1,
+      5,
+      8,
+      23,
+      35,
+      40,
+      41,
+      55,
+      57,
+      61,
+      64,
+      69,
+      82,
+      86,
+      87,
+      88,
+      89,
+      93,
+      94,
+      95,
+      96,
+      97,
+      100,
+      101,
+      102,
+      103,
+      105,
+      107,
+      115,
+      122,
+      124,
+      126,
+      131,
+      132,
+      133,
+      134,
+      135,
+      136,
+      137,
+      138,
+      139,
+      140,
+      145,
+      146,
+      147,
+      148,
+      149,
+      150,
+      156,
+      157,
+      159,
+      160,
+      161,
+      163,
+      165,
+      166,
+      167,
+      168,
+      170,
+      171,
+      172,
+      182,
+      183,
+      186,
+      187,
+      189,
+      191,
+      192,
+      198,
+      199,
+      211,
+      216,
+      236,
+      238,
+      249,
+      250,
+      251,
+      252,
+      253,
+      254,
+      255,
+      256,
+      257,
+      258,
+      294,
+      308,
+      320,
+      321,
+      323,
+      324,
+      325,
+      327,
+      331,
+      332,
+      334,
+      336,
+      337,
+      338,
+      339,
+      341,
+      345,
+      349,
+      352,
+      353,
+      362,
+      365,
+      366,
+      373,
+      374,
+      376,
+      378,
+      379,
+      380,
+      384,
+      387,
+      388,
+      393,
+      413,
+      414,
+      419,
+      422,
+      424,
+      426,
+      427,
+      428,
+      431,
+      434,
+      435,
+      438,
+      443,
+      445,
+      446,
+      447,
+      448,
+      453,
+      454,
+      455,
+      456,
+      458,
+      462,
+      466,
+      467,
+      473,
+      474,
+      475,
+      492,
+      495,
+      496,
+      498,
+      507,
+      509,
+      519,
+      527,
+      529,
+      532,
+      542,
+      543,
+      553,
+      556,
+      559,
+      560,
+      568,
+      570,
+      571,
+      572,
+      577,
+      579,
+      581,
+      590,
+      592,
+      598,
+      601,
+      602,
+      603,
+      604,
+      606,
+      607,
+      608,
+      609,
+      611,
+      616,
+      618,
+      621,
+      622,
+      623,
+      667,
+      668,
+      675,
+      676,
+      678,
+      718,
+      732,
+      734,
+      735,
+      739,
+      740,
+      743,
+      744,
+      745,
+      754,
+      760,
+      763,
+      766,
+      767,
+      778,
+      779,
+      780,
+      781,
+      785,
+      787,
+      789,
+      791,
+      794,
+      798,
+      800,
+      801,
+      802,
+      803,
+      804,
+      806,
+      807,
+      834,
+      835,
+      842,
+      848,
+      852,
+      854,
+      855,
+      859,
+      862,
+      876,
+      878,
+      880,
+      881,
+      882,
+      883,
+      885,
+      886,
+      887,
+      892,
+      897,
+      898,
+      900,
+      906,
+      907,
+      913,
+      915,
+      917,
+      921,
+      922,
+      923,
+      924,
+      925,
+      926,
+      928,
+      931,
+      932,
+      934,
+      943,
+      945,
+      952,
+      962,
+      967,
+      968,
+      970,
+      971,
+      973,
+      974,
+      975,
+      976,
+      977,
+      979,
+      980,
+      981,
+      982,
+      983,
+      984,
+      985,
+      1001,
+      1004,
+      1008,
+      1009,
+      1010,
+      1016,
+      1019,
+      1033,
+      1040,
+      1042,
+      1050,
+      1060,
+      1064,
+      1081,
+      1082,
+      1084,
+      1091,
+      1094,
+      1099,
+      1109,
+      1112,
+      1116,
+      1118,
+      1119,
+      1122,
+      1123
+    ]
+  }
+}

results/deepseek-v4-pro/dsh/export-verification.json

@@ -0,0 +1,32 @@
+{
+  "all_attempts": 1087,
+  "kind": "dsh-public-export-verification-v1",
+  "known_and_unknown_costs_preserved": true,
+  "local_flags_read_from_original_sealed_scoring": true,
+  "metric_counts_verified_against_exported_flags": true,
+  "model_calls_performed": 0,
+  "official_parity_verified": false,
+  "original_attempts": 1000,
+  "output_sha256": {
+    "README.md": "ccf9f54ce467c8a977074498eeb86a2ce1b7b69a73b370b025663f9582a8d991",
+    "accounting.json": "6f92c5376ab6b96b9ab5ddab6faff88cd792d0ec999f1cc96cdd48e32302c9e1",
+    "all-attempts.jsonl": "33faf46bce3d4a152bb04cd3eb7cded2d948b2bf5ec2beac49941b145e92c522",
+    "config.json": "df322f13761e06dc78cb553c5dac347ce4361599937af4017a902c56a7df9d77",
+    "diagnostics.json": "d6500c012259ed5e3b514d6f62597fdcadfb7548e41308eed442a1b130f0e336",
+    "first-pass-results.jsonl": "153d15ff14c3dba6ece8fd9c81ae2d9fa1f0b6a240e90efd6b1d6e46f40893e2",
+    "first-pass-summary.json": "b264e4796ea32220a7c67fe84d990ac6e040784f7331c662e66ca5d91c71e9c1",
+    "provenance.json": "8e03b3fd76310ab80643f51ce791e85545062d2d85ac3c6121cc1143ebfb5a28",
+    "results.csv": "954a6adff76d6bae31c77cd969b17fa5203ae7ba38abf76130c6f066e39a9c58",
+    "results.jsonl": "b12427d5f4476351e571e4b29f61e0463540a1701bec51e642753d717dbc1df2",
+    "summary.json": "62936f76b1ca0de5e761d1ea1f894343695503ce934bf0213dc9fde1ea72a782"
+  },
+  "private_fields_allowlisted_out": true,
+  "rescoring_performed": false,
+  "selected_tasks": 1000,
+  "selection_verified_against_frozen_plan": true,
+  "source_manifest_sha256": "89b41604d0f1fd5c8b35b5a41f2a83b24c27f2ea093444c64b57ab34a6216b0c",
+  "supplementary_attempts": 87,
+  "underlying_model_weights_verified": false,
+  "unique_selected_tasks": 1000,
+  "unknown_usage_attempts": 38
+}

results/deepseek-v4-pro/dsh/first-pass-summary.json

@@ -0,0 +1,113 @@
+{
+  "all_slots_sealed": true,
+  "arm": "dsh",
+  "assumptions": {
+    "attempt_selection": "The caller supplies one row per task under its frozen attempt policy.",
+    "coefficient_boundary": "5% includes equality. Numeric decimal representations are compared exactly without epsilon.",
+    "compilation_evidence": "Caller supplies True, False, or None from verified execution evidence, including exit status, timeout, termination confirmation, and execution errors. Missing or uncertain evidence maps to None.",
+    "denominator": "All planned tasks remain in every denominator. Confirmed compilation failures are False; missing or uncertain execution evidence is unknown. For the three result metrics, failed execution, missing predictions, unscored rows, and unsupported reference fields are unknown. Unknowns contribute no successes and are counted separately.",
+    "field_validation": "Each metric requires only its own finite numeric fields. Booleans, numeric strings, and inequalities are not coerced. Standard errors do not affect any of these three formulas.",
+    "official_rules": "Webpage interpretation only; official scorer parity is not verified.",
+    "prediction_evidence": "Only execution_status=succeeded and scoring_status=scored rows are evaluated. The current caller requires a complete valid prediction triplet; invalid outputs are not recovered here.",
+    "significance_categories": "Four categories use strict .01, .05, and .1 boundaries. Both p-values must be numeric and within [0, 1]. These boundaries and the absent direction gate are explicit local choices.",
+    "zero_reference_coefficient": "A zero reference coefficient is unknown for relative error and positive/negative direction; it does not block significance. A zero prediction against a nonzero reference is incorrect."
+  },
+  "complete": false,
+  "dataset_revision": "59f9512a38e594528807744214a60ee00367434e",
+  "definitions": {
+    "coefficient_direction": "Matching strictly positive or strictly negative coefficient signs.",
+    "compilation_success": "Trusted evidence of complete code execution; no prediction JSON gate.",
+    "partial_replication": "Coefficient relative error <= .05; no standard-error or p-value gate.",
+    "significance_level": "Equal p-value category: p < .01, p < .05, p < .1, otherwise; no direction gate."
+  },
+  "display_name": "DeepSeek API (deepseek-v4-pro alias) + DSH",
+  "display_role": "supporting_original_report",
+  "evidence_class": "original_first_pass",
+  "generated_at": "2026-09-19T18:18:48.070003+00:00",
+  "kind": "inferencenet-offline-leaderboard-export",
+  "legacy_local_paper_export_status": "not_exported_for_original_first_pass",
+  "metric_profile": "hf-leaderboard-v1",
+  "model": "deepseek-v4-pro",
+  "model_alias": "deepseek-v4-pro",
+  "official_parity_verified": false,
+  "original_protocol_complete": false,
+  "publication_kind": "research_results_with_technical_recovery",
+  "result": {
+    "columns": [
+      "Model ID",
+      "Compilation Success",
+      "Partial Replication",
+      "Correct Coefficient Direction",
+      "Significant Level Correctness"
+    ],
+    "definite_outcome_count": 928,
+    "expected_count": 1000,
+    "invalid_evidence_count": 0,
+    "metrics": {
+      "coefficient_direction": {
+        "assessable": 661,
+        "count": 648,
+        "coverage_percent": 66.1,
+        "denominator": 1000,
+        "failure_count": 13,
+        "rate": 0.648,
+        "score": 64.8,
+        "unknown_count": 339
+      },
+      "compilation_success": {
+        "assessable": 900,
+        "count": 662,
+        "coverage_percent": 90.0,
+        "denominator": 1000,
+        "failure_count": 238,
+        "rate": 0.662,
+        "score": 66.2,
+        "unknown_count": 100
+      },
+      "partial_replication": {
+        "assessable": 661,
+        "count": 527,
+        "coverage_percent": 66.1,
+        "denominator": 1000,
+        "failure_count": 134,
+        "rate": 0.527,
+        "score": 52.7,
+        "unknown_count": 339
+      },
+      "significance_level": {
+        "assessable": 659,
+        "count": 566,
+        "coverage_percent": 65.9,
+        "denominator": 1000,
+        "failure_count": 93,
+        "rate": 0.566,
+        "score": 56.6,
+        "unknown_count": 341
+      }
+    },
+    "resolved_count": 1000,
+    "row": {
+      "Compilation Success": 66.2,
+      "Correct Coefficient Direction": 64.8,
+      "Model ID": "DeepSeek API (deepseek-v4-pro alias) + DSH",
+      "Partial Replication": 52.7,
+      "Significant Level Correctness": 56.6
+    },
+    "sealed_count": 1000,
+    "task_counts": {
+      "completed": 1000,
+      "failed": 266,
+      "invalid_evidence": 0,
+      "not_started": 0,
+      "planned": 1000,
+      "succeeded": 662,
+      "unknown": 72,
+      "unsealed": 0
+    },
+    "task_unknown_count": 72,
+    "valid_prediction_count": 662
+  },
+  "schema_version": 3,
+  "underlying_model_weights_verified": false,
+  "verification_scope": "Preserved verified flags and result/score hashes; no new inference, execution or rescoring."
+}

results/deepseek-v4-pro/dsh/manifest.sha256

@@ -0,0 +1,12 @@
+ccf9f54ce467c8a977074498eeb86a2ce1b7b69a73b370b025663f9582a8d991  README.md
+6f92c5376ab6b96b9ab5ddab6faff88cd792d0ec999f1cc96cdd48e32302c9e1  accounting.json
+33faf46bce3d4a152bb04cd3eb7cded2d948b2bf5ec2beac49941b145e92c522  all-attempts.jsonl
+df322f13761e06dc78cb553c5dac347ce4361599937af4017a902c56a7df9d77  config.json
+d6500c012259ed5e3b514d6f62597fdcadfb7548e41308eed442a1b130f0e336  diagnostics.json
+8d7cae2ea2f33130636367f5a5e8f92b5df0471aa9889aa54094e2d1664dd48b  export-verification.json
+153d15ff14c3dba6ece8fd9c81ae2d9fa1f0b6a240e90efd6b1d6e46f40893e2  first-pass-results.jsonl
+b264e4796ea32220a7c67fe84d990ac6e040784f7331c662e66ca5d91c71e9c1  first-pass-summary.json
+8e03b3fd76310ab80643f51ce791e85545062d2d85ac3c6121cc1143ebfb5a28  provenance.json
+954a6adff76d6bae31c77cd969b17fa5203ae7ba38abf76130c6f066e39a9c58  results.csv
+b12427d5f4476351e571e4b29f61e0463540a1701bec51e642753d717dbc1df2  results.jsonl
+62936f76b1ca0de5e761d1ea1f894343695503ce934bf0213dc9fde1ea72a782  summary.json

results/deepseek-v4-pro/dsh/provenance.json

@@ -0,0 +1,454 @@
+{
+  "arm": "dsh",
+  "kind": "dsh-public-recovery-provenance-v1",
+  "model": "deepseek-v4-pro",
+  "model_identity": {
+    "api_request_alias": "deepseek-v4-pro",
+    "conflicting_official_descriptions": [
+      {
+        "claim": "Announcement says alias routes to V4.1 Flash from 2026-09-14 12:00 Beijing time until V4.1 Pro launches.",
+        "url": "https://api-docs.deepseek.com/zh-cn/news/news260910/"
+      },
+      {
+        "claim": "Model table lists alias as DeepSeek-V4-Pro-0813.",
+        "url": "https://api-docs.deepseek.com/quick_start/pricing/"
+      }
+    ],
+    "interpretation": "Conflicting official descriptions; neither underlying weight version is independently established.",
+    "successful_response_model_label": "deepseek-v4-pro",
+    "underlying_model_weights_verified": false
+  },
+  "original_artifacts_modified": false,
+  "paid_calls_performed_for_export": 0,
+  "plan_id": "15486e52db982d80aaa0f6582f3d67469757919e5b8605e28ce7e98698d15ac3",
+  "plan_sha256": "4f903cda0cc6cd0f01f194d743d687eb9cc29fa84dcb7f1d68d35f76d9b21fe9",
+  "rescoring_performed": false,
+  "selected_internal_results_sha256": "12861e6fdfe58c5432b08c09fb9f14306c3b29c55e49cdc32dfb0fb1526b1343",
+  "selection": {
+    "best_of_attempts": false,
+    "infrastructure_tasks": 60,
+    "maximum_supplementary_attempts_per_task": 1,
+    "original_attempts": 1000,
+    "original_results_modified": false,
+    "policy": "Freeze eligibility before supplementary outcomes; select attempt 2 regardless of its score or status. Never best-of or fall back to a better attempt.",
+    "schema_amendment": {
+      "exact_prompt_parity": false,
+      "generation.prompt_profile": {
+        "from": "raw-role-schema",
+        "to": "raw-role"
+      },
+      "tools_and_per_attempt_budgets_changed": false
+    },
+    "schema_tasks": 27,
+    "selected_primary_attempts": 913,
+    "selected_supplementary_attempts": 87,
+    "supplementary_attempts": 87,
+    "total_attempts_charged": 1087
+  },
+  "source_hashes": {
+    "failure_audit": "1a75f87f904026051d0e3eca1cc3d8cfe68bbc6828e7de071b36e72be7d2ee47",
+    "legacy_flags_export": "3f22a49f736a0679aef8dc758521c10f9ec8e4505fbab689a4a2b5f28021831b",
+    "legacy_metrics_audit": "abb056be811678b198f8f95c8961c4766ee3543dec9493441946b8113a276baf",
+    "public_exporter": "ea1f1f27ae1f06ff0419bf60157222c9d6ffa6b6d58ded2b2e5f980ddfab151d",
+    "recovered_manifest": "89b41604d0f1fd5c8b35b5a41f2a83b24c27f2ea093444c64b57ab34a6216b0c"
+  },
+  "sources": {
+    "infrastructure": {
+      "bindings": [
+        {
+          "continuation_receipt_sha256": null,
+          "frozen_identity": {
+            "inputs.manifest.json": "58e63dddeed8f6bb39de2abd4c37dc1789fd101a876628229988cd24bdf5495d",
+            "protocol.json": "08e2cb843107daa4f85be7d2f9d7b21899b983b4e29a5b8e9193bce6e9bec058",
+            "schedule.json": "670c71226010832fe118390cb0cdec9475f4262bd2e3a235f44681b908bcdb75",
+            "study.json": "63694ebaf42f4387f264281d57214ecc1ba40742bc5f91a72d452cb8d0e400f7"
+          },
+          "run_id": "infrastructure",
+          "source_commit": "3ced22b417aea99b0956b504794561964dbf7b33",
+          "source_identity": {
+            "dsh/__init__.py": "8e24d82c627405e0a8f9de89ebdbf1ce23db63337ef44aacd4a789cea9697908",
+            "dsh/profile.patch.yml": "81a92895c5f3c4c85bb602e3349a509262eb144c63a566331a43fa73081e2731",
+            "dsh/requirements-controller.txt": "aee6d7d23cbeecc3a101ec56687af9cc7aecbd6e96443295042a5c36307587d4",
+            "dsh/requirements.lock.txt": "ed57fde608bf1cdabaa1fc5d23e5b254983606652be2f732ce4ed2f2c69ea7a1",
+            "dsh/run-python.mjs": "3681bcbdfc6fa10cbb0742fadcc8f3f94a745b428662c94214b370b01bbea67e",
+            "dsh/runtime.lock.json": "e241e17af63e1f2b5ebaab3845a40c8f35c8643c3b5622b778541663428aba44",
+            "dsh/runtime.py": "2223a5573a3bb6c7fec46e264920f5cacf3dedaffb48f6a732e7f4371380c5e7",
+            "ntu_gifts/dsh/__init__.py": "11855c8b3a9276249180fa9ddf0f4d967d21140d57fff4bc4a638642864483ff",
+            "ntu_gifts/dsh/__main__.py": "b54ebc80f16c01be4b84a44474b3567e61acdf1e694813764869060ed002a762",
+            "ntu_gifts/dsh/adapter.py": "c0184cbecaba44c58d2779ac4972e33912b4c5235b00d52b20ae0eacb1aa93c9",
+            "ntu_gifts/dsh/bridge.py": "f56ef6509cd831ec7de2c61b5331a7496771cf0754fcfaf0b29aeb88fa9be0f9",
+            "ntu_gifts/dsh/bwrap_payload.py": "02b1eb6067e4ab4d283d98148bd6b260bcc9a0ac465dfd490acb93f7ab79542e",
+            "ntu_gifts/dsh/dependencies.py": "e0618ffbb448082e1b5ee5eb6a3ca384375f0ddbf042be72726ee26053227254",
+            "ntu_gifts/dsh/evidence.py": "bb0f5be46a80c4fcaf993e8071b22160fc04d5d871a9e446340a4041a9ef7591",
+            "ntu_gifts/dsh/execution_bwrap.py": "f37c3818c3b38bad1de8885c02f1c644a2e9abf7abe9b31a2823ee4ae941b632",
+            "ntu_gifts/dsh/failure_policy.py": "50f39ce6228e1eaca24278dd46774d13a39ecd2e7773a5c36e952ee70451083f",
+            "ntu_gifts/dsh/late_evidence.py": "5f6231a93f7668107f6d15501177013a64e27af42a904b8a8acba502edcab197",
+            "ntu_gifts/dsh/protocol.py": "9467ea64de8a60f60988c2dd2ddd804efea53a9bedab8e6c1dfa28ef8b963a5d",
+            "ntu_gifts/dsh/reporting.py": "823067b7a99f03cc682fb448aa26992b06b75ee15096567d93d85b31e715e9f9",
+            "ntu_gifts/dsh/runner.py": "082f3878ee2c24ecf15b0b5229e5feb4eab801dbca78fa1fc1c14aa09c8ed42e",
+            "ntu_gifts/dsh/study.py": "57c3495a97f12baae8aa2bb74c2289bfc5a7cbad0cc70205f34e66c734c28008",
+            "ntu_gifts/dsh/worker.py": "4d5ed66f60161f19e197bf702d4acee3aee37a9bfb99ef0a52a448b0de76810f",
+            "ntu_gifts/harness/budget.py": "84e27094f08676328c0899c2abd14097238efde43a5f38c110f28d32bcb2c385",
+            "ntu_gifts/harness/study.py": "7264540b065df45f76d4d6705bf8485f4a9d778b900035efbcfe39a7520abc6c",
+            "ntu_gifts/workspace.py": "2b5295ade967f7b586ec8b44d4f55a31efb50a27cf75290ada0ee00e9c756346",
+            "scripts/setup_dsh.py": "fe835598584191d1a10c680afd769201efc6a7cf689967f681d8bc64c0b914e4"
+          }
+        },
+        {
+          "continuation_receipt_sha256": "775f69ede2c79404271e44fe2832251cad21ba233c5add0a6fb6fd31d5c6f5cf",
+          "frozen_identity": {
+            "inputs.manifest.json": "f325f884f3c1e8fe67934f73c1274c9e10713440e15fc397d93099b35a513833",
+            "protocol.json": "e85ff3bc2c4c8eef0deb48567a2e13642cd546bd64296621fe3d10640a629ffb",
+            "schedule.json": "d2cc1c2d9d23e6732e78879872970d927351c6da0d73ba9fe64a614c1a756370",
+            "study.json": "78afa10631db305ba2edc76dd5a4ee664fa2b71abc592b4745cfeae7d908b749"
+          },
+          "run_id": "infrastructure-auto01",
+          "source_commit": "3ced22b417aea99b0956b504794561964dbf7b33",
+          "source_identity": {
+            "dsh/__init__.py": "8e24d82c627405e0a8f9de89ebdbf1ce23db63337ef44aacd4a789cea9697908",
+            "dsh/profile.patch.yml": "81a92895c5f3c4c85bb602e3349a509262eb144c63a566331a43fa73081e2731",
+            "dsh/requirements-controller.txt": "aee6d7d23cbeecc3a101ec56687af9cc7aecbd6e96443295042a5c36307587d4",
+            "dsh/requirements.lock.txt": "ed57fde608bf1cdabaa1fc5d23e5b254983606652be2f732ce4ed2f2c69ea7a1",
+            "dsh/run-python.mjs": "3681bcbdfc6fa10cbb0742fadcc8f3f94a745b428662c94214b370b01bbea67e",
+            "dsh/runtime.lock.json": "e241e17af63e1f2b5ebaab3845a40c8f35c8643c3b5622b778541663428aba44",
+            "dsh/runtime.py": "2223a5573a3bb6c7fec46e264920f5cacf3dedaffb48f6a732e7f4371380c5e7",
+            "ntu_gifts/dsh/__init__.py": "11855c8b3a9276249180fa9ddf0f4d967d21140d57fff4bc4a638642864483ff",
+            "ntu_gifts/dsh/__main__.py": "b54ebc80f16c01be4b84a44474b3567e61acdf1e694813764869060ed002a762",
+            "ntu_gifts/dsh/adapter.py": "c0184cbecaba44c58d2779ac4972e33912b4c5235b00d52b20ae0eacb1aa93c9",
+            "ntu_gifts/dsh/bridge.py": "f56ef6509cd831ec7de2c61b5331a7496771cf0754fcfaf0b29aeb88fa9be0f9",
+            "ntu_gifts/dsh/bwrap_payload.py": "02b1eb6067e4ab4d283d98148bd6b260bcc9a0ac465dfd490acb93f7ab79542e",
+            "ntu_gifts/dsh/dependencies.py": "e0618ffbb448082e1b5ee5eb6a3ca384375f0ddbf042be72726ee26053227254",
+            "ntu_gifts/dsh/evidence.py": "bb0f5be46a80c4fcaf993e8071b22160fc04d5d871a9e446340a4041a9ef7591",
+            "ntu_gifts/dsh/execution_bwrap.py": "f37c3818c3b38bad1de8885c02f1c644a2e9abf7abe9b31a2823ee4ae941b632",
+            "ntu_gifts/dsh/failure_policy.py": "50f39ce6228e1eaca24278dd46774d13a39ecd2e7773a5c36e952ee70451083f",
+            "ntu_gifts/dsh/late_evidence.py": "5f6231a93f7668107f6d15501177013a64e27af42a904b8a8acba502edcab197",
+            "ntu_gifts/dsh/protocol.py": "9467ea64de8a60f60988c2dd2ddd804efea53a9bedab8e6c1dfa28ef8b963a5d",
+            "ntu_gifts/dsh/reporting.py": "823067b7a99f03cc682fb448aa26992b06b75ee15096567d93d85b31e715e9f9",
+            "ntu_gifts/dsh/runner.py": "082f3878ee2c24ecf15b0b5229e5feb4eab801dbca78fa1fc1c14aa09c8ed42e",
+            "ntu_gifts/dsh/study.py": "57c3495a97f12baae8aa2bb74c2289bfc5a7cbad0cc70205f34e66c734c28008",
+            "ntu_gifts/dsh/worker.py": "4d5ed66f60161f19e197bf702d4acee3aee37a9bfb99ef0a52a448b0de76810f",
+            "ntu_gifts/harness/budget.py": "84e27094f08676328c0899c2abd14097238efde43a5f38c110f28d32bcb2c385",
+            "ntu_gifts/harness/study.py": "7264540b065df45f76d4d6705bf8485f4a9d778b900035efbcfe39a7520abc6c",
+            "ntu_gifts/workspace.py": "2b5295ade967f7b586ec8b44d4f55a31efb50a27cf75290ada0ee00e9c756346",
+            "scripts/setup_dsh.py": "fe835598584191d1a10c680afd769201efc6a7cf689967f681d8bc64c0b914e4"
+          }
+        }
+      ],
+      "recovery_receipt_sha256": "d14de0ebd364f8ad6b2f1d639027bb7d5f80784a70ec76f626044d7651d9940c",
+      "report_sha256": "4746df24341152fc33b21fbb25eb43e5055e16835b53548aa5e3c471f235bd4f"
+    },
+    "primary": {
+      "bindings": [
+        {
+          "continuation_receipt_sha256": null,
+          "frozen_identity": {
+            "inputs.manifest.json": "f03642df73a8ceeca15f61d2ad329291b896f19148e134aa0cd9edfb14b04ccf",
+            "protocol.json": "04528e17dc02f945fb8d673925d66ddc0d51bd0f653751001b43f20dc80b263e",
+            "schedule.json": "7144a62e17ebfd6920a1de5f1bc49f4b610a81c95cdf4528dabf451df187da26",
+            "study.json": "8c1b83b2b8af544e78423e0ff3f76a1e0725065b465a424416bd14e2788494bc"
+          },
+          "run_id": "dsh-v4-pro-1000-20260918",
+          "source_commit": "5079157d791e2213e6f45a04f82182cc8c700f89",
+          "source_identity": {
+            "dsh/__init__.py": "8e24d82c627405e0a8f9de89ebdbf1ce23db63337ef44aacd4a789cea9697908",
+            "dsh/profile.patch.yml": "418253f372e18e8e64d57aab19c0d28bdb01ca0eb0a9092cbf1953602066edb6",
+            "dsh/requirements-controller.txt": "aee6d7d23cbeecc3a101ec56687af9cc7aecbd6e96443295042a5c36307587d4",
+            "dsh/requirements.lock.txt": "ed57fde608bf1cdabaa1fc5d23e5b254983606652be2f732ce4ed2f2c69ea7a1",
+            "dsh/run-python.mjs": "3681bcbdfc6fa10cbb0742fadcc8f3f94a745b428662c94214b370b01bbea67e",
+            "dsh/runtime.lock.json": "e241e17af63e1f2b5ebaab3845a40c8f35c8643c3b5622b778541663428aba44",
+            "dsh/runtime.py": "994b59ab0620f8f0210e89572cd053b7d26d347358c73ad105ddce2ccb8b3119",
+            "ntu_gifts/dsh/__init__.py": "11855c8b3a9276249180fa9ddf0f4d967d21140d57fff4bc4a638642864483ff",
+            "ntu_gifts/dsh/__main__.py": "b54ebc80f16c01be4b84a44474b3567e61acdf1e694813764869060ed002a762",
+            "ntu_gifts/dsh/adapter.py": "cfd0cbea8fe52ff9a41077c8a8ffeead186ff42322b76aa34d77b2e32ca24815",
+            "ntu_gifts/dsh/bridge.py": "2b8a440717456012f51201a557f6ca4f86947a75ce5498cb0f473700d8ee612a",
+            "ntu_gifts/dsh/bwrap_payload.py": "02b1eb6067e4ab4d283d98148bd6b260bcc9a0ac465dfd490acb93f7ab79542e",
+            "ntu_gifts/dsh/dependencies.py": "e0618ffbb448082e1b5ee5eb6a3ca384375f0ddbf042be72726ee26053227254",
+            "ntu_gifts/dsh/evidence.py": "a3ad48ec6163ddd2844265272f487d733a9f38f7ee886bf361ec240f5a69d224",
+            "ntu_gifts/dsh/execution_bwrap.py": "f37c3818c3b38bad1de8885c02f1c644a2e9abf7abe9b31a2823ee4ae941b632",
+            "ntu_gifts/dsh/protocol.py": "9467ea64de8a60f60988c2dd2ddd804efea53a9bedab8e6c1dfa28ef8b963a5d",
+            "ntu_gifts/dsh/reporting.py": "6fa53a97eec2d89c9eb2793e603564a7c00f65b8f6b5c15c4b043de28341ef0a",
+            "ntu_gifts/dsh/runner.py": "3ae5c3495e63fbf7266bfa071cc79a969fc8f3a8494e2fc86f98ec0f3d0ac534",
+            "ntu_gifts/dsh/study.py": "e0ba44f4fa4be343b1eae4c56f4c5db7e0424da350e371c5e93840883295683f",
+            "ntu_gifts/dsh/worker.py": "4d5ed66f60161f19e197bf702d4acee3aee37a9bfb99ef0a52a448b0de76810f",
+            "ntu_gifts/harness/budget.py": "84e27094f08676328c0899c2abd14097238efde43a5f38c110f28d32bcb2c385",
+            "ntu_gifts/harness/study.py": "7264540b065df45f76d4d6705bf8485f4a9d778b900035efbcfe39a7520abc6c",
+            "ntu_gifts/workspace.py": "2b5295ade967f7b586ec8b44d4f55a31efb50a27cf75290ada0ee00e9c756346",
+            "scripts/setup_dsh.py": "fe835598584191d1a10c680afd769201efc6a7cf689967f681d8bc64c0b914e4"
+          }
+        },
+        {
+          "continuation_receipt_sha256": "36289c1785f2877a672f3128f6bc716910ca8316539d30a83c7df34bc160c69b",
+          "frozen_identity": {
+            "inputs.manifest.json": "522833a923004b38dfaa821fc0ea5f9c9f8370aee2e7c98a4741775e3c71e876",
+            "protocol.json": "bcb08cf8e47a5484629cecb7f897894f15ffffad8ff08695246c330711da476c",
+            "schedule.json": "aeff62de51a9e04a8c721af99fc31238dbf6436bf248953e871288b335e3f44f",
+            "study.json": "158716e4d5d5e4b6ac5015f35114d0bb338ed9db46934b6491d9993348b3a15f"
+          },
+          "run_id": "dsh-v4-pro-1000-cont01-20260918",
+          "source_commit": "2e26cfc6835a19792fcd09173fde1d5e2524c259",
+          "source_identity": {
+            "dsh/__init__.py": "8e24d82c627405e0a8f9de89ebdbf1ce23db63337ef44aacd4a789cea9697908",
+            "dsh/profile.patch.yml": "81a92895c5f3c4c85bb602e3349a509262eb144c63a566331a43fa73081e2731",
+            "dsh/requirements-controller.txt": "aee6d7d23cbeecc3a101ec56687af9cc7aecbd6e96443295042a5c36307587d4",
+            "dsh/requirements.lock.txt": "ed57fde608bf1cdabaa1fc5d23e5b254983606652be2f732ce4ed2f2c69ea7a1",
+            "dsh/run-python.mjs": "3681bcbdfc6fa10cbb0742fadcc8f3f94a745b428662c94214b370b01bbea67e",
+            "dsh/runtime.lock.json": "e241e17af63e1f2b5ebaab3845a40c8f35c8643c3b5622b778541663428aba44",
+            "dsh/runtime.py": "2223a5573a3bb6c7fec46e264920f5cacf3dedaffb48f6a732e7f4371380c5e7",
+            "ntu_gifts/dsh/__init__.py": "11855c8b3a9276249180fa9ddf0f4d967d21140d57fff4bc4a638642864483ff",
+            "ntu_gifts/dsh/__main__.py": "b54ebc80f16c01be4b84a44474b3567e61acdf1e694813764869060ed002a762",
+            "ntu_gifts/dsh/adapter.py": "c0184cbecaba44c58d2779ac4972e33912b4c5235b00d52b20ae0eacb1aa93c9",
+            "ntu_gifts/dsh/bridge.py": "4a1f2d2f9ab71a9f6575b1c3da4ca181f09c8d73e6eaac0fec8e00943e4cab7c",
+            "ntu_gifts/dsh/bwrap_payload.py": "02b1eb6067e4ab4d283d98148bd6b260bcc9a0ac465dfd490acb93f7ab79542e",
+            "ntu_gifts/dsh/dependencies.py": "e0618ffbb448082e1b5ee5eb6a3ca384375f0ddbf042be72726ee26053227254",
+            "ntu_gifts/dsh/evidence.py": "a3ad48ec6163ddd2844265272f487d733a9f38f7ee886bf361ec240f5a69d224",
+            "ntu_gifts/dsh/execution_bwrap.py": "f37c3818c3b38bad1de8885c02f1c644a2e9abf7abe9b31a2823ee4ae941b632",
+            "ntu_gifts/dsh/protocol.py": "9467ea64de8a60f60988c2dd2ddd804efea53a9bedab8e6c1dfa28ef8b963a5d",
+            "ntu_gifts/dsh/reporting.py": "6fa53a97eec2d89c9eb2793e603564a7c00f65b8f6b5c15c4b043de28341ef0a",
+            "ntu_gifts/dsh/runner.py": "3ae5c3495e63fbf7266bfa071cc79a969fc8f3a8494e2fc86f98ec0f3d0ac534",
+            "ntu_gifts/dsh/study.py": "e0ba44f4fa4be343b1eae4c56f4c5db7e0424da350e371c5e93840883295683f",
+            "ntu_gifts/dsh/worker.py": "4d5ed66f60161f19e197bf702d4acee3aee37a9bfb99ef0a52a448b0de76810f",
+            "ntu_gifts/harness/budget.py": "84e27094f08676328c0899c2abd14097238efde43a5f38c110f28d32bcb2c385",
+            "ntu_gifts/harness/study.py": "7264540b065df45f76d4d6705bf8485f4a9d778b900035efbcfe39a7520abc6c",
+            "ntu_gifts/workspace.py": "2b5295ade967f7b586ec8b44d4f55a31efb50a27cf75290ada0ee00e9c756346",
+            "scripts/setup_dsh.py": "fe835598584191d1a10c680afd769201efc6a7cf689967f681d8bc64c0b914e4"
+          }
+        },
+        {
+          "continuation_receipt_sha256": "454d4ee4568805c4149f9c03de2e5d7501e8a31e85d2a75d781b59b2b6fb4394",
+          "frozen_identity": {
+            "inputs.manifest.json": "2f14277588845c559e695c60499e3e37ad79274262bbc80d68660f5473609780",
+            "protocol.json": "fad42532a34c4f12fe7a2de80c1b69ad93b024e789d07c0861e828e52e301e56",
+            "schedule.json": "a7f1e2d5616efdc5c975ed941b0ff91240f39585130ad38057b115109802b53b",
+            "study.json": "63b6ea5de0bc970b913942f88d7e5568f45c336075619177471eeb02287b6192"
+          },
+          "run_id": "dsh-v4-pro-1000-cont02-20260919",
+          "source_commit": "88dfd135a6258d235a3d496c9e8044c4fec6ef2e",
+          "source_identity": {
+            "dsh/__init__.py": "8e24d82c627405e0a8f9de89ebdbf1ce23db63337ef44aacd4a789cea9697908",
+            "dsh/profile.patch.yml": "81a92895c5f3c4c85bb602e3349a509262eb144c63a566331a43fa73081e2731",
+            "dsh/requirements-controller.txt": "aee6d7d23cbeecc3a101ec56687af9cc7aecbd6e96443295042a5c36307587d4",
+            "dsh/requirements.lock.txt": "ed57fde608bf1cdabaa1fc5d23e5b254983606652be2f732ce4ed2f2c69ea7a1",
+            "dsh/run-python.mjs": "3681bcbdfc6fa10cbb0742fadcc8f3f94a745b428662c94214b370b01bbea67e",
+            "dsh/runtime.lock.json": "e241e17af63e1f2b5ebaab3845a40c8f35c8643c3b5622b778541663428aba44",
+            "dsh/runtime.py": "2223a5573a3bb6c7fec46e264920f5cacf3dedaffb48f6a732e7f4371380c5e7",
+            "ntu_gifts/dsh/__init__.py": "11855c8b3a9276249180fa9ddf0f4d967d21140d57fff4bc4a638642864483ff",
+            "ntu_gifts/dsh/__main__.py": "b54ebc80f16c01be4b84a44474b3567e61acdf1e694813764869060ed002a762",
+            "ntu_gifts/dsh/adapter.py": "c0184cbecaba44c58d2779ac4972e33912b4c5235b00d52b20ae0eacb1aa93c9",
+            "ntu_gifts/dsh/bridge.py": "a2aae1fce301be15cf57faea045800ce8f311fbfc65968807c3da9218221cbb2",
+            "ntu_gifts/dsh/bwrap_payload.py": "02b1eb6067e4ab4d283d98148bd6b260bcc9a0ac465dfd490acb93f7ab79542e",
+            "ntu_gifts/dsh/dependencies.py": "e0618ffbb448082e1b5ee5eb6a3ca384375f0ddbf042be72726ee26053227254",
+            "ntu_gifts/dsh/evidence.py": "a3ad48ec6163ddd2844265272f487d733a9f38f7ee886bf361ec240f5a69d224",
+            "ntu_gifts/dsh/execution_bwrap.py": "f37c3818c3b38bad1de8885c02f1c644a2e9abf7abe9b31a2823ee4ae941b632",
+            "ntu_gifts/dsh/protocol.py": "9467ea64de8a60f60988c2dd2ddd804efea53a9bedab8e6c1dfa28ef8b963a5d",
+            "ntu_gifts/dsh/reporting.py": "6fa53a97eec2d89c9eb2793e603564a7c00f65b8f6b5c15c4b043de28341ef0a",
+            "ntu_gifts/dsh/runner.py": "3ae5c3495e63fbf7266bfa071cc79a969fc8f3a8494e2fc86f98ec0f3d0ac534",
+            "ntu_gifts/dsh/study.py": "e0ba44f4fa4be343b1eae4c56f4c5db7e0424da350e371c5e93840883295683f",
+            "ntu_gifts/dsh/worker.py": "4d5ed66f60161f19e197bf702d4acee3aee37a9bfb99ef0a52a448b0de76810f",
+            "ntu_gifts/harness/budget.py": "84e27094f08676328c0899c2abd14097238efde43a5f38c110f28d32bcb2c385",
+            "ntu_gifts/harness/study.py": "7264540b065df45f76d4d6705bf8485f4a9d778b900035efbcfe39a7520abc6c",
+            "ntu_gifts/workspace.py": "2b5295ade967f7b586ec8b44d4f55a31efb50a27cf75290ada0ee00e9c756346",
+            "scripts/setup_dsh.py": "fe835598584191d1a10c680afd769201efc6a7cf689967f681d8bc64c0b914e4"
+          }
+        },
+        {
+          "continuation_receipt_sha256": "c43ec808dfa7a0f2dc7eb6ef46749648f8642e1086e6724341b36ddca39eb637",
+          "frozen_identity": {
+            "inputs.manifest.json": "98566f98dee3eda6f9ed372cb16f2f7370829b0fedc83c9f95c80d2be9a25ad0",
+            "protocol.json": "e80906d5329ed81db3d7e5afeae6480a512a40961a8768fa3d5974e2026e4413",
+            "schedule.json": "4c5f6a7539534a64b27438d49cfa50b259d89be3dea91e83e4b61fdc2aac57e7",
+            "study.json": "9c6d9f22e04b1a8b5535e35d60119213476cc5d7b296d0247fab18f5e3a974a2"
+          },
+          "run_id": "dsh-v4-pro-1000-cont02-20260919-auto01",
+          "source_commit": "88dfd135a6258d235a3d496c9e8044c4fec6ef2e",
+          "source_identity": {
+            "dsh/__init__.py": "8e24d82c627405e0a8f9de89ebdbf1ce23db63337ef44aacd4a789cea9697908",
+            "dsh/profile.patch.yml": "81a92895c5f3c4c85bb602e3349a509262eb144c63a566331a43fa73081e2731",
+            "dsh/requirements-controller.txt": "aee6d7d23cbeecc3a101ec56687af9cc7aecbd6e96443295042a5c36307587d4",
+            "dsh/requirements.lock.txt": "ed57fde608bf1cdabaa1fc5d23e5b254983606652be2f732ce4ed2f2c69ea7a1",
+            "dsh/run-python.mjs": "3681bcbdfc6fa10cbb0742fadcc8f3f94a745b428662c94214b370b01bbea67e",
+            "dsh/runtime.lock.json": "e241e17af63e1f2b5ebaab3845a40c8f35c8643c3b5622b778541663428aba44",
+            "dsh/runtime.py": "2223a5573a3bb6c7fec46e264920f5cacf3dedaffb48f6a732e7f4371380c5e7",
+            "ntu_gifts/dsh/__init__.py": "11855c8b3a9276249180fa9ddf0f4d967d21140d57fff4bc4a638642864483ff",
+            "ntu_gifts/dsh/__main__.py": "b54ebc80f16c01be4b84a44474b3567e61acdf1e694813764869060ed002a762",
+            "ntu_gifts/dsh/adapter.py": "c0184cbecaba44c58d2779ac4972e33912b4c5235b00d52b20ae0eacb1aa93c9",
+            "ntu_gifts/dsh/bridge.py": "a2aae1fce301be15cf57faea045800ce8f311fbfc65968807c3da9218221cbb2",
+            "ntu_gifts/dsh/bwrap_payload.py": "02b1eb6067e4ab4d283d98148bd6b260bcc9a0ac465dfd490acb93f7ab79542e",
+            "ntu_gifts/dsh/dependencies.py": "e0618ffbb448082e1b5ee5eb6a3ca384375f0ddbf042be72726ee26053227254",
+            "ntu_gifts/dsh/evidence.py": "a3ad48ec6163ddd2844265272f487d733a9f38f7ee886bf361ec240f5a69d224",
+            "ntu_gifts/dsh/execution_bwrap.py": "f37c3818c3b38bad1de8885c02f1c644a2e9abf7abe9b31a2823ee4ae941b632",
+            "ntu_gifts/dsh/protocol.py": "9467ea64de8a60f60988c2dd2ddd804efea53a9bedab8e6c1dfa28ef8b963a5d",
+            "ntu_gifts/dsh/reporting.py": "6fa53a97eec2d89c9eb2793e603564a7c00f65b8f6b5c15c4b043de28341ef0a",
+            "ntu_gifts/dsh/runner.py": "3ae5c3495e63fbf7266bfa071cc79a969fc8f3a8494e2fc86f98ec0f3d0ac534",
+            "ntu_gifts/dsh/study.py": "e0ba44f4fa4be343b1eae4c56f4c5db7e0424da350e371c5e93840883295683f",
+            "ntu_gifts/dsh/worker.py": "4d5ed66f60161f19e197bf702d4acee3aee37a9bfb99ef0a52a448b0de76810f",
+            "ntu_gifts/harness/budget.py": "84e27094f08676328c0899c2abd14097238efde43a5f38c110f28d32bcb2c385",
+            "ntu_gifts/harness/study.py": "7264540b065df45f76d4d6705bf8485f4a9d778b900035efbcfe39a7520abc6c",
+            "ntu_gifts/workspace.py": "2b5295ade967f7b586ec8b44d4f55a31efb50a27cf75290ada0ee00e9c756346",
+            "scripts/setup_dsh.py": "fe835598584191d1a10c680afd769201efc6a7cf689967f681d8bc64c0b914e4"
+          }
+        },
+        {
+          "continuation_receipt_sha256": "e59e8c8f8fe0cc912afbda777fe529919480e360641dcbfbe2e9e49cb259b910",
+          "frozen_identity": {
+            "inputs.manifest.json": "8014f57fc0ed0a4ffd656803b8224d221a0b4e47972248fc58d97dd894ff9af4",
+            "protocol.json": "04da83504c6f92d1caccaec0642f964cd0db2ae13816c69606bc749875e8c822",
+            "schedule.json": "a9387ccfc5201ef3b0b39525241dd655710734239275b4ae0d33528265389098",
+            "study.json": "a854e9dbad49b0350009b757bd0d843550ac543159e9bf87b65fab00689de254"
+          },
+          "run_id": "dsh-v4-pro-1000-cont03-20260919",
+          "source_commit": "e6c90cb95a9c11592d2dd6107304cf8a66c072c4",
+          "source_identity": {
+            "dsh/__init__.py": "8e24d82c627405e0a8f9de89ebdbf1ce23db63337ef44aacd4a789cea9697908",
+            "dsh/profile.patch.yml": "81a92895c5f3c4c85bb602e3349a509262eb144c63a566331a43fa73081e2731",
+            "dsh/requirements-controller.txt": "aee6d7d23cbeecc3a101ec56687af9cc7aecbd6e96443295042a5c36307587d4",
+            "dsh/requirements.lock.txt": "ed57fde608bf1cdabaa1fc5d23e5b254983606652be2f732ce4ed2f2c69ea7a1",
+            "dsh/run-python.mjs": "3681bcbdfc6fa10cbb0742fadcc8f3f94a745b428662c94214b370b01bbea67e",
+            "dsh/runtime.lock.json": "e241e17af63e1f2b5ebaab3845a40c8f35c8643c3b5622b778541663428aba44",
+            "dsh/runtime.py": "2223a5573a3bb6c7fec46e264920f5cacf3dedaffb48f6a732e7f4371380c5e7",
+            "ntu_gifts/dsh/__init__.py": "11855c8b3a9276249180fa9ddf0f4d967d21140d57fff4bc4a638642864483ff",
+            "ntu_gifts/dsh/__main__.py": "b54ebc80f16c01be4b84a44474b3567e61acdf1e694813764869060ed002a762",
+            "ntu_gifts/dsh/adapter.py": "c0184cbecaba44c58d2779ac4972e33912b4c5235b00d52b20ae0eacb1aa93c9",
+            "ntu_gifts/dsh/bridge.py": "a2aae1fce301be15cf57faea045800ce8f311fbfc65968807c3da9218221cbb2",
+            "ntu_gifts/dsh/bwrap_payload.py": "02b1eb6067e4ab4d283d98148bd6b260bcc9a0ac465dfd490acb93f7ab79542e",
+            "ntu_gifts/dsh/dependencies.py": "e0618ffbb448082e1b5ee5eb6a3ca384375f0ddbf042be72726ee26053227254",
+            "ntu_gifts/dsh/evidence.py": "a3ad48ec6163ddd2844265272f487d733a9f38f7ee886bf361ec240f5a69d224",
+            "ntu_gifts/dsh/execution_bwrap.py": "f37c3818c3b38bad1de8885c02f1c644a2e9abf7abe9b31a2823ee4ae941b632",
+            "ntu_gifts/dsh/failure_policy.py": "ad3f5c6f5dc4b6a4dce7d2373b0f155293826bbd61caa4a0cbb06cd475dd8764",
+            "ntu_gifts/dsh/protocol.py": "9467ea64de8a60f60988c2dd2ddd804efea53a9bedab8e6c1dfa28ef8b963a5d",
+            "ntu_gifts/dsh/reporting.py": "6fa53a97eec2d89c9eb2793e603564a7c00f65b8f6b5c15c4b043de28341ef0a",
+            "ntu_gifts/dsh/runner.py": "082f3878ee2c24ecf15b0b5229e5feb4eab801dbca78fa1fc1c14aa09c8ed42e",
+            "ntu_gifts/dsh/study.py": "e0ba44f4fa4be343b1eae4c56f4c5db7e0424da350e371c5e93840883295683f",
+            "ntu_gifts/dsh/worker.py": "4d5ed66f60161f19e197bf702d4acee3aee37a9bfb99ef0a52a448b0de76810f",
+            "ntu_gifts/harness/budget.py": "84e27094f08676328c0899c2abd14097238efde43a5f38c110f28d32bcb2c385",
+            "ntu_gifts/harness/study.py": "7264540b065df45f76d4d6705bf8485f4a9d778b900035efbcfe39a7520abc6c",
+            "ntu_gifts/workspace.py": "2b5295ade967f7b586ec8b44d4f55a31efb50a27cf75290ada0ee00e9c756346",
+            "scripts/setup_dsh.py": "fe835598584191d1a10c680afd769201efc6a7cf689967f681d8bc64c0b914e4"
+          }
+        },
+        {
+          "continuation_receipt_sha256": "19b3accee1764accedf3bc27e764d655bd69ab4c46f3fa8583c5853d77065039",
+          "frozen_identity": {
+            "inputs.manifest.json": "f62e5d69edd087491fa54177903ed8fd0839f435bc290b0bc85461e95bb8c409",
+            "protocol.json": "acdd1263b564964b2aa91c8801db32351492d694be5bcc65a82e0a31a879ece2",
+            "schedule.json": "ffbfe1f4b6e21cf869947e9d251afdb5dec86c10bcfbfd3d8fda53464a93c987",
+            "study.json": "a9258462247e545bf96b4d187b4f89108660cd41654947ce630b5198f7955069"
+          },
+          "run_id": "dsh-v4-pro-1000-cont04-20260919",
+          "source_commit": "a1d3d8cbd349104bf292e03226b336faadcd461d",
+          "source_identity": {
+            "dsh/__init__.py": "8e24d82c627405e0a8f9de89ebdbf1ce23db63337ef44aacd4a789cea9697908",
+            "dsh/profile.patch.yml": "81a92895c5f3c4c85bb602e3349a509262eb144c63a566331a43fa73081e2731",
+            "dsh/requirements-controller.txt": "aee6d7d23cbeecc3a101ec56687af9cc7aecbd6e96443295042a5c36307587d4",
+            "dsh/requirements.lock.txt": "ed57fde608bf1cdabaa1fc5d23e5b254983606652be2f732ce4ed2f2c69ea7a1",
+            "dsh/run-python.mjs": "3681bcbdfc6fa10cbb0742fadcc8f3f94a745b428662c94214b370b01bbea67e",
+            "dsh/runtime.lock.json": "e241e17af63e1f2b5ebaab3845a40c8f35c8643c3b5622b778541663428aba44",
+            "dsh/runtime.py": "2223a5573a3bb6c7fec46e264920f5cacf3dedaffb48f6a732e7f4371380c5e7",
+            "ntu_gifts/dsh/__init__.py": "11855c8b3a9276249180fa9ddf0f4d967d21140d57fff4bc4a638642864483ff",
+            "ntu_gifts/dsh/__main__.py": "b54ebc80f16c01be4b84a44474b3567e61acdf1e694813764869060ed002a762",
+            "ntu_gifts/dsh/adapter.py": "c0184cbecaba44c58d2779ac4972e33912b4c5235b00d52b20ae0eacb1aa93c9",
+            "ntu_gifts/dsh/bridge.py": "92a3d3685609ef7823294fdb43eae24d92e3a75373508a28ad21d1de561d88e7",
+            "ntu_gifts/dsh/bwrap_payload.py": "02b1eb6067e4ab4d283d98148bd6b260bcc9a0ac465dfd490acb93f7ab79542e",
+            "ntu_gifts/dsh/dependencies.py": "e0618ffbb448082e1b5ee5eb6a3ca384375f0ddbf042be72726ee26053227254",
+            "ntu_gifts/dsh/evidence.py": "a3ad48ec6163ddd2844265272f487d733a9f38f7ee886bf361ec240f5a69d224",
+            "ntu_gifts/dsh/execution_bwrap.py": "f37c3818c3b38bad1de8885c02f1c644a2e9abf7abe9b31a2823ee4ae941b632",
+            "ntu_gifts/dsh/failure_policy.py": "50f39ce6228e1eaca24278dd46774d13a39ecd2e7773a5c36e952ee70451083f",
+            "ntu_gifts/dsh/protocol.py": "9467ea64de8a60f60988c2dd2ddd804efea53a9bedab8e6c1dfa28ef8b963a5d",
+            "ntu_gifts/dsh/reporting.py": "6fa53a97eec2d89c9eb2793e603564a7c00f65b8f6b5c15c4b043de28341ef0a",
+            "ntu_gifts/dsh/runner.py": "082f3878ee2c24ecf15b0b5229e5feb4eab801dbca78fa1fc1c14aa09c8ed42e",
+            "ntu_gifts/dsh/study.py": "e0ba44f4fa4be343b1eae4c56f4c5db7e0424da350e371c5e93840883295683f",
+            "ntu_gifts/dsh/worker.py": "4d5ed66f60161f19e197bf702d4acee3aee37a9bfb99ef0a52a448b0de76810f",
+            "ntu_gifts/harness/budget.py": "84e27094f08676328c0899c2abd14097238efde43a5f38c110f28d32bcb2c385",
+            "ntu_gifts/harness/study.py": "7264540b065df45f76d4d6705bf8485f4a9d778b900035efbcfe39a7520abc6c",
+            "ntu_gifts/workspace.py": "2b5295ade967f7b586ec8b44d4f55a31efb50a27cf75290ada0ee00e9c756346",
+            "scripts/setup_dsh.py": "fe835598584191d1a10c680afd769201efc6a7cf689967f681d8bc64c0b914e4"
+          }
+        },
+        {
+          "continuation_receipt_sha256": "cbe103993cf9c9e66f8971cda0e19adce752964577d468f192747d0c34c1c233",
+          "frozen_identity": {
+            "inputs.manifest.json": "373f9f9b230367f43ad862c448743d2f78ba6211686fb065f8078ac14985f568",
+            "protocol.json": "3b0ef361824b426e08c6a6aec0c959314c0bfc2c600b8eec6db7ab36e6060017",
+            "schedule.json": "cb0f1f05568a79b1ffdf94ef9866422942879bba91e099ea1064021800f7e0aa",
+            "study.json": "4cdd10f36498d393fa216c4dcfad667983b3a438386a60c25c92bd35c10a3df6"
+          },
+          "run_id": "dsh-v4-pro-1000-cont05-20260919",
+          "source_commit": "3ced22b417aea99b0956b504794561964dbf7b33",
+          "source_identity": {
+            "dsh/__init__.py": "8e24d82c627405e0a8f9de89ebdbf1ce23db63337ef44aacd4a789cea9697908",
+            "dsh/profile.patch.yml": "81a92895c5f3c4c85bb602e3349a509262eb144c63a566331a43fa73081e2731",
+            "dsh/requirements-controller.txt": "aee6d7d23cbeecc3a101ec56687af9cc7aecbd6e96443295042a5c36307587d4",
+            "dsh/requirements.lock.txt": "ed57fde608bf1cdabaa1fc5d23e5b254983606652be2f732ce4ed2f2c69ea7a1",
+            "dsh/run-python.mjs": "3681bcbdfc6fa10cbb0742fadcc8f3f94a745b428662c94214b370b01bbea67e",
+            "dsh/runtime.lock.json": "e241e17af63e1f2b5ebaab3845a40c8f35c8643c3b5622b778541663428aba44",
+            "dsh/runtime.py": "2223a5573a3bb6c7fec46e264920f5cacf3dedaffb48f6a732e7f4371380c5e7",
+            "ntu_gifts/dsh/__init__.py": "11855c8b3a9276249180fa9ddf0f4d967d21140d57fff4bc4a638642864483ff",
+            "ntu_gifts/dsh/__main__.py": "b54ebc80f16c01be4b84a44474b3567e61acdf1e694813764869060ed002a762",
+            "ntu_gifts/dsh/adapter.py": "c0184cbecaba44c58d2779ac4972e33912b4c5235b00d52b20ae0eacb1aa93c9",
+            "ntu_gifts/dsh/bridge.py": "f56ef6509cd831ec7de2c61b5331a7496771cf0754fcfaf0b29aeb88fa9be0f9",
+            "ntu_gifts/dsh/bwrap_payload.py": "02b1eb6067e4ab4d283d98148bd6b260bcc9a0ac465dfd490acb93f7ab79542e",
+            "ntu_gifts/dsh/dependencies.py": "e0618ffbb448082e1b5ee5eb6a3ca384375f0ddbf042be72726ee26053227254",
+            "ntu_gifts/dsh/evidence.py": "bb0f5be46a80c4fcaf993e8071b22160fc04d5d871a9e446340a4041a9ef7591",
+            "ntu_gifts/dsh/execution_bwrap.py": "f37c3818c3b38bad1de8885c02f1c644a2e9abf7abe9b31a2823ee4ae941b632",
+            "ntu_gifts/dsh/failure_policy.py": "50f39ce6228e1eaca24278dd46774d13a39ecd2e7773a5c36e952ee70451083f",
+            "ntu_gifts/dsh/late_evidence.py": "5f6231a93f7668107f6d15501177013a64e27af42a904b8a8acba502edcab197",
+            "ntu_gifts/dsh/protocol.py": "9467ea64de8a60f60988c2dd2ddd804efea53a9bedab8e6c1dfa28ef8b963a5d",
+            "ntu_gifts/dsh/reporting.py": "823067b7a99f03cc682fb448aa26992b06b75ee15096567d93d85b31e715e9f9",
+            "ntu_gifts/dsh/runner.py": "082f3878ee2c24ecf15b0b5229e5feb4eab801dbca78fa1fc1c14aa09c8ed42e",
+            "ntu_gifts/dsh/study.py": "57c3495a97f12baae8aa2bb74c2289bfc5a7cbad0cc70205f34e66c734c28008",
+            "ntu_gifts/dsh/worker.py": "4d5ed66f60161f19e197bf702d4acee3aee37a9bfb99ef0a52a448b0de76810f",
+            "ntu_gifts/harness/budget.py": "84e27094f08676328c0899c2abd14097238efde43a5f38c110f28d32bcb2c385",
+            "ntu_gifts/harness/study.py": "7264540b065df45f76d4d6705bf8485f4a9d778b900035efbcfe39a7520abc6c",
+            "ntu_gifts/workspace.py": "2b5295ade967f7b586ec8b44d4f55a31efb50a27cf75290ada0ee00e9c756346",
+            "scripts/setup_dsh.py": "fe835598584191d1a10c680afd769201efc6a7cf689967f681d8bc64c0b914e4"
+          }
+        }
+      ],
+      "report_sha256": "f8f0c4f5b40578f15b40be5b1965ff1a7c4007ca4154b8e95da82e788a3e1c3a"
+    },
+    "schema": {
+      "bindings": [
+        {
+          "continuation_receipt_sha256": null,
+          "frozen_identity": {
+            "inputs.manifest.json": "38ce82733f9e2d7a9551ef3ca46a8799579b3d45337c9cf7e4d4d3354322eb63",
+            "protocol.json": "63d6e27ba6ac078fcfa809380f926b30b774ed4c70bd64420c406ea190f413a5",
+            "schedule.json": "cde6543225e535909044c596d0e0a969d72ff6742eca60ea38dd5b02f83a7daa",
+            "study.json": "ef2cdb869ccf2a4618ef829570a3776292be47b6e9e30c115cb5c3b7edce8dbb"
+          },
+          "run_id": "schema",
+          "source_commit": "3ced22b417aea99b0956b504794561964dbf7b33",
+          "source_identity": {
+            "dsh/__init__.py": "8e24d82c627405e0a8f9de89ebdbf1ce23db63337ef44aacd4a789cea9697908",
+            "dsh/profile.patch.yml": "81a92895c5f3c4c85bb602e3349a509262eb144c63a566331a43fa73081e2731",
+            "dsh/requirements-controller.txt": "aee6d7d23cbeecc3a101ec56687af9cc7aecbd6e96443295042a5c36307587d4",
+            "dsh/requirements.lock.txt": "ed57fde608bf1cdabaa1fc5d23e5b254983606652be2f732ce4ed2f2c69ea7a1",
+            "dsh/run-python.mjs": "3681bcbdfc6fa10cbb0742fadcc8f3f94a745b428662c94214b370b01bbea67e",
+            "dsh/runtime.lock.json": "e241e17af63e1f2b5ebaab3845a40c8f35c8643c3b5622b778541663428aba44",
+            "dsh/runtime.py": "2223a5573a3bb6c7fec46e264920f5cacf3dedaffb48f6a732e7f4371380c5e7",
+            "ntu_gifts/dsh/__init__.py": "11855c8b3a9276249180fa9ddf0f4d967d21140d57fff4bc4a638642864483ff",
+            "ntu_gifts/dsh/__main__.py": "b54ebc80f16c01be4b84a44474b3567e61acdf1e694813764869060ed002a762",
+            "ntu_gifts/dsh/adapter.py": "c0184cbecaba44c58d2779ac4972e33912b4c5235b00d52b20ae0eacb1aa93c9",
+            "ntu_gifts/dsh/bridge.py": "f56ef6509cd831ec7de2c61b5331a7496771cf0754fcfaf0b29aeb88fa9be0f9",
+            "ntu_gifts/dsh/bwrap_payload.py": "02b1eb6067e4ab4d283d98148bd6b260bcc9a0ac465dfd490acb93f7ab79542e",
+            "ntu_gifts/dsh/dependencies.py": "e0618ffbb448082e1b5ee5eb6a3ca384375f0ddbf042be72726ee26053227254",
+            "ntu_gifts/dsh/evidence.py": "bb0f5be46a80c4fcaf993e8071b22160fc04d5d871a9e446340a4041a9ef7591",
+            "ntu_gifts/dsh/execution_bwrap.py": "f37c3818c3b38bad1de8885c02f1c644a2e9abf7abe9b31a2823ee4ae941b632",
+            "ntu_gifts/dsh/failure_policy.py": "50f39ce6228e1eaca24278dd46774d13a39ecd2e7773a5c36e952ee70451083f",
+            "ntu_gifts/dsh/late_evidence.py": "5f6231a93f7668107f6d15501177013a64e27af42a904b8a8acba502edcab197",
+            "ntu_gifts/dsh/protocol.py": "9467ea64de8a60f60988c2dd2ddd804efea53a9bedab8e6c1dfa28ef8b963a5d",
+            "ntu_gifts/dsh/reporting.py": "823067b7a99f03cc682fb448aa26992b06b75ee15096567d93d85b31e715e9f9",
+            "ntu_gifts/dsh/runner.py": "082f3878ee2c24ecf15b0b5229e5feb4eab801dbca78fa1fc1c14aa09c8ed42e",
+            "ntu_gifts/dsh/study.py": "57c3495a97f12baae8aa2bb74c2289bfc5a7cbad0cc70205f34e66c734c28008",
+            "ntu_gifts/dsh/worker.py": "4d5ed66f60161f19e197bf702d4acee3aee37a9bfb99ef0a52a448b0de76810f",
+            "ntu_gifts/harness/budget.py": "84e27094f08676328c0899c2abd14097238efde43a5f38c110f28d32bcb2c385",
+            "ntu_gifts/harness/study.py": "7264540b065df45f76d4d6705bf8485f4a9d778b900035efbcfe39a7520abc6c",
+            "ntu_gifts/workspace.py": "2b5295ade967f7b586ec8b44d4f55a31efb50a27cf75290ada0ee00e9c756346",
+            "scripts/setup_dsh.py": "fe835598584191d1a10c680afd769201efc6a7cf689967f681d8bc64c0b914e4"
+          }
+        }
+      ],
+      "recovery_receipt_sha256": "a3b879571ae3b3c95aeeb3fe7a3bf59ed775dd5bc19272b6f20e2b899d42b1f3",
+      "report_sha256": "53f433a20463235254142f509580b7afb8640caeb282a63b629f2551738c2fca"
+    }
+  }
+}

results/deepseek-v4-pro/dsh/summary.json

@@ -0,0 +1,175 @@
+{
+  "all_slots_sealed": true,
+  "arm": "dsh",
+  "assumptions": {
+    "attempt_selection": "The caller supplies one row per task under its frozen attempt policy.",
+    "coefficient_boundary": "5% includes equality. Numeric decimal representations are compared exactly without epsilon.",
+    "compilation_evidence": "Caller supplies True, False, or None from verified execution evidence, including exit status, timeout, termination confirmation, and execution errors. Missing or uncertain evidence maps to None.",
+    "denominator": "All planned tasks remain in every denominator. Confirmed compilation failures are False; missing or uncertain execution evidence is unknown. For the three result metrics, failed execution, missing predictions, unscored rows, and unsupported reference fields are unknown. Unknowns contribute no successes and are counted separately.",
+    "field_validation": "Each metric requires only its own finite numeric fields. Booleans, numeric strings, and inequalities are not coerced. Standard errors do not affect any of these three formulas.",
+    "official_rules": "Webpage interpretation only; official scorer parity is not verified.",
+    "prediction_evidence": "Only execution_status=succeeded and scoring_status=scored rows are evaluated. The current caller requires a complete valid prediction triplet; invalid outputs are not recovered here.",
+    "significance_categories": "Four categories use strict .01, .05, and .1 boundaries. Both p-values must be numeric and within [0, 1]. These boundaries and the absent direction gate are explicit local choices.",
+    "zero_reference_coefficient": "A zero reference coefficient is unknown for relative error and positive/negative direction; it does not block significance. A zero prediction against a nonzero reference is incorrect."
+  },
+  "complete": false,
+  "dataset_revision": "59f9512a38e594528807744214a60ee00367434e",
+  "definitions": {
+    "coefficient_direction": "Matching strictly positive or strictly negative coefficient signs.",
+    "compilation_success": "Trusted evidence of complete code execution; no prediction JSON gate.",
+    "partial_replication": "Coefficient relative error <= .05; no standard-error or p-value gate.",
+    "significance_level": "Equal p-value category: p < .01, p < .05, p < .1, otherwise; no direction gate."
+  },
+  "display_name": "DeepSeek API (deepseek-v4-pro alias) + DSH",
+  "display_role": "standalone_dsh_harness_entry",
+  "evidence_class": "technical_recovery_view",
+  "generated_at": "2026-09-19T18:18:48.070003+00:00",
+  "kind": "inferencenet-offline-leaderboard-export",
+  "legacy_local_paper": {
+    "definitions": {
+      "coefficient_only": "coefficient relative error <= .05",
+      "direction": "matching strictly positive or strictly negative coefficient signs",
+      "partial": "coefficient and SE relative errors < .05; no p gate; includes perfect",
+      "perfect": "coefficient and SE relative errors <= .01 and p absolute error <= .01",
+      "significance": "direction and equal category: p < .01, p < .05, p < .1, otherwise"
+    },
+    "metrics": {
+      "coefficient_only": {
+        "assessable": 695,
+        "count": 555,
+        "coverage_percent": 69.5,
+        "denominator": 1000,
+        "failure_count": 140,
+        "rate": 0.555,
+        "score": 55.5,
+        "unknown_count": 305
+      },
+      "direction": {
+        "assessable": 695,
+        "count": 680,
+        "coverage_percent": 69.5,
+        "denominator": 1000,
+        "failure_count": 15,
+        "rate": 0.68,
+        "score": 68.0,
+        "unknown_count": 305
+      },
+      "partial": {
+        "assessable": 695,
+        "count": 460,
+        "coverage_percent": 69.5,
+        "denominator": 1000,
+        "failure_count": 235,
+        "rate": 0.46,
+        "score": 46.0,
+        "unknown_count": 305
+      },
+      "perfect": {
+        "assessable": 695,
+        "count": 372,
+        "coverage_percent": 69.5,
+        "denominator": 1000,
+        "failure_count": 323,
+        "rate": 0.372,
+        "score": 37.2,
+        "unknown_count": 305
+      },
+      "significance": {
+        "assessable": 695,
+        "count": 591,
+        "coverage_percent": 69.5,
+        "denominator": 1000,
+        "failure_count": 104,
+        "rate": 0.591,
+        "score": 59.1,
+        "unknown_count": 305
+      }
+    },
+    "profile": "local-paper-v1",
+    "verification": "Original sealed scoring.json flags; no rescoring."
+  },
+  "metric_profile": "hf-leaderboard-v1",
+  "model": "deepseek-v4-pro",
+  "model_alias": "deepseek-v4-pro",
+  "official_parity_verified": false,
+  "original_protocol_complete": false,
+  "publication_kind": "research_results_with_technical_recovery",
+  "result": {
+    "columns": [
+      "Model ID",
+      "Compilation Success",
+      "Partial Replication",
+      "Correct Coefficient Direction",
+      "Significant Level Correctness"
+    ],
+    "definite_outcome_count": 970,
+    "expected_count": 1000,
+    "invalid_evidence_count": 0,
+    "metrics": {
+      "coefficient_direction": {
+        "assessable": 698,
+        "count": 683,
+        "coverage_percent": 69.8,
+        "denominator": 1000,
+        "failure_count": 15,
+        "rate": 0.683,
+        "score": 68.3,
+        "unknown_count": 302
+      },
+      "compilation_success": {
+        "assessable": 970,
+        "count": 699,
+        "coverage_percent": 97.0,
+        "denominator": 1000,
+        "failure_count": 271,
+        "rate": 0.699,
+        "score": 69.9,
+        "unknown_count": 30
+      },
+      "partial_replication": {
+        "assessable": 698,
+        "count": 556,
+        "coverage_percent": 69.8,
+        "denominator": 1000,
+        "failure_count": 142,
+        "rate": 0.556,
+        "score": 55.6,
+        "unknown_count": 302
+      },
+      "significance_level": {
+        "assessable": 696,
+        "count": 598,
+        "coverage_percent": 69.6,
+        "denominator": 1000,
+        "failure_count": 98,
+        "rate": 0.598,
+        "score": 59.8,
+        "unknown_count": 304
+      }
+    },
+    "resolved_count": 1000,
+    "row": {
+      "Compilation Success": 69.9,
+      "Correct Coefficient Direction": 68.3,
+      "Model ID": "DeepSeek API (deepseek-v4-pro alias) + DSH",
+      "Partial Replication": 55.6,
+      "Significant Level Correctness": 59.8
+    },
+    "sealed_count": 1000,
+    "task_counts": {
+      "completed": 1000,
+      "failed": 271,
+      "invalid_evidence": 0,
+      "not_started": 0,
+      "planned": 1000,
+      "succeeded": 699,
+      "unknown": 30,
+      "unsealed": 0
+    },
+    "task_unknown_count": 30,
+    "valid_prediction_count": 699
+  },
+  "schema_version": 3,
+  "underlying_model_weights_verified": false,
+  "verification_scope": "Preserved verified flags and result/score hashes; no new inference, execution or rescoring."
+}

results/index.json

@@ -1,81 +1,87 @@
-{
-  "schema_version": 1,
-  "kind": "results_archive",
-  "archived_at_utc": "2026-09-15T07:29:39.569697+00:00",
-  "displayed_leaderboard_updated": false,
-  "entries": [
-    {
-      "model": "gpt-5.6-sol",
-      "arm": "baseline",
-      "rows": 1000,
-      "path": "results/gpt-5.6-sol/baseline"
-    },
-    {
-      "model": "gpt-5.6-sol",
-      "arm": "deepagents",
-      "rows": 1000,
-      "path": "results/gpt-5.6-sol/deepagents"
-    },
-    {
-      "model": "claude-opus-4-8",
-      "arm": "baseline",
-      "rows": 1000,
-      "path": "results/claude-opus-4-8/baseline"
-    },
-    {
-      "model": "claude-opus-4-8",
-      "arm": "deepagents",
-      "rows": 1000,
-      "path": "results/claude-opus-4-8/deepagents"
-    },
-    {
-      "model": "kimi-k3",
-      "arm": "baseline",
-      "rows": 1000,
-      "path": "results/kimi-k3/baseline"
-    },
-    {
-      "model": "kimi-k3",
-      "arm": "deepagents",
-      "rows": 1000,
-      "path": "results/kimi-k3/deepagents"
-    },
-    {
-      "model": "gemini-3.1-pro-preview",
-      "arm": "baseline",
-      "rows": 1000,
-      "path": "results/gemini-3.1-pro-preview/baseline"
-    },
-    {
-      "model": "gemini-3.1-pro-preview",
-      "arm": "deepagents",
-      "rows": 1000,
-      "path": "results/gemini-3.1-pro-preview/deepagents"
-    },
-    {
-      "model": "qwen3.7-max",
-      "arm": "baseline",
-      "rows": 1000,
-      "path": "results/qwen3.7-max/baseline"
-    },
-    {
-      "model": "qwen3.7-max",
-      "arm": "deepagents",
-      "rows": 1000,
-      "path": "results/qwen3.7-max/deepagents"
-    },
-    {
-      "model": "deepseek-v4-pro",
-      "arm": "baseline",
-      "rows": 1000,
-      "path": "results/deepseek-v4-pro/baseline"
-    },
-    {
-      "model": "deepseek-v4-pro",
-      "arm": "deepagents",
-      "rows": 1000,
-      "path": "results/deepseek-v4-pro/deepagents"
-    }
-  ],
-  "updated_at_utc": "2026-09-16T17:18:47.373930+00:00"
-}
+{
+  "schema_version": 1,
+  "kind": "results_archive",
+  "archived_at_utc": "2026-09-15T07:29:39.569697+00:00",
+  "displayed_leaderboard_updated": false,
+  "entries": [
+    {
+      "model": "gpt-5.6-sol",
+      "arm": "baseline",
+      "rows": 1000,
+      "path": "results/gpt-5.6-sol/baseline"
+    },
+    {
+      "model": "gpt-5.6-sol",
+      "arm": "deepagents",
+      "rows": 1000,
+      "path": "results/gpt-5.6-sol/deepagents"
+    },
+    {
+      "model": "claude-opus-4-8",
+      "arm": "baseline",
+      "rows": 1000,
+      "path": "results/claude-opus-4-8/baseline"
+    },
+    {
+      "model": "claude-opus-4-8",
+      "arm": "deepagents",
+      "rows": 1000,
+      "path": "results/claude-opus-4-8/deepagents"
+    },
+    {
+      "model": "kimi-k3",
+      "arm": "baseline",
+      "rows": 1000,
+      "path": "results/kimi-k3/baseline"
+    },
+    {
+      "model": "kimi-k3",
+      "arm": "deepagents",
+      "rows": 1000,
+      "path": "results/kimi-k3/deepagents"
+    },
+    {
+      "model": "gemini-3.1-pro-preview",
+      "arm": "baseline",
+      "rows": 1000,
+      "path": "results/gemini-3.1-pro-preview/baseline"
+    },
+    {
+      "model": "gemini-3.1-pro-preview",
+      "arm": "deepagents",
+      "rows": 1000,
+      "path": "results/gemini-3.1-pro-preview/deepagents"
+    },
+    {
+      "model": "qwen3.7-max",
+      "arm": "baseline",
+      "rows": 1000,
+      "path": "results/qwen3.7-max/baseline"
+    },
+    {
+      "model": "qwen3.7-max",
+      "arm": "deepagents",
+      "rows": 1000,
+      "path": "results/qwen3.7-max/deepagents"
+    },
+    {
+      "model": "deepseek-v4-pro",
+      "arm": "baseline",
+      "rows": 1000,
+      "path": "results/deepseek-v4-pro/baseline"
+    },
+    {
+      "model": "deepseek-v4-pro",
+      "arm": "deepagents",
+      "rows": 1000,
+      "path": "results/deepseek-v4-pro/deepagents"
+    },
+    {
+      "model": "deepseek-v4-pro",
+      "arm": "dsh",
+      "rows": 1000,
+      "path": "results/deepseek-v4-pro/dsh"
+    }
+  ],
+  "updated_at_utc": "2026-09-19T18:47:27.943320+00:00"
+}

results/manifest.sha256

@@ -1,80 +1,93 @@
-4ca82d5601f12b48b24e22beb1ef624a17f1369074d296a6151bfd6f4a5d0668  README.md
-2ad22f4778525b1460f1857e1095f2eea5620063fba1ffa6314ca7fd50f14703  claude-opus-4-8/baseline/README.md
-8058afcd14c34d666277c0f32ac5962398b10618df5b1aa0e13cd9862344dc56  claude-opus-4-8/baseline/config.json
-4d5f859e472b79195c9be6b1f0e9c5feecf59a81dfe32ac08eb4c1d4967ed98e  claude-opus-4-8/baseline/results.csv
-b9d6bd51c9eb81a634e78e422c86da16f655d97da87d3f5afa4c9aa0f96809f4  claude-opus-4-8/baseline/results.jsonl
-7b5b06bd9b3d7b47952b1dc424a01fcdf04ddccca0b3f64d04132358d6380cdc  claude-opus-4-8/baseline/summary.json
-dfb00967c2a2ff320d5a6d8c142cbc6ef37d4fb144abcc7a451673515c2b4c50  claude-opus-4-8/deepagents/README.md
-12215dfcba5d2d692268a305364514a03ff7579094b2d4af12f493383b8e0f21  claude-opus-4-8/deepagents/config.json
-bafc0a72cd4d8f53c1da33030250171f36ce441488bc8456cba8c3f52b80e6bd  claude-opus-4-8/deepagents/results.csv
-3bef3b9850f24b9cb0b5806efd8425c4e4cc650735a6b28b45befdd55a6ec28a  claude-opus-4-8/deepagents/results.jsonl
-f86da9c76f8becb75b4970c486160013209a6aa4239923018b217e55f1b0a614  claude-opus-4-8/deepagents/summary.json
-c738e75419e319515b15c823300a16e9bdf1cbe790612bbbc24ff93c71438b3f  deepseek-v4-pro/baseline/README.md
-4cef524f37a325029ea4ad325b163f72f9cf725f507f20cd76424d06e71e24ad  deepseek-v4-pro/baseline/config.json
-98f4f77003f4788509360a2cbe41554279df408fd20d65058b85836a25858ecb  deepseek-v4-pro/baseline/results.csv
-7827218f6f18f39988e8c439f0c93cab611c04d158d8704aa50c9274cd438837  deepseek-v4-pro/baseline/results.jsonl
-c51e4f1de78eba8f30e4a59276fb7821132cd103f0ae3f9d04c75c98344f2e34  deepseek-v4-pro/baseline/summary.json
-f1297897da23ab01751289cd93da69a944834af59204cd73f7fca64a112bd6a9  deepseek-v4-pro/deepagents/README.md
-400c28e4008d5241bf50bc71c2674861846d6b6d2e7da21f2a7602a60ecc0ae9  deepseek-v4-pro/deepagents/config.json
-a842bac2b7b608ab124d95883194e01700c6fba648593a2128db6f9d40511cf8  deepseek-v4-pro/deepagents/results.csv
-a006c8b0b80064295faeb3aa7395a2408f0208677853448816b361bf67e39bb6  deepseek-v4-pro/deepagents/results.jsonl
-231e4264b0403e12788136d01af465f86fe9ba3c3ab992510925a7ef4c2a40e9  deepseek-v4-pro/deepagents/summary.json
-eb38fada3bb400d40e37fd97aeb41c8eea6a1475406395e12ad9ac1c14f71e34  deepseek-v4-pro/diagnostics.json
-d811ae94edc6589e2b7ee8c39ac6c4da1c66b5c3dad8acbc589b17767f9c5690  deepseek-v4-pro/export-verification.json
-891a8e0c9e1be24652b56d477bbdc0c98cf961c18010177535d33395dc43a132  deepseek-v4-pro/paired_comparison.csv
-8be2d803fc96ad6a7840c0c3eb52b52e0e1d8e8523f6a4349b4adfbdc4c320f3  deepseek-v4-pro/provenance.json
-ac7253c934e43a88d0a2f6b105d1bb13424ef004e6838d2a632d5e9c366c72f2  gemini-3.1-pro-preview/baseline/README.md
-ff9884d2095695a1f40f5fc573982b9a49fd53d41dca2a6fc36d04dc81f10998  gemini-3.1-pro-preview/baseline/config.json
-fc708f327a76b22ebe6f453206b0c42e4cf46176afb8cb12eda2c1b71fe90bde  gemini-3.1-pro-preview/baseline/results.csv
-2f1e2019157d105efee7ba311a49eba99abe4a7b84f0a4b4e0ac174e2d56cdba  gemini-3.1-pro-preview/baseline/results.jsonl
-a8ba5100a28f1f0aeae6949eb3669c7717c844c4e2eee6fa42ded351aef6b5c7  gemini-3.1-pro-preview/baseline/summary.json
-49ff37264a63f3b5615632213bb9248d2b6b8feb66b4ec989ed655b45e5faed2  gemini-3.1-pro-preview/deepagents/README.md
-f09d9d99d26328ae2f54e9173b22f69d68f7d7bb50c3cc5e352c996b98dd57f9  gemini-3.1-pro-preview/deepagents/config.json
-af16c74fdc1163682a9a98f597ac0d101bc0de64d60c6a8582f69312948c8069  gemini-3.1-pro-preview/deepagents/results.csv
-c2824f8277b283a6dd60118b6e6a0cb0419e4c8b9193de15a7bcf7aaaad24181  gemini-3.1-pro-preview/deepagents/results.jsonl
-bf9ce721c94103f91b0ca576f717f4f5dca7cc9965601658a65fc6d2a15b7045  gemini-3.1-pro-preview/deepagents/summary.json
-67b24d52eeabbcd84baf00c7395e72d4efe7cfcc82d7de1b0f8ee30f8018a1f7  gemini-3.1-pro-preview/diagnostics.json
-885b4de7e83334c416f33ea1535a72f1f70dfe61834d1b622e01ddb3cccfaed4  gemini-3.1-pro-preview/export-verification.json
-930a9ae67dc0dd4210e7e0c96098cab24899c765e4b3d041f75455f58fc072fe  gemini-3.1-pro-preview/paired_comparison.csv
-84378a81ca5533439b21dc1cf683925f5accc23532d392d140a4477e59c4edfb  gemini-3.1-pro-preview/provenance.json
-ed22b41edf2afe7a500f4caf1b61f0b4c6e6c73759522c543fb86ce478eb4d04  gpt-5.6-sol/baseline/README.md
-90ec4e0781b57498fef52de31eacf5c8ec9424549643865a7140a1ccdb04c1d4  gpt-5.6-sol/baseline/config.json
-a54171a3d3e7ecda3e61311414a84131e74a680bfa1342e8627cfaa32fc7c873  gpt-5.6-sol/baseline/results.csv
-3b12160382043e27531ff5cb3e23bccc9a8f7807e4121a82ffa13df28f65a859  gpt-5.6-sol/baseline/results.jsonl
-6d6ab1aaffe754e08f9656dcb5f51fcd5c772e60660dc34d86cfc21f3c5c03d7  gpt-5.6-sol/baseline/summary.json
-99a2c4edc5f8a2984e09211c7608f08cc29131e37241cc79b6c77b774d3b21fe  gpt-5.6-sol/deepagents/README.md
-a3e24175f2395cffbbcd6d9d3c89689bd3a3b22dd4ce61bb78d5da33c6b6cc1b  gpt-5.6-sol/deepagents/config.json
-27abe57119d4be49c56b29c318a1bc2757a97f44427764d7fc8d188be4d10671  gpt-5.6-sol/deepagents/results.csv
-378cdec8e7a880f091ba442649d1c034997acefdc623898b60a063baad29d7be  gpt-5.6-sol/deepagents/results.jsonl
-821ca6e55a66006c7fb958951a7859a9dccde29d87de8415d5c3f07f523563c5  gpt-5.6-sol/deepagents/summary.json
-b6376d43e3107a6ac45647cdd1b0fa382437f92e33655715146d4cf6e89467d5  index.json
-29825997a250cc8fe292450006b82a392fe90c8e0493a1335f74eea72c1de104  kimi-k3/baseline/README.md
-7191c406e8695a5cb760ee8fa99c4fb0dd71bca59347acc4d6d33fe0630be081  kimi-k3/baseline/config.json
-f87d2dd3b390695f6b420a78e126b504abc3d292180a3443deaead174ec1b464  kimi-k3/baseline/results.csv
-0502ae8892dd6c66ee92890a83c678527cff6c3396aa4c13faaadcf41e1857c1  kimi-k3/baseline/results.jsonl
-2edd2e96f3b448ceceb6189edb7f23c737b245c00c9820728c69f5237feb34b7  kimi-k3/baseline/summary.json
-d6fe2ba624876d49373d4a0c180e0c27067405501b247e59c2d8ca5c74427ff7  kimi-k3/deepagents/README.md
-312c9260c01d5988878e4f2b88da55a16362ee994e8217c63b15ba772da3857c  kimi-k3/deepagents/config.json
-572c3270a3ec255544b587aada9d8bd56cc2555d5246ce26feea2a3d35eb8f94  kimi-k3/deepagents/results.csv
-bdab99f8f1d81885f74326cf9c8c8e57b9bb8a8d616bfbb1b5a2f6bf32b737ca  kimi-k3/deepagents/results.jsonl
-81c8cabaa58acc65ef6afba792241b70951b666ab90c3f7c25194b61b0d9a228  kimi-k3/deepagents/summary.json
-bfaac06563fc53d5f564197fdfb69e24ddbdff622b85d67bbf2a13fb7fb5d9ba  kimi-k3/diagnostics.json
-b96e0cc3352efac6140c33cef30873e522db547891101c3c49c6aca42085becb  kimi-k3/export-verification.json
-3fe1378a23cfe59a37558521ec5bf47ea299a6ad4903a14684147a1b8066ee03  kimi-k3/paired_comparison.csv
-4a0f6dde38c5410a6f379192f30eac6d330403c80fcdbc2b357dbe15427a331b  kimi-k3/provenance.json
-1606c3170bec39e7c074659fd8c8d011e02e98b1e69161338554a9e1c416a363  qwen3.7-max/baseline/README.md
-9f553440e741863e82f32fec11470affe6f0dbffa03c668c647bd63d8f73227f  qwen3.7-max/baseline/config.json
-f88769bb10cbebd47ed1101a7e41c4cc5ba680bf49a768ef010aa3681bfc3845  qwen3.7-max/baseline/results.csv
-4924e929d67956f9deb6c319605042d09f40b12f7bd2f18baa0464b8f7c9851a  qwen3.7-max/baseline/results.jsonl
-e506814d33789673fcd6b8e6869cdcbd52371cd3d0455ce53ae27e1098a52214  qwen3.7-max/baseline/summary.json
-c032c3f9953bbd6e7685028f1b08be3539aee4fc9d186c941b5a248895c2289c  qwen3.7-max/deepagents/README.md
-3862d438d381ca3e57b1bf68921ef01a8c38727902152ada2c6e1fcf7b70cc77  qwen3.7-max/deepagents/config.json
-09d09b2411c67b5c026e5d82662b016f36bd7373563db541e40b2fa0429998dc  qwen3.7-max/deepagents/results.csv
-e009f8c5c88b05896836330822afbda33af463c1f264f9adb9bbf5b094e61dfa  qwen3.7-max/deepagents/results.jsonl
-a5dcb2297ccc31a921e3f4eeb025ed81a764e16556a12068733cfe3bfe0bdbe3  qwen3.7-max/deepagents/summary.json
-af5df4bd67b09d31325e71769cdedf0d2d8c4ed41ee7ce3a6eb10a2919737589  qwen3.7-max/diagnostics.json
-7b2d6aa947e21941e412e38f91aa01b75ff91b0668fea9071b5a9787f3a77347  qwen3.7-max/export-verification.json
-2600a5ed895e9cf217f96ba7cf2bf61762012886cb8f0918693888ab4bb084d3  qwen3.7-max/paired_comparison.csv
-0b4d4df9d186cb3ab5abd0d0a52bfcf89bad5efb4a135039277b282797d9580e  qwen3.7-max/provenance.json
-06ed669caaba3ca5e252d12af3de32223019d50aefd9c46751ca3cb35110fce2  verify_kimi_gemini.py
-9c6d1d6b44998a459141969065e4e23fb0126b3556074ce35803f970ce8c0869  verify_qwen_deepseek.py
+0aed0cba6dfcb66b4f22440aeb7e5721e015d26c3e195a99e944cb85c72108ef  README.md
+2ad22f4778525b1460f1857e1095f2eea5620063fba1ffa6314ca7fd50f14703  claude-opus-4-8/baseline/README.md
+8058afcd14c34d666277c0f32ac5962398b10618df5b1aa0e13cd9862344dc56  claude-opus-4-8/baseline/config.json
+4d5f859e472b79195c9be6b1f0e9c5feecf59a81dfe32ac08eb4c1d4967ed98e  claude-opus-4-8/baseline/results.csv
+b9d6bd51c9eb81a634e78e422c86da16f655d97da87d3f5afa4c9aa0f96809f4  claude-opus-4-8/baseline/results.jsonl
+7b5b06bd9b3d7b47952b1dc424a01fcdf04ddccca0b3f64d04132358d6380cdc  claude-opus-4-8/baseline/summary.json
+dfb00967c2a2ff320d5a6d8c142cbc6ef37d4fb144abcc7a451673515c2b4c50  claude-opus-4-8/deepagents/README.md
+12215dfcba5d2d692268a305364514a03ff7579094b2d4af12f493383b8e0f21  claude-opus-4-8/deepagents/config.json
+bafc0a72cd4d8f53c1da33030250171f36ce441488bc8456cba8c3f52b80e6bd  claude-opus-4-8/deepagents/results.csv
+3bef3b9850f24b9cb0b5806efd8425c4e4cc650735a6b28b45befdd55a6ec28a  claude-opus-4-8/deepagents/results.jsonl
+f86da9c76f8becb75b4970c486160013209a6aa4239923018b217e55f1b0a614  claude-opus-4-8/deepagents/summary.json
+c738e75419e319515b15c823300a16e9bdf1cbe790612bbbc24ff93c71438b3f  deepseek-v4-pro/baseline/README.md
+4cef524f37a325029ea4ad325b163f72f9cf725f507f20cd76424d06e71e24ad  deepseek-v4-pro/baseline/config.json
+98f4f77003f4788509360a2cbe41554279df408fd20d65058b85836a25858ecb  deepseek-v4-pro/baseline/results.csv
+7827218f6f18f39988e8c439f0c93cab611c04d158d8704aa50c9274cd438837  deepseek-v4-pro/baseline/results.jsonl
+c51e4f1de78eba8f30e4a59276fb7821132cd103f0ae3f9d04c75c98344f2e34  deepseek-v4-pro/baseline/summary.json
+f1297897da23ab01751289cd93da69a944834af59204cd73f7fca64a112bd6a9  deepseek-v4-pro/deepagents/README.md
+400c28e4008d5241bf50bc71c2674861846d6b6d2e7da21f2a7602a60ecc0ae9  deepseek-v4-pro/deepagents/config.json
+a842bac2b7b608ab124d95883194e01700c6fba648593a2128db6f9d40511cf8  deepseek-v4-pro/deepagents/results.csv
+a006c8b0b80064295faeb3aa7395a2408f0208677853448816b361bf67e39bb6  deepseek-v4-pro/deepagents/results.jsonl
+231e4264b0403e12788136d01af465f86fe9ba3c3ab992510925a7ef4c2a40e9  deepseek-v4-pro/deepagents/summary.json
+eb38fada3bb400d40e37fd97aeb41c8eea6a1475406395e12ad9ac1c14f71e34  deepseek-v4-pro/diagnostics.json
+ccf9f54ce467c8a977074498eeb86a2ce1b7b69a73b370b025663f9582a8d991  deepseek-v4-pro/dsh/README.md
+6f92c5376ab6b96b9ab5ddab6faff88cd792d0ec999f1cc96cdd48e32302c9e1  deepseek-v4-pro/dsh/accounting.json
+33faf46bce3d4a152bb04cd3eb7cded2d948b2bf5ec2beac49941b145e92c522  deepseek-v4-pro/dsh/all-attempts.jsonl
+df322f13761e06dc78cb553c5dac347ce4361599937af4017a902c56a7df9d77  deepseek-v4-pro/dsh/config.json
+d6500c012259ed5e3b514d6f62597fdcadfb7548e41308eed442a1b130f0e336  deepseek-v4-pro/dsh/diagnostics.json
+8d7cae2ea2f33130636367f5a5e8f92b5df0471aa9889aa54094e2d1664dd48b  deepseek-v4-pro/dsh/export-verification.json
+153d15ff14c3dba6ece8fd9c81ae2d9fa1f0b6a240e90efd6b1d6e46f40893e2  deepseek-v4-pro/dsh/first-pass-results.jsonl
+b264e4796ea32220a7c67fe84d990ac6e040784f7331c662e66ca5d91c71e9c1  deepseek-v4-pro/dsh/first-pass-summary.json
+fa67ea7aba6f55ddceb6e10695acc0c4f5f4957ff5495f0ba83ef564478e7381  deepseek-v4-pro/dsh/manifest.sha256
+8e03b3fd76310ab80643f51ce791e85545062d2d85ac3c6121cc1143ebfb5a28  deepseek-v4-pro/dsh/provenance.json
+954a6adff76d6bae31c77cd969b17fa5203ae7ba38abf76130c6f066e39a9c58  deepseek-v4-pro/dsh/results.csv
+b12427d5f4476351e571e4b29f61e0463540a1701bec51e642753d717dbc1df2  deepseek-v4-pro/dsh/results.jsonl
+62936f76b1ca0de5e761d1ea1f894343695503ce934bf0213dc9fde1ea72a782  deepseek-v4-pro/dsh/summary.json
+d811ae94edc6589e2b7ee8c39ac6c4da1c66b5c3dad8acbc589b17767f9c5690  deepseek-v4-pro/export-verification.json
+891a8e0c9e1be24652b56d477bbdc0c98cf961c18010177535d33395dc43a132  deepseek-v4-pro/paired_comparison.csv
+8be2d803fc96ad6a7840c0c3eb52b52e0e1d8e8523f6a4349b4adfbdc4c320f3  deepseek-v4-pro/provenance.json
+ac7253c934e43a88d0a2f6b105d1bb13424ef004e6838d2a632d5e9c366c72f2  gemini-3.1-pro-preview/baseline/README.md
+ff9884d2095695a1f40f5fc573982b9a49fd53d41dca2a6fc36d04dc81f10998  gemini-3.1-pro-preview/baseline/config.json
+fc708f327a76b22ebe6f453206b0c42e4cf46176afb8cb12eda2c1b71fe90bde  gemini-3.1-pro-preview/baseline/results.csv
+2f1e2019157d105efee7ba311a49eba99abe4a7b84f0a4b4e0ac174e2d56cdba  gemini-3.1-pro-preview/baseline/results.jsonl
+a8ba5100a28f1f0aeae6949eb3669c7717c844c4e2eee6fa42ded351aef6b5c7  gemini-3.1-pro-preview/baseline/summary.json
+49ff37264a63f3b5615632213bb9248d2b6b8feb66b4ec989ed655b45e5faed2  gemini-3.1-pro-preview/deepagents/README.md
+f09d9d99d26328ae2f54e9173b22f69d68f7d7bb50c3cc5e352c996b98dd57f9  gemini-3.1-pro-preview/deepagents/config.json
+af16c74fdc1163682a9a98f597ac0d101bc0de64d60c6a8582f69312948c8069  gemini-3.1-pro-preview/deepagents/results.csv
+c2824f8277b283a6dd60118b6e6a0cb0419e4c8b9193de15a7bcf7aaaad24181  gemini-3.1-pro-preview/deepagents/results.jsonl
+bf9ce721c94103f91b0ca576f717f4f5dca7cc9965601658a65fc6d2a15b7045  gemini-3.1-pro-preview/deepagents/summary.json
+67b24d52eeabbcd84baf00c7395e72d4efe7cfcc82d7de1b0f8ee30f8018a1f7  gemini-3.1-pro-preview/diagnostics.json
+885b4de7e83334c416f33ea1535a72f1f70dfe61834d1b622e01ddb3cccfaed4  gemini-3.1-pro-preview/export-verification.json
+930a9ae67dc0dd4210e7e0c96098cab24899c765e4b3d041f75455f58fc072fe  gemini-3.1-pro-preview/paired_comparison.csv
+84378a81ca5533439b21dc1cf683925f5accc23532d392d140a4477e59c4edfb  gemini-3.1-pro-preview/provenance.json
+ed22b41edf2afe7a500f4caf1b61f0b4c6e6c73759522c543fb86ce478eb4d04  gpt-5.6-sol/baseline/README.md
+90ec4e0781b57498fef52de31eacf5c8ec9424549643865a7140a1ccdb04c1d4  gpt-5.6-sol/baseline/config.json
+a54171a3d3e7ecda3e61311414a84131e74a680bfa1342e8627cfaa32fc7c873  gpt-5.6-sol/baseline/results.csv
+3b12160382043e27531ff5cb3e23bccc9a8f7807e4121a82ffa13df28f65a859  gpt-5.6-sol/baseline/results.jsonl
+6d6ab1aaffe754e08f9656dcb5f51fcd5c772e60660dc34d86cfc21f3c5c03d7  gpt-5.6-sol/baseline/summary.json
+99a2c4edc5f8a2984e09211c7608f08cc29131e37241cc79b6c77b774d3b21fe  gpt-5.6-sol/deepagents/README.md
+a3e24175f2395cffbbcd6d9d3c89689bd3a3b22dd4ce61bb78d5da33c6b6cc1b  gpt-5.6-sol/deepagents/config.json
+27abe57119d4be49c56b29c318a1bc2757a97f44427764d7fc8d188be4d10671  gpt-5.6-sol/deepagents/results.csv
+378cdec8e7a880f091ba442649d1c034997acefdc623898b60a063baad29d7be  gpt-5.6-sol/deepagents/results.jsonl
+821ca6e55a66006c7fb958951a7859a9dccde29d87de8415d5c3f07f523563c5  gpt-5.6-sol/deepagents/summary.json
+8ce61775c0f51f4d53181887131a2ac83d7615dfbda59a8fb8706b4d2abcd5bd  index.json
+29825997a250cc8fe292450006b82a392fe90c8e0493a1335f74eea72c1de104  kimi-k3/baseline/README.md
+7191c406e8695a5cb760ee8fa99c4fb0dd71bca59347acc4d6d33fe0630be081  kimi-k3/baseline/config.json
+f87d2dd3b390695f6b420a78e126b504abc3d292180a3443deaead174ec1b464  kimi-k3/baseline/results.csv
+0502ae8892dd6c66ee92890a83c678527cff6c3396aa4c13faaadcf41e1857c1  kimi-k3/baseline/results.jsonl
+2edd2e96f3b448ceceb6189edb7f23c737b245c00c9820728c69f5237feb34b7  kimi-k3/baseline/summary.json
+d6fe2ba624876d49373d4a0c180e0c27067405501b247e59c2d8ca5c74427ff7  kimi-k3/deepagents/README.md
+312c9260c01d5988878e4f2b88da55a16362ee994e8217c63b15ba772da3857c  kimi-k3/deepagents/config.json
+572c3270a3ec255544b587aada9d8bd56cc2555d5246ce26feea2a3d35eb8f94  kimi-k3/deepagents/results.csv
+bdab99f8f1d81885f74326cf9c8c8e57b9bb8a8d616bfbb1b5a2f6bf32b737ca  kimi-k3/deepagents/results.jsonl
+81c8cabaa58acc65ef6afba792241b70951b666ab90c3f7c25194b61b0d9a228  kimi-k3/deepagents/summary.json
+bfaac06563fc53d5f564197fdfb69e24ddbdff622b85d67bbf2a13fb7fb5d9ba  kimi-k3/diagnostics.json
+b96e0cc3352efac6140c33cef30873e522db547891101c3c49c6aca42085becb  kimi-k3/export-verification.json
+3fe1378a23cfe59a37558521ec5bf47ea299a6ad4903a14684147a1b8066ee03  kimi-k3/paired_comparison.csv
+4a0f6dde38c5410a6f379192f30eac6d330403c80fcdbc2b357dbe15427a331b  kimi-k3/provenance.json
+1606c3170bec39e7c074659fd8c8d011e02e98b1e69161338554a9e1c416a363  qwen3.7-max/baseline/README.md
+9f553440e741863e82f32fec11470affe6f0dbffa03c668c647bd63d8f73227f  qwen3.7-max/baseline/config.json
+f88769bb10cbebd47ed1101a7e41c4cc5ba680bf49a768ef010aa3681bfc3845  qwen3.7-max/baseline/results.csv
+4924e929d67956f9deb6c319605042d09f40b12f7bd2f18baa0464b8f7c9851a  qwen3.7-max/baseline/results.jsonl
+e506814d33789673fcd6b8e6869cdcbd52371cd3d0455ce53ae27e1098a52214  qwen3.7-max/baseline/summary.json
+c032c3f9953bbd6e7685028f1b08be3539aee4fc9d186c941b5a248895c2289c  qwen3.7-max/deepagents/README.md
+3862d438d381ca3e57b1bf68921ef01a8c38727902152ada2c6e1fcf7b70cc77  qwen3.7-max/deepagents/config.json
+09d09b2411c67b5c026e5d82662b016f36bd7373563db541e40b2fa0429998dc  qwen3.7-max/deepagents/results.csv
+e009f8c5c88b05896836330822afbda33af463c1f264f9adb9bbf5b094e61dfa  qwen3.7-max/deepagents/results.jsonl
+a5dcb2297ccc31a921e3f4eeb025ed81a764e16556a12068733cfe3bfe0bdbe3  qwen3.7-max/deepagents/summary.json
+af5df4bd67b09d31325e71769cdedf0d2d8c4ed41ee7ce3a6eb10a2919737589  qwen3.7-max/diagnostics.json
+7b2d6aa947e21941e412e38f91aa01b75ff91b0668fea9071b5a9787f3a77347  qwen3.7-max/export-verification.json
+2600a5ed895e9cf217f96ba7cf2bf61762012886cb8f0918693888ab4bb084d3  qwen3.7-max/paired_comparison.csv
+0b4d4df9d186cb3ab5abd0d0a52bfcf89bad5efb4a135039277b282797d9580e  qwen3.7-max/provenance.json
+06ed669caaba3ca5e252d12af3de32223019d50aefd9c46751ca3cb35110fce2  verify_kimi_gemini.py
+9c6d1d6b44998a459141969065e4e23fb0126b3556074ce35803f970ce8c0869  verify_qwen_deepseek.py