index.html

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="description" content="InferenceNet research leaderboard: single-pass generation, DeepAgents and an independent DeepSeek Harness result on 1,000 econometric replication tasks.">
  <meta name="color-scheme" content="dark">
  <title>InferenceNet · Agent & Harness Leaderboard</title>
  <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='6' fill='%230c1219'/%3E%3Ctext x='5' y='23' font-family='sans-serif' font-size='21' fill='%2378dbc0'%3EiN%3C/text%3E%3C/svg%3E">
  <link rel="stylesheet" href="./leaderboard.css">
  <script src="./leaderboard.js" defer></script>
</head>
<body>
  <a class="skip" href="#rankings">Skip to rankings</a>
  <main class="page">
    <nav class="topbar" aria-label="Main navigation">
      <a class="brand" href="https://easonai-5589.github.io/inferencenet-challenge/" target="_blank" rel="noopener noreferrer"><span class="brand-mark" aria-hidden="true">iN</span>InferenceNet</a>
      <div class="nav-links">
        <a href="https://huggingface.co/datasets/CamoAiLab/InferenceNet" target="_blank" rel="noopener noreferrer" data-i18n="dataset">Dataset ↗</a>
        <a href="https://huggingface.co/spaces/CamoAiLab/InferenceNet-Leaderboard/tree/main/results" target="_blank" rel="noopener noreferrer" data-i18n="archive">Results archive ↗</a>
        <button id="language" type="button" aria-label="Switch to Chinese">中文</button>
      </div>
    </nav>

    <header class="hero">
      <p class="eyebrow"><span class="dot" aria-hidden="true"></span><span data-i18n="edition">RESEARCH LEADERBOARD · SEPTEMBER 2026</span></p>
      <h1 data-i18n="title">Models and harnesses.<br><span>Real research, reproduced.</span></h1>
      <p class="intro" data-i18n="intro">Explore single-pass agents, DeepAgents and a separate DeepSeek Harness result on real econometric replication tasks.</p>
      <div class="facts" aria-label="Dataset coverage">
        <span><strong id="model-count">—</strong><span data-i18n="models">requested model aliases</span></span>
        <span><strong id="group-count">—</strong><span data-i18n="groups">experiment groups</span></span>
        <span><strong id="task-count">—</strong><span data-i18n="tasks">tasks per group</span></span>
        <span><strong id="record-count">—</strong><span data-i18n="records">task records</span></span>
        <span class="update" id="published-date"></span>
      </div>
    </header>

    <section class="protocols" aria-label="Experimental settings">
      <div><p class="protocol-tag baseline"><span aria-hidden="true">A</span><strong data-i18n="single">Single-pass Agent</strong></p><p data-i18n="singleDesc">One code-generation call, followed by execution. No iterative agent loop.</p></div>
      <div><p class="protocol-tag harness"><span aria-hidden="true">B</span><strong data-i18n="deepagents">DeepAgents</strong></p><p data-i18n="harnessDesc">DeepAgents plans, uses tools and revises code. Up to 6 model calls, plus final execution.</p></div>
      <div><p class="protocol-tag independent"><span aria-hidden="true">C</span><strong data-i18n="dsh">DeepSeek Harness · separate run</strong></p><p data-i18n="dshDesc">Official DSH SDK, requested v4-pro API alias. A fixed technical recovery selection; underlying model weights are unverified.</p></div>
    </section>

    <section id="rankings" aria-labelledby="rankings-title">
      <div class="section-head"><div><p class="eyebrow" data-i18n="resultsLabel">01 / RESULTS</p><h2 id="rankings-title" data-i18n="rankings">Agent & Harness leaderboard</h2></div><span class="badge" data-i18n="research">Research results · provisional</span></div>
      <p class="scope-note" data-i18n="scope">Locally scored, archived results. Official scorer parity is not yet verified. Different budgets and run protocols mean the lift is descriptive, not an equal-cost comparison.</p>
      <aside id="independent-notice" class="independent-notice" hidden><strong data-i18n="dshNoticeTitle">DSH is an independent result.</strong><p data-i18n="dshNotice">The request used deepseek-v4-pro; the underlying model weights were not verified. The fixed 1,000-task selection includes 87 technical replacement attempts (27 with changed input-schema prompts), with 30 task outcomes still unknown. It is excluded from the paired DeepAgents lift.</p><a id="dsh-protocol" data-i18n="dshProtocol" target="_blank" rel="noopener noreferrer">Read run protocol & evidence ↗</a></aside>

      <div class="toolbar">
        <div class="view-switch" role="group" aria-label="Leaderboard view">
          <button type="button" data-view="compare" aria-pressed="false" data-i18n="compare">Baseline / DeepAgents pairs</button>
          <button type="button" data-view="baseline" aria-pressed="false" data-i18n="single">Single-pass Agent</button>
          <button type="button" data-view="harness" aria-pressed="true" data-i18n="harness">All harness runs</button>
        </div>
        <a class="download" href="./leaderboard.csv" download data-i18n="download">Download CSV ↓</a>
      </div>
      <div class="filters">
        <label class="metric-control"><span data-i18n="metric">Metric</span><select id="metric">
          <optgroup label="Local paper profile" data-group="local">
            <option value="perfect" data-i18n="perfect">Full replication · local</option>
          </optgroup>
          <optgroup label="HF-style profile · parity unverified" data-group="hf">
            <option value="partial_replication" data-i18n="partial_replication">Partial replication · HF-style</option>
            <option value="compilation_success" data-i18n="compilation_success">Execution success · HF-style</option>
            <option value="coefficient_direction" data-i18n="coefficient_direction">Coefficient direction · HF-style</option>
            <option value="significance_level" data-i18n="significance_level">Significance level · HF-style</option>
          </optgroup>
        </select></label>
        <label id="sort-control"><span data-i18n="sort">Sort by</span><select id="sort"><option value="deepagents" data-i18n="harnessScore">DeepAgents score</option><option value="baseline" data-i18n="singleScore">Single-pass score</option><option value="gain" data-i18n="lift">Lift</option></select></label>
        <label class="search-control"><span data-i18n="search">Find a model</span><input id="search" type="search" placeholder="Search models…" autocomplete="off"></label>
      </div>
      <p id="metric-description" class="metric-description"></p>
      <p id="load-state" role="status" class="load-state">Loading archived results…</p>
      <div id="error" class="error" role="alert" hidden><p data-i18n="error">Results could not be loaded. Please retry or open the results archive.</p><button type="button" id="retry" data-i18n="retry">Retry</button></div>
      <div id="table-wrap" class="table-wrap" tabindex="0" aria-label="Scrollable leaderboard" hidden>
        <table><caption id="caption" class="sr-only"></caption><thead id="table-head"></thead><tbody id="table-body"></tbody></table>
      </div>
      <p id="empty" class="load-state" hidden data-i18n="empty">No matching models.</p>
      <div class="table-foot"><span id="result-count" aria-live="polite"></span><span data-i18n="denominator">Every score uses all 1,000 tasks. Unknown and invalid outcomes earn no successes.</span></div>
    </section>

    <section class="method" aria-labelledby="method-title">
      <div class="section-head"><div><p class="eyebrow" data-i18n="methodLabel">02 / HOW TO READ THIS</p><h2 id="method-title" data-i18n="methodTitle">A comparison you can inspect.</h2></div><a href="./legacy.html" data-i18n="historical">Historical leaderboard ↗</a></div>
      <div class="method-grid">
        <article><h3 data-i18n="fixedTasks">Fixed task set</h3><p data-i18n="fixedDesc">The same Selected_1000 task IDs and dataset revision are used for every group. Each score is successes ÷ 1,000; failed or unknown tasks stay in the denominator.</p></article>
        <article><h3 data-i18n="separateMetrics">Two scoring profiles</h3><p data-i18n="profilesDesc">Full replication is the local paper metric. The four HF-style metrics reproduce the published descriptions locally; they are not yet verified against the official scorer.</p></article>
        <article><h3 data-i18n="budgetTitle">Read lift with the budgets</h3><p data-i18n="budgetDesc">Baseline: 1 model call. DeepAgents and DSH: up to 6 model calls and 4 trial tool calls per attempt. DSH adds 87 recovery attempts; full environment and reasoning parity is unverified. DSH has no paired lift; these are not equal-cost causal estimates.</p></article>
      </div>
      <details class="evidence"><summary data-i18n="evidence">Evidence status & metric definitions</summary>
        <p data-i18n="unknownDesc">“Metric unknown” means the selected metric cannot be assessed, including failed execution or unavailable outputs. It is different from a task whose final status is unknown. Neither is dropped or counted as a success.</p>
        <ul id="definitions"></ul>
        <div class="evidence-table-wrap"><table class="evidence-table"><caption data-i18n="evidenceCaption">Status from each accepted archive</caption><thead><tr><th data-i18n="model">Model / API alias</th><th data-i18n="protocol">Protocol</th><th data-i18n="evidenceCol">Evidence</th></tr></thead><tbody id="evidence-body"></tbody></table></div>
        <p data-i18n="verification">HF-style counts are checked against archived task flags. Full-replication counts for Sol and Opus come from accepted summaries; all other displayed runs, including DSH, have per-task local flags. This refresh does not rerun or rescore any experiment.</p>
        <p data-i18n="archiveNote">Per-run archive files remain frozen, including their original publication metadata. This page is a new display derived from those archives. GPT-5.5 and unarchived experiments are not included in this edition.</p>
      </details>
    </section>
    <footer><span>InferenceNet <span class="muted">/</span> <span data-i18n="footer">Econometric replication, measured.</span></span><div><a id="revision-link" href="https://huggingface.co/spaces/CamoAiLab/InferenceNet-Leaderboard/tree/main/results" target="_blank" rel="noopener noreferrer" data-i18n="source">Source snapshot ↗</a><a href="./leaderboard-data.json" data-i18n="json">Data JSON ↗</a><a href="https://easonai-5589.github.io/inferencenet-challenge/" target="_blank" rel="noopener noreferrer" data-i18n="project">Project ↗</a></div></footer>
  </main>
  <noscript><p>This leaderboard needs JavaScript. <a href="./leaderboard.csv">Download the results CSV</a> or <a href="https://huggingface.co/spaces/CamoAiLab/InferenceNet-Leaderboard/tree/main/results">browse the archives</a>.</p></noscript>
</body>
</html>