agent.html

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='6' fill='%230c1219'/%3E%3Ctext x='5' y='23' font-family='sans-serif' font-size='21' fill='%2378dbc0'%3EiN%3C/text%3E%3C/svg%3E">
<meta name="color-scheme" content="light dark">
<meta name="description" content="How single-pass execution, DeepAgents and DeepSeek Harness approach an InferenceNet task.">
<title>Agent / Harness — InferenceNet Challenge</title>
<link rel="stylesheet" href="./leaderboard.css">
<link rel="stylesheet" href="./portal.css">
<script type="module" src="./portal.js">
</script>
</head>
<body>
<a class="skip" href="#main">
<span data-en="Skip to content" data-zh="跳转到正文" >Skip to content</span>
</a>
<div class="page">
<nav class="topbar" aria-label="Main navigation">
<a class="brand" href="./index.html">InferenceNet <small>CHALLENGE</small>
</a>
<div class="nav-links">
<a href="./index.html">
<span data-en="Home" data-zh="主页" >Home</span>
</a>
<a href="./data.html">
<span data-en="Data" data-zh="数据" >Data</span>
</a>
<a href="./leaderboard.html">
<span data-en="Leaderboard" data-zh="榜单" >Leaderboard</span>
</a>
<a href="./agent.html" aria-current="page">
<span data-en="Agent / Harness" data-zh="Agent / Harness" >Agent / Harness</span>
</a>
<button id="language" type="button" aria-label="Switch to Chinese">中文</button>
<button id="theme" type="button" aria-pressed="false">
<span class="theme-icon" aria-hidden="true">
</span>
<span id="theme-label">Light</span>
</button>
</div>
</nav>
<main id="main">
<header class="portal-hero">
<p class="eyebrow">AGENT / HARNESS</p>
<h1 data-en="From one attempt to an execution loop." data-zh="从单次生成，到执行反馈循环。" >From one attempt to an execution loop.</h1>
<p data-en="The model proposes the analysis. The harness manages tools, feedback, budgets and final artifacts. InferenceNet makes the resulting process and output inspectable." data-zh="模型提出分析方案；Harness 管理工具、反馈、预算与最终产物。InferenceNet 让执行过程和结果都可以被检查。" class="portal-lead">The model proposes the analysis. The harness manages tools, feedback, budgets and final artifacts. InferenceNet makes the resulting process and output inspectable.</p>
</header>
<section class="portal-section">
<div class="portal-heading">
<p data-en="01 / EXECUTION MODES" data-zh=" 01 / EXECUTION MODES" class="eyebrow">01 / EXECUTION MODES</p>
<h2 data-en="The harness is part of the experiment." data-zh="Harness 是实验的一部分。" >The harness is part of the experiment.</h2>
</div>
<div class="profile-grid">
<article class="profile">
<p class="eyebrow">A / BASELINE</p>
<h3>Single-pass Agent</h3>
<div class="profile-budget">1 <span data-en="model call" data-zh="次模型调用" >model call</span>
</div>
<p data-en="Generate one program, then execute it. There is no iterative tool-feedback loop." data-zh="生成一个程序后执行，不进行多轮工具反馈循环。" >Generate one program, then execute it. There is no iterative tool-feedback loop.</p>
<a href="./leaderboard.html" class="text-link">
<span data-en="See published results →" data-zh="查看已发布结果 →" >See published results →</span>
</a>
</article>
<article class="profile">
<p class="eyebrow">B / AGENT LOOP</p>
<h3>DeepAgents</h3>
<div class="profile-budget">≤ 6 <span data-en="model calls" data-zh="次模型调用" >model calls</span>
</div>
<p data-en="Plan, inspect data, execute trial programs and revise. Up to four trial tool calls, followed by final execution." data-zh="规划、检查数据、试运行与修订；最多四次试运行工具调用，然后执行最终程序。" >Plan, inspect data, execute trial programs and revise. Up to four trial tool calls, followed by final execution.</p>
<a href="https://easonai-5589.github.io/inferencenet-challenge/agent.html" class="text-link" target="_blank" rel="noopener noreferrer">
<span data-en="Read the full walkthrough ↗" data-zh="阅读完整执行记录 ↗" >Read the full walkthrough ↗</span>
</a>
</article>
<article class="profile">
<p class="eyebrow">C / INDEPENDENT</p>
<h3>DeepSeek Harness · DSH</h3>
<div class="profile-budget">≤ 6 <span data-en="calls / attempt" data-zh="次调用 / 尝试" >calls / attempt</span>
</div>
<p data-en="Official DSH SDK. The published run uses technical recovery and stays separate from the baseline / DeepAgents paired comparison." data-zh="使用官方 DSH SDK；已发布实验包含技术补测，独立于单次生成 / DeepAgents 配对比较展示。" >Official DSH SDK. The published run uses technical recovery and stays separate from the baseline / DeepAgents paired comparison.</p>
<a href="https://huggingface.co/spaces/CamoAiLab/InferenceNet-Leaderboard/blob/6ea7c5684664bfe5c6e3d282a5e28f68244acd9e/results/deepseek-v4-pro/dsh/README.md" class="text-link" target="_blank" rel="noopener noreferrer">
<span data-en="Read the run protocol ↗" data-zh="查看运行协议 ↗" >Read the run protocol ↗</span>
</a>
</article>
</div>
<p data-en="These are the published run settings, not universal framework limits. Unequal total budgets and differing protocols do not support an equal-cost causal claim." data-zh="以上为已发布实验的设定，并非框架本身的限制。各组总预算与协议存在差异，不能据此作等成本的因果判断。" class="source-note">These are the published run settings, not universal framework limits. Unequal total budgets and differing protocols do not support an equal-cost causal claim.</p>
</section>
<section class="portal-section">
<div class="portal-heading">
<p data-en="02 / FEEDBACK" data-zh=" 02 / FEEDBACK" class="eyebrow">02 / FEEDBACK</p>
<h2 data-en="A loop with an explicit stopping point." data-zh="有明确停止条件的反馈循环。" >A loop with an explicit stopping point.</h2>
</div>
<div class="agent-loop">
<ol class="workflow">
<li>
<span class="step-number">01</span>
<h3 data-en="Read the task" data-zh="读取任务" >Read the task</h3>
<p data-en="A research specification and its source data." data-zh="研究要求与对应的数据文件。" >A research specification and its source data.</p>
</li>
<li>
<span class="step-number">02</span>
<h3 data-en="Write &amp; run code" data-zh="编写并执行代码" >Write &amp; run code</h3>
<p data-en="Fit the requested econometric model." data-zh="按照要求估计计量模型。" >Fit the requested econometric model.</p>
</li>
<li>
<span class="step-number">03</span>
<h3 data-en="Inspect &amp; revise" data-zh="检查并修订" >Inspect &amp; revise</h3>
<p data-en="A harness can use execution feedback." data-zh="Harness 可根据执行反馈修订程序。" >A harness can use execution feedback.</p>
</li>
<li>
<span class="step-number">04</span>
<h3 data-en="Recover the finding" data-zh="复现研究结果" >Recover the finding</h3>
<p data-en="Return the coefficient, standard error and p-value." data-zh="输出系数、标准误和 p 值。" >Return the coefficient, standard error and p-value.</p>
</li>
</ol>
<div class="loop-feedback">↶ <span data-en="Execution feedback returns to the model until it finalizes or reaches its budget." data-zh="执行反馈返回模型，直到模型提交最终程序或达到预算上限。" >Execution feedback returns to the model until it finalizes or reaches its budget.</span>
</div>
</div>
</section>
<section class="portal-section" id="case">
<div class="portal-heading">
<p data-en="03 / RECORDED CASE" data-zh=" 03 / RECORDED CASE" class="eyebrow">03 / RECORDED CASE</p>
<h2 data-en="Same task. Two execution paths." data-zh="同一道任务，两种执行路径。" >Same task. Two execution paths.</h2>
</div>
<p class="portal-lead">
<span data-en="Task 0011 asks for a monthly-clustered regression on Spain, 1905–1945. This recorded GPT-5.5 example shows why inspecting the data can change an outcome." data-zh="任务 0011 要求对 1905–1945 年的西班牙样本进行按月聚类的回归。这个已记录的 GPT-5.5 案例展示了检查数据如何影响执行结果。" >Task 0011 asks for a monthly-clustered regression on Spain, 1905–1945. This recorded GPT-5.5 example shows why inspecting the data can change an outcome.</span>
</p>
<div class="case-pair">
<article>
<p class="eyebrow">SINGLE-PASS</p>
<h3 data-en="A guessed schema stops execution." data-zh="对数据结构的猜测导致执行失败。" >A guessed schema stops execution.</h3>
<p data-en="The generated program searches for common month/date column names. It misses the dataset’s tid column and exits before producing a result." data-zh="生成的程序只搜索常见的月份 / 日期列名，遗漏数据中的 tid 列，因此未能输出结果。" >The generated program searches for common month/date column names. It misses the dataset’s tid column and exits before producing a result.</p>
<div class="case-outcome">1 <span data-en="model call · no valid result JSON" data-zh="次模型调用 · 未生成有效结果 JSON" >model call · no valid result JSON</span>
</div>
</article>
<article>
<p class="eyebrow">DEEPAGENTS</p>
<h3 data-en="Inspect, estimate, then finalize." data-zh="先检查，再估计，最后提交。" >Inspect, estimate, then finalize.</h3>
<p data-en="The agent inspects 38 columns, checks the time structure, then fits OLS with tid-clustered errors on 977 observations. Its final program recovers all three statistics." data-zh="Agent 检查了 38 列数据和时间结构，然后对 977 条观测进行按 tid 聚类的 OLS 估计。最终程序复现了三项统计量。" >The agent inspects 38 columns, checks the time structure, then fits OLS with tid-clustered errors on 977 observations. Its final program recovers all three statistics.</p>
<div class="case-outcome">4 <span data-en="model calls · 3 tool calls" data-zh="次模型调用 · 3 次工具调用" >model calls · 3 tool calls</span>
</div>
</article>
</div>
<div class="artifact">
<div>
<h3 data-en="The final artifact" data-zh="最终产物" >The final artifact</h3>
<p data-en="A program writes these three numeric fields. Execution logs and the evaluation record are stored separately." data-zh="程序写出这三个数值字段；执行日志与评分记录单独保存。" >A program writes these three numeric fields. Execution logs and the evaluation record are stored separately.</p>
</div>
<pre>
<code>{
  "coefficient": 0.27273630753072026,
  "standard_error": 0.054224701707830926,
  "p_value": 6.899021793750105e-7
}</code>
</pre>
</div>
<p data-en="Illustrative internal run from 13 September 2026, not a result in the current leaderboard. One example does not establish an average harness benefit." data-zh="2026 年 9 月 13 日的内部执行示例，不计入当前榜单。单个案例不代表 Harness 的平均提升。" class="source-note">Illustrative internal run from 13 September 2026, not a result in the current leaderboard. One example does not establish an average harness benefit.</p>
<div class="actions">
<a href="https://easonai-5589.github.io/inferencenet-challenge/agent.html#walkthrough" class="text-link" target="_blank" rel="noopener noreferrer">
<span data-en="Inspect the complete trace ↗" data-zh="查看完整轨迹 ↗" >Inspect the complete trace ↗</span>
</a>
<a href="./leaderboard.html#protocol-details" class="text-link">
<span data-en="Scoring definitions &amp; evidence →" data-zh="评分定义与证据 →" >Scoring definitions &amp; evidence →</span>
</a>
</div>
</section>
</main>
<footer>
<span>InferenceNet <span class="muted">/</span> <span data-en="AI for empirical research." data-zh="面向实证研究的 AI。" >AI for empirical research.</span>
</span>
<div>
<a href="https://huggingface.co/datasets/CamoAiLab/InferenceNet" class="" target="_blank" rel="noopener noreferrer">
<span data-en="Dataset ↗" data-zh="数据集 ↗" >Dataset ↗</span>
</a>
<a href="https://easonai-5589.github.io/inferencenet-challenge/" class="" target="_blank" rel="noopener noreferrer">
<span data-en="Project website ↗" data-zh="项目官网 ↗" >Project website ↗</span>
</a>
<a href="https://easonai-5589.github.io/inferencenet-challenge/teams.html" class="" target="_blank" rel="noopener noreferrer">
<span data-en="Team &amp; contact ↗" data-zh="团队与联系 ↗" >Team &amp; contact ↗</span>
</a>
</div>
</footer>
</div>
</body>
</html>