The work turns loop quality into a measurable task or score. An eight-week longitudinal study of a personal-assistant agent runtime in continuous production since March 2026, with 22 incidents carrying full root-cause postmortems, as close as the literature gets to a real operations log for a recurring, stateful agent system rather than a benchmark.

Use When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime to bound risk before recurring or unattended execution.

Research source arXiv:2606.14589; inspect its method and evaluation before treating results as production evidence.

[2606.14589] When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime

LLM agent systems increasingly run as long-lived autonomous runtimes: scheduling jobs, calling tools, maintaining memory, and pushing results to humans. We present a longitudinal study of silent failures in one such system: a personal-assistant agent runtime in continuous production since March 2026, with roughly 40 scheduled jobs, 8 LLM providers, a tool-governance proxy, and a knowledge-base memory plane, defended by 4,286 unit tests and 827 governance checks. Over eight weeks we documented 22 incidents with full root-cause postmortems, in which one meta-pattern -- a failure whose error signal never reaches a human in actionable form -- manifested at least 28 times. We derive a five-class, mechanism-oriented taxonomy: (A) environment and platform quirks, (B) design-assumption mismatches, (C) error swallowing and dilution, (D) chained hallucination and fabrication, (E) operational omission and forensic blind spots. Class D is unique to LLM systems and the most dangerous: the system does not merely fail to report an error -- the LLM transforms it into fluent, plausible narrative delivered to the user. We term this fail-plausible: gray failure's differential observability escalated -- the observer is not just blind, it is convincingly lied to by the failure itself. Three findings: about 70% of silent failures were caught by human user-view observation, not tests or audits; a retrospective audit of 15 incidents found 0% ex-ante prevention but 87% regression blocking -- audits are regression engines, not prediction engines; incident latency (13 hours to 60 days) tracks failure mechanism, not code complexity -- the longest-lived failures lived in the seams between components, where no test runs. We describe the resulting defense framework and distill design principles for agent systems whose failures are loud, attributable, and boring. All postmortems and artifacts are public.

18 pages, 5 figures, 2 tables. 22 incident postmortems and all defense-framework artifacts publicly available at https://github.com/bisdom-cell/openclaw-model-bridge; governance engine on PyPI (openclaw-ontology-engine)


Evaluation data is used as the feedback signal for improving loop behavior. A taxonomy of seven failure modes unique to production agentic systems, grounded in observations from systems operating at billion-event scale rather than in benchmark runs, the distinguishing feature relative to the many failure taxonomies built from curated traces.

Use Evaluating Agentic AI in the Wild: Failure Modes, Drift Patterns, and a Production Evaluation Framework to bound risk before recurring or unattended execution.

Research source arXiv:2605.01604; inspect its method and evaluation before treating results as production evidence.

Bound permissions, cost, failure, and escalation.

[2605.01604] Evaluating Agentic AI in the Wild: Failure Modes, Drift Patterns, and a Production Evaluation Framework

Existing evaluation frameworks for large language models -- including HELM, MT-Bench, AgentBench, and BIG-bench -- are designed for controlled, single-session, lab-scale settings. They do not address the evaluation challenges that emerge when agentic AI systems operate continuously in production: compounding decision errors, tool failure cascades, non-deterministic output drift, and the absence of ground truth for long-horizon tasks. This paper makes three contributions. First, we present a taxonomy of seven failure modes unique to production agentic systems, each grounded in observations from systems operating at billion-event scale. Second, we demonstrate empirically where standard metrics -- ROUGE, BERTScore, accuracy/AUC, and the agentic benchmarks above -- fail to detect each failure mode. Third, we propose PAEF (Production Agentic Evaluation Framework), a five-dimension evaluation framework with an open-source reference implementation, designed for continuous evaluation on production traffic rather than episodic benchmark runs. Our analysis shows that standard metrics fail to detect four of the seven failure modes entirely and detect three others only after a lag of multiple evaluation cycles.

11 pages, 6 tables, 1 figure. Reference implementation: https://github.com/mukund1985/llm-eval-toolkit


Translates agent-loop ideas into operator-facing workflows for repeated delegated work. Most agent postmortems stop at 'the run failed,' which tells you nothing about where the repair belongs.

Use Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures to bound risk before recurring or unattended execution.

Research source arXiv:2607.28802; inspect its method and evaluation before treating results as production evidence.

[2607.28802] Model or Harness? An Interaction-Centric Taxonomy for Localizing Agent Failures

Existing evaluations often reduce agent failures to system-level outcomes, obscuring where the fault originated and which intervention would improve the agent system. This creates a repair-assignment problem: the same visible failure may call for model post-training, harness engineering, environment redesign, or benchmark repair depending on its source. Because agent behavior emerges from interactions among models, harnesses, users, tools, memory, and environments, outcome-level labels are often insufficient for improvement. Most failure taxonomies do little to resolve this problem because they are benchmark-specific and lack a shared structure. We introduce an interaction-centric taxonomy that localizes failures to the interactions in which they originate and identifies the responsible component. It organizes 41 failure modes by assigning each to an edge between two components and a fault side indicating where the repair belongs. This makes the taxonomy actionable: model-side failures identify targets for post-training, harness-side failures point to scaffolding and tool-integration fixes, and environment or grader failures reveal evaluation conditions requiring redesign. The schema applies across agent architectures, from coding assistants to long-horizon personal assistants and multi-agent systems. We ground the taxonomy in worked examples from public benchmarks, model system cards, published reports, and logged agent trajectories, and evaluate its reproducibility using independent reasoning agents as judges. Across four frontier models, the strongest judge reaches Cohen's $\kappa=0.76$ against human category labels, suggesting that the categories capture shared structure rather than annotator-specific preferences.

Harsh Raj; Vipul Gupta; Anas Mahmoud; Razvan-Gabriel Dumitru; Darvin Yi; Aakash Sabharwal; Yunzhong He


Evaluation data is used as the feedback signal for improving loop behavior. Proposes the ProofAgent Index, a governance-readiness score across Evaluation, Context, Compliance, and Governance, and argues against collapsing governance evidence into a single aggregate number. Validated in healthcare and finance deployments, with context engineering emerging as a dominant reliability factor.

Use Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness to bound risk before recurring or unattended execution.

Research source arXiv:2607.27677; inspect its method and evaluation before treating results as production evidence.

[2607.27677] Stop Shipping AI Agents on Faith: Capability Is Not Production Readiness

AI agents are moving into production workflows where they retrieve information, call tools, maintain state, and act on behalf of users or organizations, but many release decisions still rely on capability signals, demos, or behavioral tests that do not show whether an agent is ready to operate under production constraints. Capability is therefore not production readiness. This paper introduces the ProofAgent Index (PAI), a governance readiness index for AI agents. PAI combines four dimensions of deployment evidence: Evaluation, Context, Compliance, and Governance. Evaluation measures observed behavior, Context measures the operating environment that shapes that behavior, Compliance measures alignment with applicable rules and controls, and Governance measures whether the organization can authorize, monitor, audit, and control the agent during operation. PAI is implemented inside ProofAgent Harness, an open source infrastructure for auditable AI agent evaluation and governance. Validation across two heavily regulated domains, healthcare and finance, shows that PAI carries held out readiness signal and separates higher risk from lower risk configurations. The results show that context engineering strongly changes reliability, capability improves behavior but does not determine readiness, and governance evidence must remain visible rather than averaged away. PAI reframes agent release from a faith based deployment decision into an auditable readiness decision.


Translates agent-loop ideas into operator-facing workflows for repeated delegated work. Formalizes the core operational question of running many loops with one reviewer: how to spend a limited audit budget when agent confidence is miscalibrated and errors are correlated.

Use One Human, N Agents: Audit-Budget Allocation for LLM Agent Fleets under Miscalibrated, Correlated Confidence to bound risk before recurring or unattended execution.

Research source arXiv:2607.28317; inspect its method and evaluation before treating results as production evidence.

[2607.28317] One Human, $N$ Agents: Audit-Budget Allocation for LLM Agent Fleets under Miscalibrated, Correlated Confidence

A single human must audit $N$ LLM agents under a budget of $B \ll N$ audits per round, guided by self-reported confidence that may be adversarially miscalibrated and by correlated errors. We model this as budgeted noisy inspection over a two-level Gaussian copula and locate the miscalibration threshold $\delta^*$ past which confidence-ranked auditing is \emph{worse} than random. Two a-priori expectations reverse: $\delta^*$ \emph{rises} as the budget shrinks, and cross-family correlation is not low---shared difficulty dominates lineage. Five open-weight LLMs show operationally useless (near-constant) confidence, point estimates at or beyond the flip though CIs straddle it; a proprietary model is informative and lands below it. We give a quantitative criterion for \emph{vacuous} oversight, and replaying policies on recorded traces confirms the ordering.

Cesare Zavattari; Alessandro Tommasi; Giuseppe Prencipe


Resource entry template

The resource is directly reusable as a starting artifact. Format for adding a single resource with evidence quality and category fit.

Use Resource entry template to reuse a concrete artifact or connect it to the wider ecosystem.

Repository file; inspect the linked schema, example, guide, or implementation.

Reuse, adapt, and contribute concrete loop artifacts.


Loop pattern template

The resource is directly reusable as a starting artifact. Template for documenting an operational loop such as PR babysitting, CI repair, or feedback clustering.

Use Loop pattern template to reuse a concrete artifact or connect it to the wider ecosystem.


Loop contract schema

The contribution is machine-readable and validation-friendly. Machine-readable schema for portable loop specs.

Use Loop contract schema to reuse a concrete artifact or connect it to the wider ecosystem.


Loop contract preview script

The contribution is machine-readable and validation-friendly. Dependency-free demo that validates and renders a loop contract JSON file.

Use Loop contract preview script to reuse a concrete artifact or connect it to the wider ecosystem.


Translation guide

The resource is directly reusable as a starting artifact. How to add or maintain a language translation without drifting from the full English guide.

Use Translation guide to reuse a concrete artifact or connect it to the wider ecosystem.


Pattern library index

Verification is promoted from a final check to a loop-control signal. Practical loop patterns with triggers, state, verification gates, budgets, and escalation paths.

Use Pattern library index to reuse a concrete artifact or connect it to the wider ecosystem.


Loop contract catalog

Durable execution and replay are treated as first-class loop infrastructure. Connects all 22 patterns to schema-valid contracts, deterministic gates, durable receipts, and four worked implementation paths.

Use Loop contract catalog to reuse a concrete artifact or connect it to the wider ecosystem.


Loop contract library

The contribution is machine-readable and validation-friendly. Adaptable contracts for build, operate, optimize, and govern loops, checked against the shared schema with explicit permissions, budgets, and handoffs.

Use Loop contract library to reuse a concrete artifact or connect it to the wider ecosystem.


Runnable test-repair loop

Durable execution and replay are treated as first-class loop infrastructure. Repeats a failing deterministic check with durable progress, duplicate-failure detection, and a hard retry budget.

Use Runnable test-repair loop to reuse a concrete artifact or connect it to the wider ecosystem.


Runnable loop guide

State persistence is explicit enough for repeated runs and handoff. Compares 8 starters by trigger, state, gate, and runtime, including executable test-repair, threshold-monitor, and queue-worker loops.

Use Runnable loop guide to reuse a concrete artifact or connect it to the wider ecosystem.


Loop gallery guide

The resource is directly reusable as a starting artifact. Quality bar for contributed loop examples with receipts and lessons learned.

Use Loop gallery guide to reuse a concrete artifact or connect it to the wider ecosystem.


Loop gallery template

Verification is promoted from a final check to a loop-control signal. Markdown template for sharing a loop's trigger, intake, state, verification, escalation, and safety notes.

Use Loop gallery template to reuse a concrete artifact or connect it to the wider ecosystem.


PR babysitter reference loop

Turns loop adoption into shareable cases with enough structure to compare lessons learned. Reference gallery entry for keeping a pull request moving.

Use PR babysitter reference loop to reuse a concrete artifact or connect it to the wider ecosystem.


CI repair reference loop

Verification is promoted from a final check to a loop-control signal. Reference gallery entry for turning failing CI into a verified patch or escalation.

Use CI repair reference loop to reuse a concrete artifact or connect it to the wider ecosystem.


Docs drift reference loop

Turns loop adoption into shareable cases with enough structure to compare lessons learned. Reference gallery entry for recurring docs/code consistency checks.

Use Docs drift reference loop to reuse a concrete artifact or connect it to the wider ecosystem.


Keeps adoption grounded in known failure modes, economics, and operational limits. Useful caution against adopting loops before the task, signal, and economics justify them.

Use Most Developers Do Not Need Agent Loops Yet to bound risk before recurring or unattended execution.

Contextual source from alphasignalai.substack.com; useful for practice signals or boundary conditions, not independent validation.

The patterns were documented in 2024. Here’s who it pays off for, and the four conditions that decide.


Verification is promoted from a final check to a loop-control signal. Cautions that agentic systems fail at boundaries when permissions, verification, traceability, and escalation are weak.

Use Engineering Agentic Systems for Reliability to bound risk before recurring or unattended execution.

Contextual source from pruningmypothos.com; useful for practice signals or boundary conditions, not independent validation.

Engineering Agentic Systems for Reliability | Sans Serif Systems

A practical reliability model for agentic systems built around governed steps, verification, escalation, and observability.


Keeps adoption grounded in known failure modes, economics, and operational limits. Compares self-correction patterns and their cost/failure tradeoffs.

Use Self-Correcting Agents: Reflexion, CRITIC, and ReAct Loops Compared to bound risk before recurring or unattended execution.

Contextual source from callsphere.ai; useful for practice signals or boundary conditions, not independent validation.

Self-Correcting Agents: Reflexion, CRITIC, and ReAct Loops Compared | CallSphere Blog

Three self-correction patterns dominate 2026 agent design. Side-by-side analysis of where each one wins, where each one fails, and how to combine them.


Evaluation data is used as the feedback signal for improving loop behavior. Broad guide with useful warnings on data readiness, permissions, context management, and evaluation.

Use How to Build an AI Agent Harness: A 2026 Complete Guide to bound risk before recurring or unattended execution.

Contextual source from atlan.com; useful for practice signals or boundary conditions, not independent validation.

How to Build an AI Agent Harness: Step-by-Step Tutorial (2026)

Most agent harnesses fail at the data layer, not the loop. Build one the right way in 10 steps, with code and a done test for each. Start at Step 0.


Context is managed as durable loop state rather than a single prompt payload. Adjacent framing that helps avoid confusing loop engineering with the surrounding harness discipline.

Use Harness Engineering vs Prompt Engineering vs Context Engineering Explained to bound risk before recurring or unattended execution.

Contextual source from medium.com; useful for practice signals or boundary conditions, not independent validation.


The work turns loop quality into a measurable task or score. Argues benchmark scores conflate the model with the harness and penalize valid alternatives, so headline numbers hide which loop and harness choices actually move performance.

Use Position: Coding Benchmarks Are Misaligned with Agentic Software Engineering to bound risk before recurring or unattended execution.

Research source arXiv:2606.17799; inspect its method and evaluation before treating results as production evidence.

[2606.17799] Position: Coding Benchmarks Are Misaligned with Agentic Software Engineering

Coding agents have become a major mode of software engineering, but the benchmarks we use to compare them were designed in a pre-agent era: they collapse model, harness, and environment into a single end-to-end score, typically computed against one reference solution, with no component-level signal for iteration. We argue that current coding benchmarks are misaligned with agentic software engineering. A coding agent in practice is not a model: it is a system harness -- a composite of models, harnesses, contexts, environments, and feedback signals, any one of which can move the benchmark score by margins comparable to those between adjacent model generations. We discuss three symptoms: (i) benchmark scores conflate the model with the rest of the harness; (ii) grading against a single reference solution penalises equally valid alternatives; and (iii) the absence of signal at the level of individual harness components makes the end-to-end system score difficult to iterate on.

Maria I. Gorinova; Macey Baker; Amy Heineike; Maksim Shaposhnikov; Rob Willoughby; Dru Knox


Keeps adoption grounded in known failure modes, economics, and operational limits. Empirically isolates three hidden design choices that make self-improving agent loops succeed or fail - starting artifacts, credit horizons over execution traces, and batching strategy - explaining why iterative refinement loops stay brittle in production.

Use Understanding the Challenges in Iterative Generative Optimization with LLMs to bound risk before recurring or unattended execution.

Research source arXiv:2603.23994; inspect its method and evaluation before treating results as production evidence.

[2603.23994] Understanding the Challenges in Iterative Generative Optimization with LLMs

Generative optimization uses large language models (LLMs) to iteratively improve artifacts (such as code, workflows or prompts) using execution feedback. It is a promising approach to building self-improving agents, yet in practice remains brittle: despite active research, only 9% of surveyed agents used any automated optimization. We argue that this brittleness arises because, to set up a learning loop, an engineer must make ``hidden'' design choices: What can the optimizer edit and what is the "right" learning evidence to provide at each update? We investigate three factors that affect most applications: the starting artifact, the credit horizon for execution traces, and batching trials and errors into learning evidence. Through case studies in MLAgentBench, Atari, and BigBench Extra Hard, we find that these design decisions can determine whether generative optimization succeeds, yet they are rarely made explicit in prior work. Different starting artifacts determine which solutions are reachable in MLAgentBench, truncated traces can still improve Atari agents, and larger minibatches do not monotonically improve generalization on BBEH. We conclude that the lack of a simple, universal way to set up learning loops across domains is a major hurdle for productionization and adoption. We provide practical guidance for making these choices.

Allen Nie; Xavier Daull; Zhiyi Kuang; Abhinav Akkiraju; Anish Chaudhuri; Max Piasevoli; Ryan Rong; YuCheng Yuan; Prerit Choudhary; Shannon Xiao; Rasool Fakoor; Adith Swaminathan; Ching-An Cheng


The Illusion of Multi-Agent Advantage

Evaluation data is used as the feedback signal for improving loop behavior. Systematic evaluation showing automatically generated multi-agent systems consistently underperform chain-of-thought self-consistency while costing up to 10x more, cautioning that auto-designed orchestration adds complexity without functional benefit.

Use The Illusion of Multi-Agent Advantage to bound risk before recurring or unattended execution.

Research source arXiv:2606.13003; inspect its method and evaluation before treating results as production evidence.

[2606.13003] The Illusion of Multi-Agent Advantage

Prevailing wisdom posits that Multi-Agent Systems (MAS) are superior to Single-Agent Systems (SAS), citing advantages like context protection, parallel processing and distributed decision-making. However, empirical support for this claim relies primarily on comparisons with SAS baselines using benchmarks that prioritize isolated reasoning tasks, which do not adequately assess these advantages. Focusing on automatically generated MAS that are designed for enhanced generalizability over manually-designed counterparts, we perform a rigorous, systematic evaluation against SAS, specifically Chain-of-Thought with Self-Consistency (CoT-SC). Across traditional reasoning datasets and tasks with interactive multi-step workflows (e.g., BrowseComp-Plus), we demonstrate that automatic MAS consistently underperform CoT-SC despite being up to 10x more expensive. To isolate these failures from limitations inherent to task structure, we introduce a diagnostic synthetic dataset tailored for MAS featuring explicit task decomposition, context separation and parallelization potential. We show that expert-architected MAS consistently outperforms automatically generated architectures in both raw performance and cost-efficiency on this dataset, demonstrating that existing evaluation frameworks mask critical architectural gaps and inefficiencies of complex MAS by failing to account for the marginal utility of increased computational cost. Critically, systematic deconstruction of the generated MAS architectures reveals that current automated design paradigms produce architectural bloat that prioritizes superficial complexity which does not translate into functional utility, exposing a fundamental misalignment with multi-agent principles.

Prathyusha Jwalapuram; Hehai Lin; Chuyuan Li; Fangkai Jiao; Sudong Wang; Yifei Ming; Zixuan Ke; Chengwei Qin; Giuseppe Carenini; Shafiq Joty


The Coming Loop

Keeps adoption grounded in known failure modes, economics, and operational limits. Flask creator Armin Ronacher's skeptical essay on harness loops, examining what continuously re-driving agents past their natural stopping points does to code quality, review capacity, and human understanding of the resulting systems.

Use The Coming Loop to bound risk before recurring or unattended execution.

Contextual source from lucumr.pocoo.org; useful for practice signals or boundary conditions, not independent validation.

The Coming Loop | Armin Ronacher's Thoughts and Writings

Loops, harnesses, and why even loop skeptics may end up with them.


Keeps adoption grounded in known failure modes, economics, and operational limits. The Register's report on the June 2026 loop-engineering discussion, collecting the Steinberger, Osmani, and Cherny quotes while arguing that vendor token-consumption incentives and model non-determinism keep humans in the loop.

Use Loop Engineering, the Latest AI Buzzword, Still Needs Humans in the Loop to bound risk before recurring or unattended execution.

Contextual source from www.theregister.com; useful for practice signals or boundary conditions, not independent validation.

Loop engineering, latest AI buzzword, still needs humans in the loop

Prompting less and automating more comes with a price


Keeps adoption grounded in known failure modes, economics, and operational limits. Characterizes infinite agentic loops, a failure class where unbounded feedback paths make agents repeat calls, tools, or handoffs forever, and ships IAL-Scan, a static analyzer that confirmed 68 real cases across 47 of 6,549 scanned agent projects at 91.9% precision.

Use When Agents Do Not Stop: Uncovering Infinite Agentic Loops in LLM Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.01641; inspect its method and evaluation before treating results as production evidence.

[2607.01641] When Agents Do Not Stop: Uncovering Infinite Agentic Loops in LLM Agents

LLM agents increasingly rely on iterative execution to solve tasks through planning, tool use, state updates, and agent collaboration. While this design enables flexible automation, it also creates a new class of failures: an agent may repeatedly execute model calls, tools, workflow transitions, or agent handoffs when the feedback path is not effectively bounded. We call this problem Infinite Agentic Loops (IALs). IALs are not ordinary programming loops; they arise from the interaction between agent logic, framework semantics, runtime observations, and termination mechanisms. Such failures can amplify a single request into long running model and tool execution, causing cost exhaustion, model denial of service, context growth, and repeated external side effects. We propose IAL-Scan, a static analysis tool for detecting IAL failures in real-world LLM agent projects. IAL-Scan abstracts heterogeneous agent code into a framework independent Agent IR, builds an Agentic Loop Dependence Graph (ALDG) to recover explicit and framework induced feedback paths, and checks whether these paths can repeatedly reach costly or state growing operations without an effective bound. We evaluate IAL-Scan on 6,549 LLM agent repositories. It reports 74 potential findings, among which manual review confirms 68 IAL failures across 47 projects, achieving 91.9% precision.

Xinyi Hou; Shenao Wang; Yanjie Zhao; Haoyu Wang


Keeps adoption grounded in known failure modes, economics, and operational limits. Shows via corrupted-reward analysis that false-pass bias in an LLM judge silently disables the skill-retirement mechanism that keeps a self-evolving agent's growing skill library from drifting below the no-skill baseline.

Use The Blind Curator: How a Biased Judge Silently Disables Skill Retirement in Self-Evolving Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.07436; inspect its method and evaluation before treating results as production evidence.

[2607.07436] The Blind Curator: How a Biased Judge Silently Disables Skill Retirement in Self-Evolving Agents

A self-evolving agent retires its bad skills by watching them fail, so what happens when the judge cannot see the failures? Skill retirement is the structural constraint that keeps a growing library from drifting below the no-skill baseline, but its guarantee assumes an unbiased reward, which is false for the LLM judges that reference-free tasks force upon us. We show that a biased judge does not merely add noise; it \emph{silently switches off the curator}. We make this precise with a corrupted-reward analysis and, isolating the causal channel by injecting corruption on top of a deterministic reward, a behavioral study on a reference-free report-writing testbed with a code-generation cross-check. Symmetric noise leaves retirement intact, but \emph{false-pass} bias (failures slipping through as passes) disables contribution-based retirement past a sharp threshold that no amount of data can cross. Separating genuine retirement from cap-eviction churn shows this \emph{mechanism} failure is universal, holding across domains and failure rates and sparing only near-zero-false-pass, verifier-like graders. The downstream \emph{outcome}, though, is regime-dependent: eval quality degrades only where the same corruption also starves skill synthesis, and otherwise holds steady, so the disabled curator is \emph{silent}, surfacing in no aggregate metric. The contribution is a behavioral safety result, not a performance one. A cheap defect-injection audit then tells an operator, before deployment, which side of the threshold their judge occupies.

Xing Zhang; Yanwei Cui; Guanghui Wang; Ziyuan Li; Wei Qiu; Bing Zhu; Peiyang He


Keeps adoption grounded in known failure modes, economics, and operational limits. Negative result for low-curation skill libraries: across four data-science lifecycle stages (56 tasks), fully LLM-generated skill files show no reliable improvement over plain task prompting, and component ablations find no skill part that helps either (all p > 0.396). Useful counterweight to the skill-generation enthusiasm in self-evolving agent stacks, curation still matters.

Use Do LLM-Generated Skills Make Better AI Data Scientists? A Component Ablation Across Data-Science Workflows to bound risk before recurring or unattended execution.

Research source arXiv:2607.07504; inspect its method and evaluation before treating results as production evidence.

[2607.07504] Do LLM-Generated Skills Make Better AI Data Scientists? A Component Ablation Across Data-Science Workflows

Product data scientists often ask LLM-based agents to help with recurring execution tasks such as cleaning data, writing SQL, choosing statistical tests, and formatting results. Reusable skill files are meant to avoid prompting from scratch by packaging guidance for a task family. Expert-written skills can encode high-quality guidance, but writing and maintaining them across many data-science task families creates a manual bottleneck. We ask whether LLM-generated skills offer a useful low-curation alternative: do they improve performance over the task prompt alone? We test this question across four lifecycle stages: data preparation, data extraction, statistical analysis, and reporting, using one generated skill per stage. We find no reliable improvement from full generated skills over No-Skill prompting. We then ask whether any part of the skill is useful by ablating different skill components. The main ablation covers 56 tasks, nine model configurations, and three providers, yielding 7,560 runs. Compared with prompting using the task alone, neither the full generated skill nor any ablated skill variant significantly improves performance; all p-values are at least 0.396, and the total spread across variants is only 1.2 pp. A supplemental token-matched control adds 1,512 runs and finds that Full skills perform similarly to task-irrelevant skill-formatted content. The results caution against using one LLM-generated skill per data-science workflow as a default single-shot prompting strategy.

Accepted at KDD Workshop on AI Data Scientist; the linked arXiv record is the available paper version.

Current arXiv acceptance note and official workshop page


Verification is promoted from a final check to a loop-control signal. Position paper arguing verification has become harder than generation for coding agents: every verifier is only a proxy for underspecified human intent, so reward design faces a horizon that no single verification mechanism crosses.

Use The Verification Horizon: No Silver Bullet for Coding Agent Rewards to bound risk before recurring or unattended execution.

Research source arXiv:2606.26300; inspect its method and evaluation before treating results as production evidence.

[2606.26300] The Verification Horizon: No Silver Bullet for Coding Agent Rewards

A classical intuition holds that verifying a solution is easier than producing one. For today's coding agents, this intuition is being inverted: as foundation models develop stronger reasoning capabilities and engineering harnesses grow more sophisticated, generating complex candidate solutions is no longer difficult -- reliably verifying them has become the harder problem. Every verifier we can build is only a proxy for human intent, never the intent itself. This makes verification subject to a twofold difficulty: first, intent is underspecified by nature, making it inherently hard to faithfully check whether it has been fulfilled; second, during model training, optimization widens the gap between proxy and intent -- manifesting as reward hacking or signal saturation. To address this, we characterize the quality of verification signals along three dimensions -- scalability, faithfulness, and robustness -- and argue that achieving all three simultaneously is the central challenge. We further study four reward constructions: a test verifier for general coding tasks, a rubric verifier for frontend tasks, the user as verifier for real-world agent tasks, and an automated agent verifier for long-horizon tasks. Across different task types and policy capability levels, we conduct in-depth analysis and experiments on the core challenges of reward design and how to more effectively leverage reward signals. Experiments show that targeted verification design can effectively suppress reward hacking, improve task completion quality, and achieve significant gains across multiple internal and public benchmarks. These experiences collectively point to a core observation: no fixed reward function can remain effective as policy capability continues to grow; and verification must co-evolve with the generator.

Binghai Wang; Chenlong Zhang; Dayiheng Liu; Jiajun Zhang; Jiawei Chen; Mingze Li; Mouxiang Chen; Rongyao Fang; Siyuan Zhang; Xuwu Wang; Yuheng Jing; Zeyao Ma; Zeyu Cui

Authors are listed alphabetically by their first names


Keeps adoption grounded in known failure modes, economics, and operational limits. Argues that agent-driven codebases create a compounding feedback loop where every merged shortcut becomes training signal for the next generation of changes, so code quality standards matter more, not less, under automation.

Use Write Code Like a Human Will Maintain It to bound risk before recurring or unattended execution.

Contextual source from unstack.io; useful for practice signals or boundary conditions, not independent validation.

Write code like a human will maintain it

One of the best things about LLMs is that they'll write code for you, all day long. Who cares about DRY? You don't have to be the one updating the same long con...


The work separates roles across agents, verifiers, or orchestration layers. July 12, 2026 proxy-interception study of per-turn harness overhead: Claude Code sends ~33k tokens of scaffolding before user input versus OpenCode's ~7k (a 4.7x gap that narrows to 3.3x on newer models), mid-session cache-block rewrites produce up to 54x more cache-write tokens on identical tasks, a 72KB instruction file adds ~20k tokens per request, five MCP servers add 5-7k more, and subagent delegation alone multiplied total cost 4.2x.

Use Claude Code Sends 33k Tokens Before Reading the Prompt to bound risk before recurring or unattended execution.

Contextual source from systima.ai; useful for practice signals or boundary conditions, not independent validation.

Claude Code Sends 4.7x More Tokens Than OpenCode Before Reading Your Prompt | Systima Blog

Claude Code vs OpenCode token overhead measured at the API boundary. Out-of-the-box baselines, instruction file weight, MCP schema tax, subagent multipliers, and cache-write behaviour.


Evaluation data is used as the feedback signal for improving loop behavior. Re-evaluates harness evolution on Terminal-Bench 2.1 with GPT-5.4 and Claude Opus 4.6, finding that evolved harnesses do not consistently beat budget-matched search and transfer only weakly to held-out tasks.

Use Rethinking the Evaluation of Harness Evolution for Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.12227; inspect its method and evaluation before treating results as production evidence.

[2607.12227] Rethinking the Evaluation of Harness Evolution for Agents

We revisit the evaluation of automatic harness evolution for LLM agents. Existing harness evolution methods use unit test cases to search for harness configurations and then report final performance on the same public benchmark. This protocol raises two fundamental concerns. First, harness evolution is itself an iterative search procedure that repeatedly evaluates and revises candidate harnesses using task feedback. As in agentic test-time scaling, it should therefore be compared with simple task-level search baselines under matched feedback and inference budgets to determine whether its gains arise from improved harness design or from additional search alone. Second, because the search and the final evaluation share the same benchmark, the reported gains risk overfitting to that specific task set. To address these concerns, we conduct an extensive evaluation comparing harness evolution with simple test-time scaling and discovery baselines under comparable feedback and inference budgets, and also evaluate evolved harnesses on held-out tasks to assess whether the discovered improvements generalize. Experiments on Terminal-Bench 2.1 with GPT-5.4 and Claude Opus 4.6 show that automatic harness evolution does not consistently outperform simple test-time scaling methods and exhibits limited generalization. Our results raise important questions about the effectiveness of automatic harness evolution and highlight the need for fairer evaluation protocols and benchmarks for automatic harness design. Our code is available at https://github.com/rethinking-harness-evolution.

Yike Wang; Huaisheng Zhu; Zhengyu Hu; Yige Yuan; Zhengyu Chen; Shakti Senthil; Hannaneh Hajishirzi; Yulia Tsvetkov; Pradeep Dasigi; Teng Xiao


Verification is promoted from a final check to a loop-control signal. Documents a Claude Code failure in which partial output from a process killed with exit 143 becomes a confirmed claim after context compaction, without re-verification, showing why receipts and process status must survive summarization.

Use Compaction as Epistemic Failure: How Agentic LLM Tools Fabricate Confirmed Results from Killed Processes to bound risk before recurring or unattended execution.

Research source arXiv:2607.13071; inspect its method and evaluation before treating results as production evidence.

[2607.13071] Compaction as Epistemic Failure: How Agentic LLM Tools Fabricate Confirmed Results from Killed Processes

Agentic LLM coding tools compress long session histories into compaction summaries that subsequent sessions inherit as ground truth. This paper documents a failure mode in Claude Code where partial standard output from timed-out commands (exit code 143) is recorded in compaction summaries as confirmed results, propagating false positives across sessions and model versions without re-verification. The underlying mechanism is a conflation of observation and persistence, where information that appeared in the terminal is treated as equivalent to information written to durable storage. This finding extends the analysis of LLM self-evaluation failures reported in prior work on non-determinism in LLM-as-judge grading by showing that agentic tools exhibit analogous reliability deficits when reporting on their own operational outcomes. The failure has direct implications for any workflow that relies on agentic session continuity for data processing, scientific computation, or multi-step automation.


Evaluation data is used as the feedback signal for improving loop behavior. Compares GEPA, Meta Harness, and RELAI-VCL under matched continual-learning budgets; only the regression-controlled method keeps improving, reaching 76.4% lifelong performance versus 66.0%, 64.6%, and 58.7% for the reported alternatives.

Use Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0 to bound risk before recurring or unattended execution.

Research source arXiv:2607.14004; inspect its method and evaluation before treating results as production evidence.

[2607.14004] Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0

Most reported gains from agent-optimization methods are one-shot: an agent is optimized against a fixed benchmark and the resulting improvement is reported as if it were a stable property of the method. This does not test the setting that matters for deployed agents, where optimization is applied recursively as new failures and new tasks appear over time. The central question this raises is whether optimizer-driven gains compound: after an agent has been optimized once, can it be optimized again on newly arrived tasks without eroding the gains the first round produced? We study this question with a two-phase continual-learning evaluation built from hard tasks in Terminal-Bench 2.0, comparing three approaches to agent-harness optimization (GEPA, Meta Harness, and RELAI's Verifiable Continual Learning, RELAI-VCL) under identical optimization budgets. All three methods improve over the baseline agent in the conventional, static, single-phase setting. However, once new tasks are introduced, the methods diverge sharply: GEPA's optimized agent transfers below the unoptimized baseline, Meta Harness transfers well but fails to improve further once given a second optimization budget, and RELAI-VCL is the only method that both transfers positively to unseen tasks and continues improving after those tasks are folded into the optimization objective, reaching the highest pass rate at every evaluated stage and the highest lifelong average pass rate overall (76.4% vs. 66.0% for GEPA, 64.6% for Meta Harness, and 58.7% for the baseline). Our key observation was that optimization gains compounded only when regression control was built into the optimization loop, providing an inductive bias against shortcut solutions that fail to generalize.

Wenxiao Wang; Priyatham Kattakinda; Soheil Feizi


Evaluation data is used as the feedback signal for improving loop behavior. In a preregistered masked study with authors of 44 meta-analyses, participants prefer single-pass feedback to two multi-agent debate systems, one using about 30x more tokens; AI judges reverse the human preference, warning against self-evaluation alone.

Use Does Multi-Agent Debate Improve AI Feedback on Research Papers? to bound risk before recurring or unattended execution.

Research source arXiv:2607.14713; inspect its method and evaluation before treating results as production evidence.

[2607.14713] Does Multi-Agent Debate Improve AI Feedback on Research Papers?

Probably not, at least for meta-analyses in economics. In a pre-registered, identity-masked, within-paper experiment, the authors of 44 meta-analyses ranked three AI reports on their own paper by usefulness for improving it: a single pass by a frontier model against two multi-agent debate tools we built and expected to win. All reports were held to a common length and template. The authors preferred the single pass, by 0.66 rank points over mad-research (95% CI 0.32 to 1.00) and 0.57 over paper-workshop (0.16 to 0.95), though paper-workshop spent roughly thirty times the tokens. Authors who recalled their journal referee report usually placed it first and never last; in a separate exercise, three AI judges almost always placed the real journal referee report last. Among the three AI reports, Gemini (the judge whose model family wrote none of the reports) would have ranked paper-workshop first in the authors' place, reversing the single-pass preference. The reversal warns against substituting an AI judge for the author. We measure perceived usefulness for finished papers; whether AI should referee papers is a separate question.

29 pages, 1 figure, 6 tables. Pre-registered on OSF; data, code, judge prompts, and blinded reports in the replication package on Zenodo. Project page: https://meta-analysis.cz/debate


Verification is promoted from a final check to a loop-control signal. Isolates 'binding drift', entity bindings that start correct then silently go wrong across sequential tool-calling steps, from ordinary error propagation, using 200 workflows and 580 entity-binding-scored steps across four enterprise domains and eight model backends; a naive entity-lock persistence mechanism amplifies injected wrong actions 3.0x on average (up to 8.5x on the most affected model), while an LLM re-verification pass cuts wrong actions 79%.

Use Binding Drift in Multi-Step Tool-Augmented Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.18316; inspect its method and evaluation before treating results as production evidence.

[2607.18316] Binding Drift in Multi-Step Tool-Augmented Agents

Tool-augmented language-model agents execute multi-step workflows over external systems, resolving an entity once and then acting on it across subsequent steps. Prior work shows that in single-step actions, agents select the correct tool but bind it to the wrong entity 24-26% of the time. We study what happens to entity bindings over time: do they stay correct, silently drift to a different entity, or, if wrong from the start, propagate and compound? We formalize binding drift (correct at step 1, wrong later) as distinct from error propagation (wrong at step 1, carried forward), and score them on disjoint workflow sets so the two cannot be conflated. In a controlled multi-step testbed (200 workflows, 580 entity-binding-scored steps, four enterprise domains, eight model backends spanning small to frontier), we find: (1) under controlled error injection, an entity lock (the intuitive "persist the first binding" fix) amplifies wrong actions from 907 to 2,746 (3.0x; bootstrap 95% CI [2.8, 3.3]), because it faithfully carries the seeded wrong entity into every later step; (2) the amplification reaches 8.5x on the most affected model (Claude Opus 4.5); (3) a practical LLM-based re-verifier (a single cheap second model call re-reading the original instruction) reduces wrong actions by 79% (0.21x; CI [0.18, 0.25]), closing the gap to within 1 percentage point of an oracle upper-bound (0.20x); and (4) in the natural (non-injected) setting, baseline agents drift on 18% of eligible workflows, with the per-step error rate rising across steps. Persistence and re-verification are not interchangeable: a defense that eliminates drift can worsen propagation, and a practical re-verifier nearly matches oracle recovery.

14 pages, 5 tables, 1 figure. Equal contribution by both authors. Code and data: https://github.com/shashank-indukuri/binding-drift


Context is managed as durable loop state rather than a single prompt payload. White-box study of skill-following degradation over long tool-using trajectories in a code-audit workflow: pass rates fall from 8/10 in clean context to 3/10 at ~299K characters even though requirement coverage stays above 92%, a detailed external checklist restores 10/10 versus 5/10 for generic self-check, and the paper taxonomizes the failure modes (lost requirements, editing drift, failed checking) that skill-driven loops must engineer around, a small single-model study of 10 runs per condition.

Use How Agent Skills Fail under Long Contexts: A White-Box Study in Code Auditing to bound risk before recurring or unattended execution.

Research source arXiv:2607.17937; inspect its method and evaluation before treating results as production evidence.

[2607.17937] When and How Context Rot Appears in Coding Agents: A White-Box Study of Agent Skills in Code Auditing

Agent Skills package procedural instructions and checks for use by general-purpose agents, but loading a skill does not guarantee that every requirement remains active throughout a long tool-using trajectory. We study this problem in a production-derived, white-box code-audit workflow. Holding the task and 24 artifact checks fixed, we vary the surrounding context and classify where failures first become visible: lost requirements, editing drift, failed checking, or non-agent evaluator/runtime failures. Codex with gpt-5.4-mini passes 8/10 runs in a 10,991-character clean context but only 3/10 in both a 299,140-character relevant context and an equal-length irrelevant context. This 50-percentage-point difference is large but remains trend-level under two-sided Fisher tests (p = 0.0698). Requirement coverage nevertheless stays above 92% in both long conditions, showing that a few omissions can invalidate an otherwise complete artifact. A second task passes all clean and long runs, so the evidence does not support a universal context-length threshold. A detailed external checklist passes 10/10 runs, compared with 5/10 for a generic self-check (p = 0.0325). Coding-agent scaffolds may help by selecting a smaller working set, but they do not eliminate failures. We do not introduce context rot or a new general monitoring method; we provide a bounded failure classification and empirical case study for white-box code auditing.


State persistence is explicit enough for repeated runs and handoff. Names a failure mode of self-improving harness loops: the proposer LLM fabricates guardrails for failure classes that provably never occurred (15 of 60 runs when input merely resembles a familiar rule), and once inside an add-only accept loop the phantom guardrail persists; ships a deterministic micro-lab with byte-exact oracle checks.

Use Phantom Guardrails: When Self-Improving Agent Harnesses Fix Failures That Never Happened to bound risk before recurring or unattended execution.

Research source arXiv:2607.13083; inspect its method and evaluation before treating results as production evidence.

[2607.13083] Phantom Guardrails: When Self-Improving Agent Harnesses Fix Failures That Never Happened

Self-improving AI agents are designed to learn from their mistakes. We show they can also hallucinate mistakes that never happened. We study this failure mode in automated harness optimization, where an LLM-based proposer edits an agent's scaffold, including prompts, parsers, filters, validators and guardrails, to eliminate observed failures. But this process rarely asks first: was there a real failure to fix? We introduce the Counterfactual Fabrication Lab, a deterministic micro-lab where the correct action is known: do nothing. The lab plants a candidate guardrail for a failure class that provably never occurs, presents only legal episodes, and uses a byte-exact oracle to check every cited violation. The proposer behaves as expected on real violations and abstains on featureless legal input. Yet when the legal input contains a harmless pattern resembling a familiar game rule, it invents a failure: in 15/60 runs, versus 0/60 on featureless input, it enables the nonexistent-rule guardrail and cites a violation the oracle refutes. The effect is structured, not indiscriminate. In single-shot proposals it appears only when three conditions coincide: a rule-shaped pattern, an open-ended rule set and an instruction that presupposes failures. Removing any of these conditions eliminates the fabrication. Because the invented guardrail changes no true outcome and cannot improve an already-perfect suppression score, the phenomenon is neither reward hacking nor over-refusal. It is a phantom guardrail: a fix for a failure that never happened, invisible to suppression-only acceptance. Inside an add-only accept loop it re-enters even without the failure-presupposing instruction, the loop's keep-adding role supplying the demand the instruction supplied in single shot, and once in it stays. We present the Counterfactual Fabrication Lab for measuring fabricated failures in self-improving agent harnesses.

Su Wang; Pin Qian; Yifan Lin; Jingzhou Xu; Yihang Chen; Xiaochong Jiang; Lifei Liu; Haoran Yu


Keeps adoption grounded in known failure modes, economics, and operational limits. Measures a 36% average rate of agents recommending non-existent skills across 15,000 prompts, and shows the same fake names recur consistently, enabling slopsquatting-style supply-chain attacks where adversaries pre-register malicious skills under the hallucinated names agents will predictably ask for.

Use Skills That Don't Exist: A Large-Scale Study of Hallucinated Skill Recommendation to bound risk before recurring or unattended execution.

Research source arXiv:2607.12340; inspect its method and evaluation before treating results as production evidence.

[2607.12340] Skills That Don't Exist: A Large-Scale Study of Hallucinated Skill Recommendation in LLM Agents

LLM agents acquire new capabilities by downloading skills from open registries. Instead of browsing these catalogs manually, developers typically ask the agent to recommend and install a skill. This convenience hides a risk: agents frequently invent names for skills that exist in no registry. We term this flaw skill name hallucination. A fake name may seem harmless, but it opens the door to supply-chain attacks. Because registries rarely verify publishers, an adversary can prompt the agent, collect the fake names it returns, pre-register malicious skills under them, and wait for a victim to install the payload. We conducted the first large-scale measurement of skill name hallucination, evaluating 15,000 prompts across 12 configurations (4 standalone LLMs and 8 agents). We conservatively counted a name as hallucinated only if it was missing from all live registries and GitHub. The results reveal a systemic vulnerability: every configuration hallucinates. Rates average 36.0% for standalone LLMs and 36.9% for agents, rising to 43.1% on real-world developer questions. In total, the systems generated 5,669 distinct hallucinated names. Crucially, these names are not random noise. Agents repeat the same fake names across prompts and models, giving attackers highly reliable targets to hijack. Finally, we tested four model-level defenses and found a severe conflict between security and usability. The strongest, retrieval grounding, cut the hallucination rate from 40.8% to 3.2% but crippled usefulness: even the best-defended system recommended the correct skill only about one in six times. Skill name hallucination is thus a highly exploitable vulnerability requiring minimal attacker effort. Fixing it cannot rely on prompt engineering or model tuning alone. It demands ecosystem-wide structural changes: registry-level name reservations and verified recommendation pipelines.

Weifeng Yuan; Wenbo Guo; Feng Dong; Haoyu Wang; Yang Liu


Token Reduction Is Not Cost Reduction

Context is managed as durable loop state rather than a single prompt payload. Analysis of 2,848 real Claude Code runs showing prompt-cache traffic accounts for ~87% of billed cost, so local token/context compression does not reliably lower the bill, argues loop cost engineering should optimize success-adjusted billed cost, not token counts. Directly actionable for anyone running high-volume agent loops.

Use Token Reduction Is Not Cost Reduction to bound risk before recurring or unattended execution.

Research source arXiv:2607.12161; inspect its method and evaluation before treating results as production evidence.

[2607.12161] Token Reduction Is Not Cost Reduction

Context-reduction layers for API-based coding agents, including command-output compressors, retrieval rankers, and API-boundary proxies, are commonly evaluated by how much context or tool output they remove. We ask a different question: which interventions actually reduce end-to-end billed cost while preserving task success? Our primary evidence is a pre-specified, hash-frozen, paired campaign of 2,908 provider-billed Claude Code runs, of which 2,848 were analyzed, covering 103 tasks, seven repositories, and three models. The campaign compared a baseline with two generations of hook-based compression and an API-boundary proxy within a broader measured program of roughly 5,500 billed executions. Three findings emerge. First, prompt-cache traffic dominated cost composition, accounting for about 87% of reconstructed four-component cost (about 80% of the actual bill), with an 8.7% dollar-weighted residual not attributable from retained telemetry. Second, local payload reduction was not a reliable predictor of end-to-end billed cost. An arm that removed 38% of estimated raw tool-output tokens incurred 6.8% higher paired cost (95% CI: +2.8% to +11.3%), while per-task reduction showed only a weak association with cost change (Pearson r = 0.15). Third, aggressive compression can remove action-critical evidence: on SWE-bench-derived Go tasks, compression reduced successful patch application from 27/40 to 15/40 by corrupting verbatim edit anchors. We propose evaluating context-reduction systems by success-adjusted billed cost rather than token reduction alone.


Keeps adoption grounded in known failure modes, economics, and operational limits. Wire-level analysis of the telemetry and payloads Grok Build transmits, a reminder that agent harnesses carry their own data-flow surface worth auditing before unattended use.

Use What xAI's Grok Build CLI Actually Sends: A Wire-Level Analysis to bound risk before recurring or unattended execution.

Contextual source from gist.github.com; useful for practice signals or boundary conditions, not independent validation.

What xAI Grok Build CLI actually sends to xAI - a wire-level analysis (grok 0.2.93) · GitHub

What xAI Grok Build CLI actually sends to xAI - a wire-level analysis (grok 0.2.93) - grok-build-cli-wire-analysis.md


The Tower Keeps Rising

Keeps adoption grounded in known failure modes, economics, and operational limits. Armin Ronacher's follow-up to The Coming Loop, on abstraction layers accumulating faster than understanding as agent tooling stacks up, and what that does to a codebase's long-term comprehensibility.

Use The Tower Keeps Rising to bound risk before recurring or unattended execution.

Contextual source from lucumr.pocoo.org; useful for practice signals or boundary conditions, not independent validation.

The Tower Keeps Rising | Armin Ronacher's Thoughts and Writings

Vibecoding and the possible collapse of a shared language.


The work targets tasks that exceed a single context window or prompt session. Shows dense next-observation prediction rewards under GRPO drive long-horizon LLM agents into a degenerate "dark room" absorbing state (prediction accuracy 1.0, task success 0), with a single-factor ablation localizing the cause to std normalization and auxiliary-loss channels as a working fix. A crisp negative result for anyone designing reward channels in agent training loops.

Use The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and What Actually Works to bound risk before recurring or unattended execution.

Research source arXiv:2607.21273; inspect its method and evaluation before treating results as production evidence.

[2607.21273] The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Works

Dense per-step supervision is the standard remedy for sparse-reward long-horizon LLM agents: reward the policy for predicting its next observation, which looks provably safe under potential-based shaping. Published prediction-reward and auxiliary-loss variants report both successes and instabilities; we supply the controlled account: 74 preregistered arms dissect one fixed prediction signal under GRPO across ALFWorld, WebShop, a synthetic POMDP, and Qwen3-1.7B/4B/8B, varying only the delivery mechanism. (1) Every run sustaining this difference-form reward under untouched std normalization (no filtering, dynamic-sampling, or decoupling mitigations) collapses: eleven runs across scales, coefficients, group sizes, and groupings (the floor-bound synthetic environment stalls instead); ALFWorld runs end in an absorbing state (prediction accuracy -> 1.0, success -> 0): the optimizer builds the "dark room". The algebra is one line: in all-fail groups z-scoring cancels the shaping coefficient; removing only std normalization restores baseline parity. (2) A signal's danger is set by its within-group variance trajectory, plus hackability as a second axis; it retrodicts every reward-channel collapse and survives preregistered prospective tests. (3) The same signal as a teacher-forced auxiliary loss is harmless on ALFWorld at 4B, but the gain is not the signal's: content-free placebos as a class match or beat gold at both matched seeds (s0: 78.8 vs 68.6; s42: 67.9 vs 57.9); the auxiliary update is the regularizer. (4) At 8B the recipe turns bistable: gold full-weight locks two of three seeds; every content-free or reduced-weight arm stays healthy. No ALFWorld or WebShop reward-channel variant measurably beats its matched-normalization baseline and no gold signal measurably outperforms its content-free placebo: the delivery channel, not the content, decides; which channel is safe is regime-dependent.


The work targets tasks that exceed a single context window or prompt session. Dex Horthy's essay from his AI Engineer World's Fair 2026 keynote arguing that lights-off software factories fail because RL training rewards passing tests with no penalty for eroding codebase maintainability, so long-horizon quality feedback cannot be trained on, proposing front-loaded human judgment gates (requirements review, architecture, vertical slices) as the missing layer above the harness.

Use Why Software Factories Fail (or: Harness Engineering Is Not Enough) to bound risk before recurring or unattended execution.

Inspectable GitHub source (2,299 stars; 166 forks; updated 2026-08-07); popularity is context, not proof of reliability.

advanced-context-engineering-for-coding-agents/wsff.md at main · humanlayer/advanced-context-engineering-for-coding-agents · GitHub

Contribute to humanlayer/advanced-context-engineering-for-coding-agents development by creating an account on GitHub.


Evaluation data is used as the feedback signal for improving loop behavior. Theory paper from Fourati, Schütze, Hüllermeier, and Gurevych challenging the assumption that humans stay in the loop only until AI capability catches up: it identifies three grounds for persistent human participation, technical complementarity, normative/developmental value, and objectives that emerge through the interaction itself, framing human-AI co-construction as a permanent feature rather than a stopgap. A conceptual counterweight for deciding which human gates in unattended loops are load-bearing versus transitional; no system or evaluation.

Use The Boundaries of Automation: A Theory of Persistent Human Participation to bound risk before recurring or unattended execution.

Research source arXiv:2607.21547; inspect its method and evaluation before treating results as production evidence.

[2607.21547] The Boundaries of Automation: A Theory of Persistent Human Participation

The rapid progress of AI has intensified the long-standing pursuit of automation: replacing human participation with algorithms wherever possible. Implicit in this pursuit is the assumption that humans remain in the loop only because current AI systems are not yet sufficiently capable. This paper challenges that assumption. Rather than asking how far automation can extend, we ask where its conceptual limits lie and argue that human participation may persist even with highly capable AI systems for three distinct reasons. Technical or complementarity grounds arise when humans contribute capabilities or perspectives unavailable to AI. Normative or developmental grounds arise when participation itself is valuable for human agency or learning. Most importantly, emergence grounds arise from target emergence: in some activities, the target is not fully specified in advance but instead emerges through the interaction itself. In these cases, human participation is not merely a means of improving execution but is constitutive of the target being produced. Human--AI co-construction, understood as the joint production of outcomes by humans and AI systems, is therefore not simply a temporary response to imperfect AI, but a persistent feature of activities whose objectives emerge through participation. This perspective has important implications for the limits of automation and for the design, evaluation, and ethics of future AI systems.

Fares Fourati; Hinrich Schütze; Eyke Hüllermeier; Iryna Gurevych


Reward Hacking in the Wild

Keeps adoption grounded in known failure modes, economics, and operational limits. Searchable corpus of 3,607 user-reported AI-agent misbehavior incidents collected from GitHub, Hacker News, LessWrong, and X, normalized and classified across fourteen misbehavior types with severity ratings. Despite the name, literal reward hacking is only about 6 percent of incidents, with overeagerness and other misalignment dominating, which itself says something about what unattended loops actually do wrong.

Use Reward Hacking in the Wild to bound risk before recurring or unattended execution.

Working implementation, framework, runtime, or repository; signal comes from usable code and ecosystem adoption.

Your AIs don't do what you want. This is really bad negligible: 1,468 (40.7%) minor: 1,373 (38.1%) significant: 618 (17.1%) severe: 121 (3.4%) unrated: 27 (0.7%)

Thousands of user-reported incidents of AI agents misbehaving, collected from public posts. Reports, not verified events.


Keeps adoption grounded in known failure modes, economics, and operational limits. Bandana Kaur gives a formal treatment of the epistemic limits of red-teaming, the practice now serving as the industry's primary safety gate for agent deployment.

Use What AI Red-Team Evaluations Can and Cannot Prove to bound risk before recurring or unattended execution.

Research source arXiv:2607.21735; inspect its method and evaluation before treating results as production evidence.

[2607.21735] What AI Red-Team Evaluations Can and Cannot Prove

Red-team evaluations of AI models support some claims and not others, and the boundary between the two is calculable rather than merely a matter of judgment. We define the evidential ceiling of an evaluation as the largest factor by which one result can move belief under a fixed testing budget, derive it in closed form for the benchmark null result, and use it to locate that boundary exactly. We find that above a calculable harm rate, a benchmark of modest size certifies a category to a stated evidentiary standard, and a clean sheet is then the stronger of the two possible observations, outweighing a single reproduced failure. Below that rate, no passive benchmark of feasible size provides the specified evidence of safety under the fixed scoring rule and approximately independent trial structure. The crossing between the two regimes has a closed form. The bound is not specific to benchmarks: written in terms of a procedure's hypothesis conditioned elicitation rates, it covers adaptive and automated red teaming as well, and shows that discrimination between the hypotheses rather than attack success is what determines evidential worth. Auditing eight evaluation suites against the boundary, we find that current benchmarks are adequate for high-frequency harm categories and several orders of magnitude short for rare, catastrophic ones. Safety benchmarks are not uninformative. They are informative about a specific and computable set of propositions, and the discipline they need is to state which.

21 pages, 4 figures, 5 tables. Code and data links provided in the manuscript. v2: corrected Figure 1(b); corrected required sample sizes in Table 4 and in Sections 4.2, 4.6 and 5.2, which had been rounded rather than taken to the ceiling; corrected the sample-size expression stated in Methods; minor corrections to Table 1 and the Figure 2 caption. No theorem, result or conclusion is affected


Keeps adoption grounded in known failure modes, economics, and operational limits. Shows coding-agent token cost varies starkly and consistently by programming language across five recent models on difficulty-controlled Python, Java, Rust, and OCaml problems. Explains why by re-executing every intermediate solution and abstracting each trajectory into test-outcome vectors: agents burn budget re-producing noncompiling solutions in unfamiliar languages and revising solutions that already pass. Trajectory-text analysis adds that agents plan in code comments and distrust the provided tests. Concrete cost-of-the-loop evidence for stack choices.

Use The Best Programming Language for Tokenmaxxing: An Investigation of Coding Agent Behavior Across Programming Languages to bound risk before recurring or unattended execution.

Research source arXiv:2607.22807; inspect its method and evaluation before treating results as production evidence.

[2607.22807] The Best Programming Language for Tokenmaxxing: An Investigation of Coding Agent Behavior Across Programming Languages

Although coding agents are now very effective in a variety of programming languages, this paper first shows that the cost (in tokens) can very significantly by programming language. We evaluate five recent models on programming problems in Python, Java, Rust, and OCaml. We carefully control for problem difficulty, and show that there can be stark variation in token consumption that is consistent across models. To understand why, we analyze both the structure and content of agent trajectories. First, we re-execute every intermediate solution and abstract each trajectory as a sequence of test-outcome vectors, then label the work between successive solutions. This reveals agents repeatedly producing noncompiling solutions in unfamiliar languages and revising solutions that already pass. Second, we analyze trajectory text, finding that agents plan solutions in code comments, distrust the provided tests in favor of inputs they invent, and sidestep unfamiliar target languages by prototyping in Python. Our results show that by-language token efficiency is a metric that should be considered when benchmarking and developing multilingual agents, and, for the tokenmaxxer, a guide to the most expensive language to work in.

Zixuan Wu; Carolyn Jane Anderson; Arjun Guha


Keeps adoption grounded in known failure modes, economics, and operational limits. Third-party LLM routers sit on the trusted path between a coding agent and its provider, able to inspect and modify every request and response, while nothing verifies that provider output matches the repository-level actions the agent ultimately executes -- so client-side permission mechanisms can silently stop working. Empirical study of router-side injection across four intervention levels of increasing subtlety. A supply-chain attack surface specific to high-autonomy agent loops that the community has largely ignored.

Use Where Is the Cost of Third-Party API Routers in Agentic Software Development? to bound risk before recurring or unattended execution.

Research source arXiv:2607.23624; inspect its method and evaluation before treating results as production evidence.

[2607.23624] Where Is the Cost of Third-Party API Routers in Agentic Software Development?

Third-party API routers have become a common layer that unifies access across increasingly diverse LLM providers. In coding-agent workflows, high-autonomy operation is widely adopted because it reduces interaction overhead. As a result, a third-party API router, which sits between the agent and the upstream provider, inevitably occupies the trusted path. It can inspect and modify every request and response, yet no mechanism verifies alignment between the provider's output and the repository-level actions ultimately executed by the agent. Consequently, client-side permission mechanisms may become ineffective in practice. Whether this control gap produces real, hard-to-detect effects on software development tasks remains empirically unmeasured. In this paper, we conduct an empirical study of router-side injection in coding agents, examining four intervention levels of increasing subtlety: Response Substitution (L1), Response Append (L2), LLM-Polished Injection (L3), and LLM-Polished with Distribution Alignment Injection (L4). Moreover, we develop SIDEL, a framework for trace recording, replay, injection, and defense evaluation, with a curated dataset of 400 samples. We evaluate four representative coding agents, and further evaluate whitelist-based execution control and LLM review. Router-side intervention substantially alters repository-level actions and remains difficult for existing client-side safeguards to detect. Without additional mitigations, all evaluated agents achieved a defense success rate of 0 percent across all injection levels. Client-side mitigations and reactive reviews improve resistance but do not fully restore end-to-end control, motivating provider-side output-integrity guarantees. Our code is available at https://github.com/Riyasushin/SIDEL.

Donghao Fu; Jingxin Li; Xue Jiang; Yihong Dong


Verification is promoted from a final check to a loop-control signal. Position paper arguing autonomous research systems are judged almost entirely on final outcome quality while search efficiency -- reaching that outcome on a small budget -- is an equally important and ignored dimension, and one that dominates as AR moves from cheap-verification domains like math and code into settings where evaluating a candidate means running a physical experiment. Proposes reporting the AUC of the Pareto frontier alongside outcome quality, and compares several families of AR systems under it.

Use Efficiency Matters in Autonomous Research to bound risk before recurring or unattended execution.

Research source arXiv:2607.24647; inspect its method and evaluation before treating results as production evidence.

[2607.24647] Efficiency Matters in Autonomous Research

AI-driven autonomous research (AR) systems are becoming increasingly effective across a broad range of tasks. Their performance, however, is still evaluated primarily by the quality of the final outcome. In this paper, we argue that the efficiency of the solution-search process is an equally important but often overlooked dimension of performance. A strong AR system should not only produce high-quality results, but also reach them with as small a budget as possible. Search efficiency will become increasingly important as AR expands from domains with inexpensive verification, such as mathematics and coding, to real-world scientific settings in which solution evaluation may require costly physical experiments. To capture this dimension, we propose evaluating AR systems using the area under the curve (AUC) of the Pareto frontier, alongside final outcome quality. We compare several families of search algorithms, including hill climbing, beam search, tree search, and evolutionary search, across twelve systems-optimization tasks. We find that no single search structure is consistently the most efficient. We also show that search efficiency and final outcome quality are distinct performance dimensions: a method that eventually achieves the best result may nevertheless improve slowly and consume substantially more evaluation budget before reaching that result. Because the most effective search policy is generally unknown in advance, we introduce an adaptive procedure called fluid search, which uses a portfolio bandit to dynamically allocate a fixed evaluation budget across a forest of search processes. Across the evaluated tasks, fluid search achieves the highest overall search efficiency, closely matching the performance of a per-task oracle that is given the best search structure for each task in advance.


Control flow is represented as an inspectable graph rather than an opaque prompt loop. Treats error propagation in multi-agent systems as an epidemic on the communication graph, with susceptible/exposed/infectious/corrected states and a derived early-invasion condition for heterogeneous topologies, then couples it to an analytic majority-vote benchmark where a clean-task reliability target imposes a minimum connectivity requirement.

Use Reliability-Contagion Feasibility in LLM Multi-Agent Networks to bound risk before recurring or unattended execution.

Research source arXiv:2607.21912; inspect its method and evaluation before treating results as production evidence.

[2607.21912] Reliability-Contagion Feasibility in LLM Multi-Agent Networks

Communication allows large language model agents to pool evidence, but it also creates paths along which an erroneous claim can spread. We formulate a correction-aware network model that tracks susceptible, exposed, infectious, and corrected agents and derive its early-invasion condition for heterogeneous communication networks. We then couple this propagation model to an analytic majority-vote benchmark in which a clean-task reliability target imposes a minimum connectivity requirement. Under fixed exposure per communication edge, reliability and error control impose opposing graph constraints. We characterize when their intersection is empty and when it contains an intermediate connectivity range, and identify regular graphs that attain the smallest invasion factor within the reliable graph class when such graphs exist. Under a fixed sender budget, the homogeneous first-order threshold is independent of network density, showing that the communication-budget convention determines whether added edges increase early propagation risk. Finite-network simulations on 21,000 trajectories illustrate these directional predictions. A controlled grok-4.3 experiment then evaluates three six-node topologies on 36 new closed-world tasks, with a balanced 12-task subset continued to full cascades. Mean first-generation offspring increased from 0.667 to 1.333 and 1.667 as degree increased from 2 to 4 and 5, while the adoption fraction among exposed neighbours remained 0.333. Mean non-seed erroneous adoption in the full-cascade subset was 0.200, 0.333, and 0.333. Together, these results provide a tractable basis for selecting connectivity under explicit reliability and propagation constraints.


State persistence is explicit enough for repeated runs and handoff. When LLM agents share a persistent resource, one agent's decision changes the conditions later agents face. Four same-family GPT, Gemini, or Grok agents act as electricity prosumers instructed to maximize operational continuity, with aggregate demand and protocol held fixed while the regeneration rate varies.

Use Draining the Energy Commons: Self-Defeating Over-Appropriation as a Coordination Failure in Agentic LLM Collectives to bound risk before recurring or unattended execution.

Research source arXiv:2607.22188; inspect its method and evaluation before treating results as production evidence.

[2607.22188] Draining the Energy Commons: Self-Defeating Over-Appropriation as a Coordination Failure in Agentic LLM Collectives

LLMs are increasingly deployed as agents that plan, use tools, and act over time. When they share persistent resources, such as compute pools or energy reserves, decisions by one agent affect the conditions faced by later agents. We study this coordination failure in a renewable energy commons. Four same-family GPT, Gemini, or Grok agents act in homogeneous self-play as electricity prosumers, instructed to maximize operational continuity. Holding aggregate residual demand and the decision protocol fixed, we vary the regeneration rate of a shared energy reserve from abundance to scarcity. All three families preserve the reserve when demand does not exceed peak renewable replacement, but over-appropriate it beyond that threshold (all nine exact scarcity contrasts survive Holm correction; largest adjusted p = 4.87e-5). The pattern is self-defeating: the same populations protect current service while undermining future service. At higher scarcity (rho = 1.2), early aggregate request pressure exceeds peak renewable replacement in every family and averages 1.21 times that level. Mean trajectories fall below the reserve level of maximum replenishment by rounds 5-7. Two offline benchmarks compare a social planner maximizing group-wide operational-service value with open access, where each prosumer maximizes its own value. At a discount factor of gamma = 0.95, both benchmarks sustain the reserve under the same dynamics. Realized depletion instead resembles outcomes under a more impatient open-access benchmark. The populations therefore behave like impatient optimizers at the level of the public trajectory. This system-level alignment failure would be missed by isolated-response evaluation.

Marcantonio Bracale Syrnicov; Federico Pierucci; Matteo Prandi; Marcello Galisai; Piercosma Bisconti; Francesco Giarrusso; Daniele Nardi


Keeps adoption grounded in known failure modes, economics, and operational limits. First large-scale empirical study of what happens after an agent posts a review comment: 54,791 comments from Copilot, Cursor, Codex, Devin, and Claude across 342 Python GitHub repositories, analyzed for resolution rates by agent and comment type, the effect of developer experience, and what makes a comment useful. Resolution varies sharply by agent (Copilot accounts for 72.9% of resolved comments) and core developers resolve the majority. Hard data on the human end of the automated review loop, where most claims are anecdotal.

Use "Go Home Copilot, You're Drunk": Understanding Developer Responses to Agent-Generated Code Review Comments to bound risk before recurring or unattended execution.

Research source arXiv:2607.21997; inspect its method and evaluation before treating results as production evidence.

[2607.21997] "Go Home Copilot, You're Drunk": Understanding Developer Responses to Agent-Generated Code Review Comments

Code review is a critical quality assurance practice in software engineering development, and AI coding agents are increasingly generating review comments on pull requests. However, little is known about how developers actually respond to such agent-generated feedback. In this paper, we present the first large-scale empirical study on the resolution of agent-generated code review comments. We analyze $54{,}791$ comments generated by five widely used coding agents (i.e., Copilot, Cursor, Codex, Devin, and Claude) across $342$ Python repositories on GitHub. We examine (1) resolution rates across agents and comment types, (2) the role of developer experience, and (3) characteristics that influence comment usefulness. Our results show that resolution rate varies considerably across agents, with Copilot accounting for the majority of resolved comments (72.9\%). Core developers resolve the majority of agent-generated feedback, particularly for \textit{design} and \textit{evolvability}-related comments, while peripheral developers are more involved in resolving \textit{functional defect} comments. Through open card sorting of 470 unresolved comment discussions, we identify \textit{ten} discussion patterns explaining why comments remain unresolved, with \textit{incorrect suggestions} and \textit{intentional design decisions} being the most prevalent. Finally, our analysis reveals that the presence of an inline \textit{code suggestion} is the strongest predictor of comment resolution, while lengthy and complex comments are less likely to be acted upon. Our findings provide insights for improving AI-generated code review feedback and its integration into development workflows.

Shamse Tasnim Cynthia; Ratnadira Widyasari; Banani Roy; Ting Zhang; David Lo


Evaluation data is used as the feedback signal for improving loop behavior. Argues benchmark scores are epistemic claims with expiry dates and that averaging heterogeneous signals inflates confidence past the weakest component, proposing formality tiers, scope declarations, and expirations, HELM rankings shift materially under weakest-link aggregation. Sharp critique of how loop teams read eval dashboards.

Use Position: Evaluation Scores Are Perishable Knowledge Claims to bound risk before recurring or unattended execution.

Research source arXiv:2607.26191; inspect its method and evaluation before treating results as production evidence.

[2607.26191] Position: Evaluation Scores Are Perishable Knowledge Claims

Evaluation methodologies for language models increasingly combine multiple signals, from automated metrics and LLM-as-judge ratings to human assessments and benchmark suite results. When these signals are aggregated via averaging, evaluation confidence can then substantially exceed the reliability of the weakest signal: a phenomenon we call trust inflation in evaluation. We argue that evaluation scores should be treated as epistemic claims with three properties: formality (human evaluation provides stronger evidence than an automated metric), scope (a benchmark result applies to the tested distribution, not universally), and validity windows (benchmark results expire as contamination accumulates and distributions shift). Several converging research traditions (chain-of-thought analysis, possibilistic logic, and algebraic theory) establish weakest-link aggregation as the conservative endpoint of a parameterized operator family controlled by a single pessimism parameter. Drawing on those traditions, and on concrete lessons from building an evaluation harness for agentic AI, we propose that evaluation results carry explicit metadata (formality tier, scope declaration, and expiration date) to make their epistemic status transparent. We illustrate the cost of mean aggregation on the public HELM leaderboard: across 54 frontier models on ten scenarios, the top-five models ranked by mean score and by weakest-link are completely disjoint.

Association for Computational Linguistics

Published in Proceedings of the Fifth Workshop on Generation, Evaluation and Metrics (GEM) 2026; the linked arXiv record remains available for open access.


Keeps adoption grounded in known failure modes, economics, and operational limits. Shows automated research loops judge ideas on a single implementation, that implementation variance dwarfs rerun variance, and that winner reversal rates exceed 25%, then offers an Idea Reliability Audit. Directly undermines single-run selection in self-improving research agents.

Use One Run Is Not an Idea: The Implementation Lottery in Automated Research to bound risk before recurring or unattended execution.

Research source arXiv:2607.26587; inspect its method and evaluation before treating results as production evidence.

[2607.26587] One Run Is Not an Idea: The Implementation Lottery in Automated Research

Automated research systems use experimental scores both to deliver artifacts and to decide which ideas to retain, transfer, and pursue. Yet one run scores one implementation of an idea. Crediting that realization-level score as evidence about the parent mechanism creates the \emph{implementation lottery}, in which an idea-level conclusion depends on which plausible implementation was sampled. The mismatch is structural whenever one run updates beliefs about a mechanism. We estimate its magnitude. The \emph{Idea Reliability Audit} measures \emph{idea reliability} by validating and freezing candidate cards, sampling fresh-session implementations, using outcome-blind fidelity labels, and rerunning saved artifacts. It reports idea ICC and leave-one-implementation-out (LOO) winner reversal. Prior work generally repeats the task; we repeat the idea. Across 312 assignments on 13 tabular tasks and two coding-agent setups, implementation variance was more than five and ten times same-artifact rerun variance, respectively, and the winner from one implementation draw differed from the winner under the other-two mean in 25.6\% and 43.6\% of decisions. Reversal survives card-level filtering under two outcome-blind review rules. An exploratory diagnostic on three materials-regression workflows with a deterministic evaluator also finds implementation variation dominating the decomposition. These findings distinguish idea reliability from best-of-$N$ artifact utility. Before a score guides idea-level branching, transfer, or research memory, evidence should cover multiple implementations.

Jingjie Ning; Shanshan Zhong; Xiaochuan Li; Ji Zeng; Chenyan Xiong


Evaluation data is used as the feedback signal for improving loop behavior. Frontier agents ran six days of largely independent engineering on open research questions from unpublished NeurIPS 2026 papers, and the original authors unambiguously rejected both outputs, the gap was research judgment and creativity, not execution. A rigorous shadow-evaluation counterweight to autonomous-research optimism.

Use Can AI agents conduct open-ended AI research? Early evidence from two case studies to bound risk before recurring or unattended execution.

Research source arXiv:2607.27191; inspect its method and evaluation before treating results as production evidence.

[2607.27191] Can AI agents conduct open-ended AI research? Early evidence from two case studies

Forecasts of explosive AI progress hinge on AI agents automating AI research. But evidence on whether agents can carry out open-ended AI research is thin. Current evaluations either test agents on narrow, verifiable tasks, which excludes open-ended research, or submit AI-generated papers to blind peer review, which is overstretched, stochastic, and suffers from poor review quality. We introduce a third way to measure progress towards AI R\&D automation. An agent takes on the central, open-ended research question of a high-quality unpublished paper, and the paper's original authors grade its output. We call these shadow evaluations. We ran shadow evaluations on two unpublished NeurIPS 2026 submissions, giving frontier agents six days and thousands of dollars of compute. The agents completed all of the engineering without human help, yet could not make substantial progress towards answering the research questions. As a result, both papers were unambiguously rejected by the authors. We identify five recurring failure modes: poor judgment about the bar for publishable research, uncreative responses to shortcomings in the research design, ineffective backtracking from dead ends, poor resource awareness, and instruction drift. A robustness check with a second model and scaffold reproduced these failures. We release the expert reviews, survey responses, agent repositories, and logs. Our results provide early evidence that today's agents can do the engineering of AI research, but struggle with critical parts of the research lifecycle.

Peter Kirgis; Sayash Kapoor; Andrew Schwartz; Stephan Rabanser; David Africa; Konstantinos Voudouris; Viet Nguyen; Toby Pilditch; Magda Dubois; Harry Coppock; Cozmin Ududec; Nitya Nadgir; Matilda Orona; Tilman Bayer; Derrick Chan-Sew; Yue Ling; Abhishek Shetty; Helen Toner; Gillian Hadfield; Seth Lazar; Steve Newman; Shoshannah Tekofsky; Rishi Bommasani; Arvind Narayanan


The work separates roles across agents, verifiers, or orchestration layers. On a 7B local model, two refinement iterations beat a five-agent architecture on math reasoning, with data formatting and task-specific tuning mattering more than topology. Practical argument against reaching for multi-agent structure before exhausting the simple loop.

Use Two Calls Beat Five Agents: Evaluating Multi-Agent Pipelines Against Self-Refinement for Local Language Models to bound risk before recurring or unattended execution.

Research source arXiv:2607.26922; inspect its method and evaluation before treating results as production evidence.

[2607.26922] Two Calls Beat Five Agents: Evaluating Multi-Agent Pipelines Against Self-Refinement for Local Language Models

Multi-agent LLM pipeline systems break down the task among multiple roles for better reasoning, but are benchmarked mainly with large-scale commercial models. In this study, we investigate Parishad, a structured multi-agent system involving five roles, by deploying it on Qwen2.5-7B-Instruct, a local model, on two datasets: GSM8K (500 questions) and HumanEval (164 questions), compared with prompting directly and two-call self-refinement. The multi-agent system drops GSM8K accuracy from 75.0\% to 45.0\% with JSON data format due to the error accumulation problem. With plaintext format, the accuracy is restored to 82.0\%. A two-call self-refinement strategy (V1) can achieve 86.2\% accuracy on GSM8K, with 7.4$\times$ lower token usage. However, the same V1 implementation on HumanEval---where direct accuracy is already 96.3\%---actively destroys performance (66.5\%). A task-aware gated redesign (V2) applied to HumanEval preserves accuracy at 95.1\%. Our results demonstrate that communication format and implementation details determine outcomes more than architectural complexity, and that simpler approaches match or outperform multi-agent pipelines for local 7B model deployment. All code and data are released.


Keeps adoption grounded in known failure modes, economics, and operational limits. Finds that resampling without showing the model its failed attempt beats self-repair, because conditioning on its own broken code anchors the model into reproducing near-identical flaws. A concrete case where the feedback in the feedback loop actively hurts.

Use Try Again, Don't Look Back: Blind Resampling Outperforms Self-Repair in Small Code Models to bound risk before recurring or unattended execution.

Research source arXiv:2607.26117; inspect its method and evaluation before treating results as production evidence.

[2607.26117] Try Again, Don't Look Back: Blind Resampling Outperforms Self-Repair in Small Code Models

Self-repair - returning a failed program to the model together with its test output and asking for a correction - is a standard component of code agents, and is almost always evaluated against a baseline that does not retry at all. We argue that this comparison confounds the value of the feedback with the value of the extra attempt. Using a placebo-controlled design on MBPP+ at three model scales (1.5B, 3B, 7B), we compare four matched-budget retry conditions: blind resampling, a content-free failure notice, genuine execution feedback, and feedback augmented with verbal self-reflection. Blind resampling is the strongest condition below 7B, and remains statistically tied with the best condition at 7B, while consuming 2.5-5.5x fewer tokens; conditioning on the model's own failed attempt costs 6.1 points at 1.5B (p=0.006), and the informational content of execution feedback adds nothing measurable over the placebo. We attribute this to anchoring: when shown its previous attempt, a model reproduces a near-identical program in 33-68% of retries, against 2-14% under blind resampling. Two further experiments delimit the effect. Retrieved solutions to other tasks change nothing (bounded to +/-3.5 points), which localizes the harm to self-conditioning rather than context length; and reflection, the only condition that measurably weakens the anchor, remains dominated on cost. Replication rules out two competing explanations: the penalty is unchanged at full precision, and it reproduces on an independent model family. Across six configurations spanning two families and two precisions, its magnitude is predicted by baseline quality alone (r=0.96) - the cost of anchoring is the cost of committing to a bad first attempt.

Code, pre-registrations and run traces: https://github.com/vermayuvraj/self-improving-agent