The work targets tasks that exceed a single context window or prompt session. Measures how agents drift from intent when specifications arrive incrementally across a long loop, and proposes a mitigation that recovers most of the loss.

Use When the Specification Emerges: Benchmarking Faithfulness Loss in Long-Horizon Coding Agents to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2603.17104; inspect its method and evaluation before treating results as production evidence.

[2603.17104] When the Specification Emerges: Benchmarking Faithfulness Loss in Long-Horizon Coding Agents

Current coding-agent benchmarks usually pro- vide the full task specification upfront. Real research coding often does not: the intended system is progressively disclosed through in- teraction, requiring the agent to track durable design commitments across a long session. We introduce a benchmark for this setting and study faithfulne Ss Loss U nder eM ergent s Pecification (SLUMP), defined as the reduc- tion in final implementation faithfulness un- der emergent specification relative to a single- shot specification control. The benchmark con- tains 20 recent ML papers (10 ICML 2025, 10 NeurIPS 2025), 371 atomic verifiable compo- nents, and interaction scripts of approximately 60 coding requests that progressively disclose the target design without revealing the paper itself. Final repositories are scored with a five-level component-faithfulness rubric and accompanied by an exposure audit to verify that scored components are recoverable from the visible interaction. Evaluated on Claude Code and Codex, the single-shot specification control achieves higher overall implementation fidelity on 16/20 and 14/20 papers, respectively. Structural integration degrades under emergent specification on both platforms, while seman- tic faithfulness loss is substantial on Claude Code and small on Codex. As a mitigation case study, we introduce ProjectGuard, an exter- nal project-state layer for specification tracking. On Claude Code, ProjectGuard recovers 90% of the faithfulness gap, increases fully faith- ful components from 118 to 181, and reduces severe failures from 72 to 49. These results identify specification tracking as a distinct eval- uation target for long-horizon coding agents.


Reflexion code

Connects Loop Engineering to prior agent-loop and feedback-loop research. Reference implementation and experiments for verbal reinforcement loops.

Use Reflexion code to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Inspectable GitHub source (3,223 stars; 314 forks; MIT license; updated 2026-08-06); popularity is context, not proof of reliability.

Understand the field and its boundaries.

GitHub - noahshinn/reflexion: [NeurIPS 2023] Reflexion: Language Agents with Verbal Reinforcement Learning · GitHub

[NeurIPS 2023] Reflexion: Language Agents with Verbal Reinforcement Learning - noahshinn/reflexion


Verification is promoted from a final check to a loop-control signal. Position paper that formalizes the loop specification (trigger, goal, verification step, stopping rule, memory) as a reusable artifact handed to an agent harness, with a taxonomy, a five-level verification ladder, and a hand-coded analysis of fifty real-world loops.

Use Stop Hand-Holding Your Coding Agent: Engineering the Loops that Replace Step-by-Step Prompting to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.00038; inspect its method and evaluation before treating results as production evidence.

[2607.00038] Stop Hand-Holding Your Coding Agent: Engineering the Loops that Replace Step-by-Step Prompting

In mid-2026 a slogan reorganized how practitioners talk about coding agents: stop prompting your agent, start designing the loop that prompts it. We take this claim seriously and give it a careful treatment. We call the object of the new practice the loop specification: a bounded, reusable artifact, made of a trigger, a goal, a verification step, a stopping rule and a memory, that a human hands to an agent harness (such as Claude Code or Codex) so the agent pursues a goal on its own, in place of step-by-step prompting. We distinguish this external loop specification from two things it is often confused with: an ordinary programming loop, and the internal perceive-act-observe cycle that the harness already provides as plumbing. We position loop engineering as a new layer in the progression from prompt to context to harness to loop, and we argue, against the stronger headlines, that it does not retire prompt engineering; loop and prompt are distinct tools with distinct uses. We offer four contributions: a definition and scope for the discipline; an anatomy and taxonomy of loop specifications organized around trigger, goal type, a five-level verification ladder, architecture, and named terminal states; a descriptive analysis of the Loop Library, a public corpus of fifty real loops that we code by hand; and a set of design principles and anti-patterns grounded in the scientific literature on self-correction, reward hacking and model-as-judge fragility. The corpus shows that practice has matured most where the discipline says it matters: seventy percent of loops verify in the autonomous zone of the ladder and seventy-four percent name their terminal states, while automated triggering and durable memory remain comparatively underdeveloped. We close with the limits the practice must respect, including the verification burden, comprehension debt and the risk of cognitive surrender.


Verification is promoted from a final check to a loop-control signal. Survey that decomposes the agent execution harness into six runtime responsibilities (observation, context, control, action, state, verification) and argues task performance emerges from the interaction of model, runtime, task structure, and evaluation rather than the model alone.

Use From Question Answering to Task Completion: A Survey on Agent System and Harness Design to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2606.20683; inspect its method and evaluation before treating results as production evidence.

[2606.20683] From Question Answering to Task Completion: A Survey on Agent System and Harness Design

LLM-based agents mark a shift from passive question answering to active task completion: they perceive environments, invoke tools, maintain state, and act over extended horizons. As agent systems have evolved from prompt engineering to workflows and context engineering, harness engineering, and agent-native training with co-evolution, a central question has become increasingly important: where does the bottleneck in agent performance reside, in the foundation model, in the execution harness, or in the coupling between them? This survey examines LLM-based agents through a model-harness lens. We first clarify the functional definition of agents and the implementation view of an LLM-based agent as a foundation model coupled with an execution harness. We then analyze the limits of model-centric scaling, trace four paradigms of agent engineering, and decompose the execution harness into six coupled runtime responsibilities: observation, context, control, action, state, and verification. Using this decomposition, we map task properties and domain pressures to harness configurations, review benchmark and evaluation practices, and synthesize model-harness evidence on how runtime design affects long-horizon task completion, efficiency, and reliability. Finally, we identify open challenges in value-aware evaluation, safety, harness generalization, and model-harness co-evolution. Rather than treating agents as models with auxiliary tools, this survey argues that agent quality -- including success, efficiency, safety, and generalization -- emerges from the interaction between model capability, runtime infrastructure, task structure, and evaluation design. A collection of papers discussed in this survey is provided in https://github.com/ggjy/Awesome-Agent-Engineering.

Jianyuan Guo; Zhiwei Hao; Chengcheng Wang; Cheng Fan; Tingzhang Luo; Hongguang Li; Ying Gao; Hefei Mei; Jiankun Peng; Rongjian Xu; Minjing Dong; Han Wu; Mengyu Zheng; Kai Han; Shiqi Wang; Chang Xu; Yunhe Wang


Durable execution and replay are treated as first-class loop infrastructure. Self-evolution loop where the agent rewrites its own source code, with each change anchored to a production failure and accepted only after deterministic replay verification with rollback, lifting a four-task mean grader score from 0.25 to 0.61 without human intervention.

Use MOSS: Self-Evolution through Source-Level Rewriting in Autonomous Agent Systems to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2605.22794; inspect its method and evaluation before treating results as production evidence.

[2605.22794] MOSS: Self-Evolution through Source-Level Rewriting in Autonomous Agent Systems

Autonomous agentic systems are largely static after deployment: they do not learn from user interactions, and recurring failures persist until the next human-driven update ships a fix. Self-evolving agents have emerged in response, but all confine evolution to text-mutable artifacts -- skill files, prompt configurations, memory schemas, workflow graphs -- and leave the agent harness untouched. Since routing, hook ordering, state invariants, and dispatch live in code rather than in any text artifact, an entire class of structural failure is physically unreachable from the text layer. We argue that source-level adaptation is a fundamentally more general medium: it is Turing-complete, a strict superset of every text-mutable scope, takes effect deterministically rather than through base-model compliance, and does not erode under long-context drift. We present MOSS, a system that performs self-rewriting at the source level on production agentic substrates. Each evolution is anchored to an automatically curated batch of production-failure evidence and proceeds through a deterministic multi-stage pipeline; code modification is delegated to a pluggable external coding-agent CLI while MOSS retains stage ordering and verdicts. Candidates are verified by replaying the batch against the candidate image in ephemeral trial workers, then promoted via user-consent-gated, in-place container swap with health-probe-gated rollback. On OpenClaw, MOSS lifts a four-task mean grader score from 0.25 to 0.61 in a single cycle without human intervention.

Qianshu Cai; Yonggang Zhang; Xianzhang Jia; Huajiang Zheng; Wei Xue; Jun Song; Xinmei Tian; Yike Guo

12 pages, 3 figures, 2 tables. Preprint. Code: https://github.com/hkgai-official/Moss


METR Time Horizon 1.1

Connects Loop Engineering to prior agent-loop and feedback-loop research. Update to METR's time-horizon methodology, expanding the task suite to 228 tasks (31 at 8+ hours), migrating to the open-source Inspect framework, and revising the post-2023 capability doubling time to roughly 131 days.

Use METR Time Horizon 1.1 to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Contextual source from metr.org; useful for practice signals or boundary conditions, not independent validation.

Time Horizon 1.1 - METR Substack twitter Bluesky

We’re releasing a new version of our time horizon estimates (TH1.1), using more tasks and a new eval infrastructure.


Connects Loop Engineering to prior agent-loop and feedback-loop research. Two-timescale recursive self-improvement where a fast loop rewrites task skills from execution traces while a slow loop evolves the meta-skill governing improvement itself, gaining up to 23.5 points on OfficeQA, SealQA, and ALFWorld.

Use MetaSkill-Evolve: Recursive Self-Improvement via Two-Timescale Meta-Skill Evolution to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.05297; inspect its method and evaluation before treating results as production evidence.

[2607.05297] MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution

Recent LLM agents tackle increasingly long-horizon, open-ended tasks, and external skills, reusable procedural knowledge supplied to the agent, further extend this capability. However, a fixed, hand-authored skill is rarely optimal, and cannot adapt to the diversity of tasks an agent encounters. Self-improving agents address this by rewriting their own skill files from execution traces, yielding meaningful gains on challenging benchmarks. Yet such self-evolution remains non-recursive: it improves only the task skill (what the agent does) while the improvement procedure (how it improves) is authored once and held fixed. We introduce MetaSkill-Evolve, a two-timescale framework that makes agentic skill improvement recursive: every branch carries both a task skill $s$ and a branch-local meta-skill $m=(\psi,\sigma,\alpha,\pi,\varepsilon)$ whose five components parameterise the Analyzer, Retriever, Allocator, Proposer, and Evolver agents of the improvement pipeline. Task skills evolve on a fast loop while the meta-skill evolves on a slower one under the same pipeline applied to itself, with no additional model or objective. With all five pipeline agents sharing a single frozen backbone, MetaSkill-Evolve outperforms no-skill, static-skill, and single-level evolution baselines on three agentic benchmarks (OfficeQA, SealQA, ALFWorld), improving held-out test accuracy over the raw backbone by +23.54, +16.09, and +1.92 points respectively.

Zefeng Wang; Minxi Yan; Jinhe Bi; Sikuan Yan; Volker Tresp; Yunpu Ma


Connects Loop Engineering to prior agent-loop and feedback-loop research. Formalizes agent skill self-evolution as zeroth-order optimization and distills it into a minimal pipeline of file-system trajectory exploration, consensus attribute mining, and independent validation gating, letting a smaller model surpass larger ones on LiveMath and SpreadsheetBench.

Use SkillOpt-Lite: Better and Faster Agent Self-Evolution via One Line of Vibe to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.03451; inspect its method and evaluation before treating results as production evidence.

[2607.03451] SkillOpt-Lite: Better and Faster Agent Self-evolution via One Line of Vibe

While skill optimization for autonomous agents has gained traction, existing methods rely on complex pipelines. This leaves a fundamental question unaddressed: What constitutes a minimal viable pipeline for skill optimization, where every component is justified by theory or empirical necessity? We formalize skill optimization via Zeroth-Order (ZO) optimization, mapping classical counterparts (central difference, trust regions) to recent literature. Noting that unlike blind numerical perturbations in classical ZO, skill trajectories serve as interpretable debugging feedback. Grounded in Claude Code philosophy and PAC learning, we establish three principles for convergence and generalization: file-system-based trajectory exploration, consensus attribute mining, and independent validation gating. Eliminating redundancies, we propose SkillOpt-Lite. It accelerates convergence and outperforms full SkillOpt: improving LiveMath by +8.8 points on GPT-5.5 and +25.4 points on GPT-5.4-nano, allowing the nano model to surpass standard GPT-5.4 optimized by SkillOpt. Finally, we integrate our framework into production coding agents like VSCode Copilot, enabling developers to evolve agent skills via one line of vibe. Because our framework treats all agent components simply as standard editable code, this minimal pipeline naturally generalizes to full harness optimization (HarnessOpt). On SpreadsheetBench, HarnessOpt enables GPT-5.4-nano to achieve 0.7758 accuracy, outperforming the larger GPT-5.5 running standard pipelines (0.7620). Code is available at https://github.com/EvolvingLMMs-Lab/SkillOpt-Lite.


Connects Loop Engineering to prior agent-loop and feedback-loop research. Survey of 1,250 arXiv papers from 2024-2026 organized along two axes, what a self-improvement loop improves and its degree of loop closure, separating bounded evaluable self-refinement from open-ended recursive self-improvement.

Use Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.07663; inspect its method and evaluation before treating results as production evidence.

[2607.07663] Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops

AI systems increasingly participate in their own improvement: revising their outputs, adapting their own harnesses during deployment, training on data they generate, and, increasingly, conducting AI research itself. This literature is described under a vocabulary ("self-refine," "self-reward," "self-play," "self-evolve") that conflates fundamentally different ambitions. We survey 1,250 arXiv papers (2024-2026) along two axes: what the system improves -- its behavior in deployment, its policy through training, its evaluator, or the research process itself -- and the degree of loop closure (human-in-the-loop to fully closed). The taxonomy separates bounded self-refinement -- convergent, evaluable, and already industrial practice -- from open-ended recursive self-improvement (RSI), which remains bounded by grounding requirements, collapse dynamics, and compute constraints on every measured axis. Its distinctive feature is a dedicated category for self-evaluation: every improvement loop is a claim that some signal can substitute for human judgment. We survey the evaluator design space -- judges, process reward models, verifiers, rubrics, meta-evaluation -- order the signals into a verification hierarchy from formal verifiers (strongest) to intrinsic self-assessment (weakest), and observe that demonstrated self-improvement strength tracks this hierarchy, that its failure modes (self-confirming loops, model collapse, diversity collapse) follow from its violations, and that the "research direction-setting" bottleneck keeping humans in the loop sits at the top of that hierarchy. We connect the technical literature to the theory of RSI limits and to the safety and governance questions raised by frontier-lab accounts of closing the loop, and identify governance-grade measurement of self-improvement as the field's most underpopulated niche.


Evaluation data is used as the feedback signal for improving loop behavior. EvoSOP has agents distill recurring execution trajectories into reusable standard operating procedures and iteratively optimize the toolset through a construction, merging, evaluation, and pruning lifecycle.

Use From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.07321; inspect its method and evaluation before treating results as production evidence.

[2607.07321] From Atomic Actions to Standard Operating Procedures: Iterative Tool Optimization for Self-Evolving LLM Agents

Tool utilization enables Large Language Model (LLM) agents to interact with the real world and resolve complex tasks. However, existing agent frameworks predominantly rely on static toolsets composed of granular atomic actions (e.g., basic file I/O or single-turn search), which forces agents to reinvent low-level logic for every recurring workflow, leading to increased reasoning overhead and failure rates. In this study, we propose that agents can achieve self-evolution by synthesizing these atomic actions into reusable Standard Operating Procedures (SOPs), which function as callable higher-order tools that encapsulate multi-step logic. We further introduce EvoSOP, a framework that empowers agents to extract SOPs from execution trajectories and iteratively optimize the toolset through a systematic lifecycle of construction, merging, evaluation, and pruning. Extensive experiments demonstrate that EvoSOP significantly boosts task success rates while substantially reducing the number of interaction rounds compared to baselines. Our analysis also reveals that iterative tool optimization fosters reliable and efficient tool-use patterns, providing a scalable pathway for the development of self-evolving agents.

Haipeng Ding; Yuexiang Xie; Zhewei Wei; Yaliang Li; Bolin Ding


TTHE: Test-Time Harness Evolution

Connects Loop Engineering to prior agent-loop and feedback-loop research. Adapts LLM agents at test time by evolving a population of candidate harnesses (the executable control program around the model) from execution traces, using a label-free agentic proposer and judge to sustain improvements on text-to-SQL and competitive programming while flagging execution-derived proxy reliability as the key open challenge.

Use TTHE: Test-Time Harness Evolution to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.08124; inspect its method and evaluation before treating results as production evidence.

[2607.08124] TTHE: Test-Time Harness Evolution

The behavior of an LLM agent is determined not only by the underlying model, but also by its harness: the executable program that constructs context, invokes tools, verifies intermediate results, and recovers from failures. Existing approaches optimize such harnesses before deployment, searching training or development data for a fixed agent workflow that is then frozen at test time. This limits adaptation when the test distribution, failure modes, or tool interactions differ from those seen during development. We ask whether the harness can instead be optimized during evaluation itself, using only the unlabeled execution traces the agent produces on the test inputs. We introduce Test-Time Harness Evolution (TTHE), which treats the executable harness as the state of test-time adaptation. During evaluation, TTHE maintains a population of candidate harnesses and refines them through an agentic proposer that reasons over their execution traces, without gold labels or task-specific supervision; a judge then commits an improved harness from execution-derived proxy signals, and the selected program persists to govern subsequent inputs. Crucially, TTHE does not update model weights, require gold labels, or train a separate adaptation model: solver, proposers, and judge are different roles and harnesses around the same frozen LLM, so all adaptation occurs through changes to the surrounding program. Across text-to-SQL, competitive programming, software engineering, data-science coding, and agentic tool-use tasks, TTHE improves fixed ReAct-style baseline harnesses, yielding persistent, inspectable improvements rather than a pre-searched workflow or per-query retries. These results recast test-time adaptation for LLM agents as evolution over executable control programs and identify execution-derived proxy reliability as a central challenge for robust unsupervised agent improvement.

Jun Nie; Yonggang Zhang; Jun Song; Qianshu Cai; Dahai Yu; Yike Guo; Xinmei Tian; Bo Han


Verification is promoted from a final check to a loop-control signal. Introduces DeepSearch-Evolve, where a deep search agent improves by self-distilling its own trajectories inside a deterministic 420K-task verifiable environment, progress verification, grounded reflection, and failure recovery replace teacher trajectories and sparse RL reward, lifting a 9B model to 31.2% BrowseComp and 61.5% GAIA.

Use DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.07820; inspect its method and evaluation before treating results as production evidence.

[2607.07820] DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environment

Training tool-use agents to improve from their own experience remains challenging, as supervised fine-tuning relies on fixed teacher-distilled trajectories, while sparse-reward reinforcement learning provides weak supervision for long-horizon interactions. We present DeepSearch-Evolve, a self-distillation framework for web agents built on DeepSearch-World, a deterministic and verifiable environment with reproducible search and page-reading tools. DeepSearch-World contains 420K multi-hop QA tasks constructed from entity-level random walks and supports key agentic cognitive behaviors useful for self-evolving, including progress verification, grounded reflection, and failure recovery. DeepSearch-Evolve iteratively performs trajectory generation, filtering, data mixing, and fine-tuning to train stronger agents. Without distillation from more capable models, DeepSearch-World-9B achieves competitive performance compared with open-source agents, reaching 31.2% on BrowseComp, 61.5% on GAIA, and 93.4% on HotpotQA, showing that verifiable environments enable scalable self-evolution for long-horizon web agents. We will release the environment, 420K training pool, validation set, model, and code to facilitate future research on self-improving deep search agents.

Xinyu Geng; Xuanhua He; Sixiang Chen; Yanjing Xiao; Fan Zhang; Shijue Huang; Haitao Mi; Zhenwen Liang; Tianqing Fang; Yi R. Fung


Connects Loop Engineering to prior agent-loop and feedback-loop research. Statistical analysis of 433 issues plus controlled multi-model experiments showing LLM repair agents succeed more when bug reports carry reproduction scripts, fix suggestions, and fault-localization cues, while longer natural-language reports correlate with lower success, directly informing how a loop's work-discovery step should specify tasks before dispatching agents.

Use What Makes a Good Bug Report for an AI Agent? to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.07593; inspect its method and evaluation before treating results as production evidence.

[2607.07593] What Makes a Good Bug Report for an AI Agent?

Automated program repair (APR) agents are transitioning from research benchmarks to developer workflows, yet they still begin with bug reports written for human developers. While decades of research have established what makes a good bug report for humans (e.g., steps to reproduce, stack traces), it remains unclear whether these features transfer to LLM-based agents. We study this question in two analyses. First, we use statistical modeling to examine associations between 27 bug-report features and repair success across 433 SWE-bench Verified issues attempted by 87 repair agents. We find that fix suggestions, reproduction scripts, repository source code, and localization info are associated with higher resolution likelihood, while longer reports are associated with lower odds. Second, we conduct controlled ablations across 2 models and 17 problem-statement mutations on SWE-bench Pro, varying the information available to an agent while holding the underlying task fixed. We remove or isolate selected bug-report content, delete fault-localization cues, and test structural changes that flatten lists or remove section headers. We find that both models depend on localization cues and expected behavior, and that structural changes alone can reduce solve rates, even without removing any content. The two models diverge in how they handle missing information: Qwen searches more widely and can exhaust its turn budget, while Gemma commits to a plausible interpretation early and patches on it. Our findings indicate that a good bug report for an agent overlaps with, but is not identical to, a good report for a human: agents benefit most from concrete, executable, and well-localized information, whereas some qualities long emphasized for human readers, such as natural language steps to reproduce and readable descriptions, contribute little or even correlate with lower success.

Lara Khatib; Noble Saji Mathews; Meiyappan Nagappan; Pengyu Nie; Thomas Zimmermann


State persistence is explicit enough for repeated runs and handoff. Names self-locking as a runtime failure mode of continuing agent loops, where accumulated state and history pull generation toward stale repetition (over 95% rolling action-repetition across an eight-model 40-day stress test), and proposes a multi-timescale loop that admits divergent material only through evidence-governed absorption, cutting macro-theme repetition from 61.8% to 36.3%.

Use AutoPersonas: A Multi-Timescale Loop Engine for Open-Ended Persona Evolution to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.08252; inspect its method and evaluation before treating results as production evidence.

[2607.08252] AutoPersonas: A Multi-Timescale Loop Engine for Open-Ended Persona Evolution

Long-term persona agents must remain identifiable while adapting to new events, relationships, evidence, and social conditions. We identify self-locking as a runtime failure mode in continuing persona-life loops: locally plausible events keep appearing while the generated life collapses toward familiar environments, weak relationships, suspended decisions, and stale life stages. We trace this failure to model-level convergence toward high-probability behavioral channels and system-level context gravity from State, memory, history, and environment summaries. We introduce AutoPersonas, a multi-timescale life-environment engine for bounded persona-level recursive self-evolution. It separates environment-side Occurrences, accumulated Observations, and persona State. Its OSO loop admits divergent future-facing material while requiring evidence-governed absorption before State or reachability changes. A three-year compressed simulation exposed environment watermark shells, occurrence-hardening gaps, slow-change accumulation failures, recursive indecision, and weak relationship persistence. An eight-model 40-day stress test generated 1,600 events and found mean rolling 5-day action-category repetition of 95.2%-97.6%, with all models crossing 90% by day 11. Semantic re-keeping found 79.0%-88.0% macro-theme repetition across all direct-loop runs. In a same-runtime 40-day A/B, context-slice masking plus per-sample divergence targeting reduced macro-theme repetition from 61.8% to 36.3% and roughly doubled cumulative theme count. A juvenile-goblin fictional-world run reproduced the anti-fixation regime without hard real-world intrusions. These results support a bounded claim: separating controlled divergence from evidence-governed absorption can reduce persona-environment self-locking while preserving identity continuity.

52 pages, 13 figures/tables, ancillary public-safe evaluation artifacts included


Agentic Data Environments

State persistence is explicit enough for repeated runs and handoff. Vision paper from the IEEE Data Engineering Bulletin reframing data systems as the active execution environment agents operate in, spanning files, APIs, applications, and system state, arguing the substrate under recurring agent loops should both amplify agent capability and enforce safety guarantees that bound the cost of failure.

Use Agentic Data Environments to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.07397; inspect its method and evaluation before treating results as production evidence.

Autonomous agents promise substantial gains in speed, scale, and labor efficiency, but their failures can impose abrupt and often irreversible costs. The central challenge for agentic automation is therefore to increase the benefits of automation while bounding the consequences of failure. While databases remain central to modern computing, agents operate over a broader data environment spanning files, APIs, applications, and system state. In this talk, I will outline early work on Agentic Data Environments -- the execution substrate in which agents operate -- that both amplify agent capabilities and enforce safety guarantees. This perspective reframes data systems from passive stores of state into active substrates for safe, reliable execution.

Elaine Ang; Chenxi Huang; Georgios Liargkovas; Jerry Liu; Jinhui Liu; Nikos Pagonas; Charlie Summers; Haonan Wang; Jiakai Xu; Tianle Zhou; Yusen Zhang; Zhou Yu; Zhuo Zhang; Tianyi Peng; Kostis Kaffes; Eugene Wu

Published in IEEE Data Engineering Bulletin 50(1); the linked arXiv record remains available for open access.


Connects Loop Engineering to prior agent-loop and feedback-loop research. Meta agent maps observed failure modes to harness adaptation strategies, letting small-model agents recover ~90% of frontier-LLM performance at ~4% of the cost, the harness itself becomes the optimization target.

Use Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.08938; inspect its method and evaluation before treating results as production evidence.

[2607.08938] Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation

Frontier LLM agents are automating many business tasks, but their high inference cost makes large-scale deployment unsustainable. Small language models (SLMs) offer a cheaper alternative, yet they typically fall short when swapped into a harness designed for a frontier LLM. We show that for many routine business tasks, SLM agents can match LLM performance at 90% lower cost, when paired with an adapted harness that can be automatically discovered by a meta agent. The key insight is that much of the task difficulty is shared across instances and can be lifted from the model into the harness via tailored instructions, tools, and orchestration loops. To study this systematically, we create a framework that maps agent failure modes to harness adaptation strategies, and build a harness optimizer that automatically discovers effective adaptations from failure trajectories. Across seven business-oriented agentic tasks and three SLM families, we found optimized harnesses significantly improve performance on 16 of 21 task-SLM pairs, with seven pairs closing the SLM-LLM performance gap and the best SLM agent recovering 89.7% of LLM performance at 4% of the cost. Our analysis further shows that adaptation works best for tasks with more repetitive workflows and for SLMs with sufficient base capabilities. Together, these results suggest that harness adaptation can expand the practical deployment range of SLM agents in routine business tasks.

Chenyang Yang; Xinran Zhao; Tongshuang Wu; Christian Kästner


Connects Loop Engineering to prior agent-loop and feedback-loop research. First large-scale empirical study of public agent-skill repositories and marketplaces, characterizing which software-engineering activities get packaged as reusable skills, their coverage across the development lifecycle, how they evolve, and how they are evaluated - an activity-centric map of the skills layer that agent loops compose (Cao, Cheung, et al., HKUST).

Use Inside the Skill Market: From Software Engineering Activities to Reusable Agent Skills to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.09065; inspect its method and evaluation before treating results as production evidence.

[2607.09065] Inside the Skill Market: From Software Engineering Activities to Reusable Agent Skills

Software engineering (abbrev. SE) has continuously evolved through increasingly powerful forms of reuse, from source code and libraries to components and services. Recent advances in AI agents have introduced a potentially new reusable artifact: skills. Emerging agent skill repositories and marketplaces enable developers to package, share, and reuse SE expertise as reusable skills. This trend raises a fundamental question: what SE activities are being encapsulated into reusable skills? Existing studies primarily focus on a broad range of skills acquisition, safety, or benchmarking, while lacking a systematic understanding of SE-specific skills and their coverage across the software development lifecycle. To address this gap, we conduct the first large-scale empirical study of SE skills in public repositories and marketplaces. We collect and analyze a large corpus of SE skills, examining the activities they encapsulate, lifecycle coverage, evolution characteristics, and evaluation mechanisms. Our findings reveal that SE activities are increasingly becoming reusable artifacts via skills and suggest promising research opportunities for skill recommendation and engineering-oriented structuring, as well as the need for mechanisms to encapsulate high-context SE activities into reusable skills. Overall, our study provides the first activity-centric characterization of SE skills and reveals how SE activities are increasingly being transformed into reusable skills. These findings offer new insights into skill reuse, ecosystem development, and the future of agent-centric SE.

Jialun Cao; Xinru Yan; Songqiang Chen; Yaojie Lu; Zhongxin Liu; Shing-Chi Cheung


Orchestration and control flow are made explicit and inspectable. Lilian Weng's deep-dive arguing the harness, the system surrounding a base model that orchestrates execution, matters as much as raw intelligence for recursive self-improvement, with a taxonomy of harness components and failure modes.

Use Harness Engineering for Self-Improvement to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Contextual source from lilianweng.github.io; useful for practice signals or boundary conditions, not independent validation.

Harness Engineering for Self-Improvement | Lil'Log

The concept of recursive self-improvement (RSI) dates back to I. J. Good (1965), where he defined an “ultraintelligent machine” as a system that can surpass humans in all intellectual activities and design better machines to improve itself. Yudkowsky (2008) used the phrase “recursive self-improvement” for a specific feedback loop: an AI uses its current intelligence to improve the cognitive machinery that produces its intelligence. This feedback loop in modern AI may indicate the model rewriting its own weights directly, or more broadly the model improves the training pipeline and the deployment system, which in turn enables a better successor model with improved performance across economically valuable tasks. The speed of research development in AI has been shown to drastically accelerated in frontier labs (Anthropic; OpenAI).


Connects Loop Engineering to prior agent-loop and feedback-loop research. Compiles safety-critical standard operating procedures into executable pseudo-code run by a program-guided stack machine that pages the active frame while the LLM does semantic execution, finding runtime guidance is capability-gated (it helps strong models and harms weak ones) across a six-model, seven-domain SOPBench study.

Use Compile, Then Page: Executable SOP Programs and a Capability-Gated Runtime to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.11346; inspect its method and evaluation before treating results as production evidence.

[2607.11346] Compile, Then Page: Executable SOP Programs and a Capability-Gated Runtime for Procedural LLM Agents

Enterprise agents must follow long-horizon, conditional, safety-critical standard operating procedures (SOPs). We compile machine-readable SOP constraints into executable pseudo-code and run them with a program-guided (PG) stack machine that pages the active frame while an LLM performs semantic execution. A three-arm SOPBench study across six models separates representation from runtime: compiled text never significantly hurts and gains up to 16.0 points where official prose underperforms. Runtime guidance is capability-gated. Two strong models independently show positive seven-domain PG contrasts (58:19 and 75:31 discordant pairs), whereas weak models are harmed. A full-program cursor ablation (active frame first, complete program retained) recovers much of the strong-model refusal gain; selective visibility adds a smaller improvement. Paired probe and audit measurements track this divide to spontaneous state discipline rather than reconstruction ability. On Bank the three primary arms rise from 70.4 to 86.4 to 92.8, with 100% refusal correctness. Practical guidance: compile first; enable active-frame paging only after a model-level discipline check.

Chenglin Yu; Li Yin; Qingxin Fan; Ying Yu; RunyangRay Zhong; Ming Li


Verification is promoted from a final check to a loop-control signal. Industrial self-evolving agentic OS that treats exploit capability as a mutable, versioned kernel: the agent observes its own failures, synthesizes new capabilities, proves them against a live target, and hot-loads them back, a self-improvement loop with in-loop verification.

Use Mako: A Self-Evolving Agentic Operating System for Autonomous Web Exploitation to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.11288; inspect its method and evaluation before treating results as production evidence.

[2607.11288] Mako: A Self-Evolving Agentic Operating System (SE-AOS) for Autonomous Web Exploitation

We introduce the Self-Evolving Agentic Operating System (SE-AOS): a new class of AI agent that treats exploit capability as a mutable, versioned kernel it extends at runtime, observing its own failures, synthesising new capabilities, proving them against a live target, and hot-loading them back into itself. Mako is the first SE-AOS instance for security research and the autonomous web exploitation engine developed within LaunchSafe. LaunchSafe builds autonomous security agents for continuous offensive testing and agent-driven security research; Mako is the core engine behind that platform. On the public XBOW validation-benchmarks, 104 containerised, CTF-style web applications spanning 26 vulnerability classes across three difficulty tiers, Mako achieves full-suite coverage: it drives every one of the 104 targets to emit a cryptographically fresh, per-build flag, under a verification regime that makes fabricated or memorised results impossible. Our central result is a law of autonomous exploitation: once a capability exists and is discoverable, difficulty collapses; capability, not reasoning, is what is scarce, together with an architecture and formalism that turn that law into a self-improving system. Mako further runs a gated self-evolution loop that proposes, sandboxes, and commits improvements to its own agents and rules when fitness does not regress. We deliberately withhold the operational results, payloads, exploit chains, and tool source, because a system that reduces full-spectrum web exploitation to a repeatable, machine-speed pipeline is dual-use research of concern. We publish the science; we withhold the weapon.

Praneeth Narisetty; Shiva Nagendra Babu Kore


Verification is promoted from a final check to a loop-control signal. Mixed-methods study of how practitioners actually design software-engineering agents, surfacing the recurring loop, harness, and verification decisions teams make and where their mental models diverge from benchmark assumptions.

Use How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Study to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.10856; inspect its method and evaluation before treating results as production evidence.

[2607.10856] How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Study

The rise of Software Engineering (SE) agents, i.e., LLM-based agents that can understand large codebases and carry out engineering tasks with limited human intervention, has been marked by rapid advances and adoption, but little is known about how developers build these systems in practice: existing studies mine repositories or examine deployment, but few investigate how SE agents are constructed. Through semi-structured interviews with 20 practitioners from 12 organizations and an online survey of 80 practitioners, this paper is the first to study how SE processes are changing in the development of SE agents and what challenges developers face. We find that as implementation becomes cheaper, bottlenecks shift rather than disappear: long-standing work in requirements, coordination, and deployment becomes more visible, while reviewing generated code and evaluating agent behavior become new and increasingly central forms of work. We characterize a seven-stage workflow and five process shifts, including a move toward evaluation-driven development, in which evaluation is increasingly defined early and steers iteration, and the emergence of specifications as first-class artifacts that teams test and version alongside code. We further identify six challenges that teams face, together with 12 corresponding practices they use or propose to address them, including unreliable evaluation signals, comprehension debt as code outpaces understanding, and behavioral changes introduced by provider-side model updates.

Yunbo Lyu; David Williams; Jieke Shi; Zhensu Sun; Chao Peng; Zhou Yang; Federica Sarro; David Lo


Connects Loop Engineering to prior agent-loop and feedback-loop research. Survey and taxonomy of how agent skill libraries are created, evaluated, retired, and reused over time, organizing the fast-growing self-evolving-skills literature into a lifecycle framework.

Use Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.10113; inspect its method and evaluation before treating results as production evidence.

[2607.10113] Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries

Large language model agents increasingly store reusable procedures outside the model. These reusable procedures are often called \emph{skills}: they may be code functions, natural-language instructions, SKILL.md packages, workflow graphs, or learned adapters that a future agent can retrieve and invoke. This taxonomy-driven survey asks how such skill libraries change over time. Across a $124$-paper $2023$--$2026$ audit set, we synthesize dynamic skill systems as \emph{lifecycle-managed, verified, evolving artifact stores}: agents collect evidence from interaction, propose skill updates, verify and admit candidates, organize them for retrieval and composition, repair or prune stale entries, and govern sharing through provenance and rollback. We organize the literature around three survey tools. First, a $\text{six}$-sense taxonomy distinguishes the structurally different artifacts called ``skills'' in current papers. Second, an $\text{eight}$-stage lifecycle architecture identifies the recurring design decisions behind evidence acquisition, proposal, verification/admission, storage, retrieval/composition, maintenance, distillation/portability, and governance. Third, a lightweight skill-record schema and $\text{ten}$-operator vocabulary provide common terms for comparing library updates without elevating them into a separate method contribution. Using this structure, we synthesize evidence-graded patterns with explicit caveats: admission and repair are repeatedly important, verifier quality materially affects skill-aware RL, flat retrieval can degrade as libraries grow, and current benchmarks still under-report library trajectories, usage--utility gaps, and safety surfaces. We close with concrete reporting standards and open problems for evaluating dynamic skills as changing libraries rather than static prompt or tool collections.

Accepted at Transactions on Machine Learning Research (TMLR); the linked arXiv record is the available paper version.

Current arXiv acceptance note and OpenReview record


Connects Loop Engineering to prior agent-loop and feedback-loop research. Co-evolves a task agent's harness and model weights through a meta-agent, target agent, and feedback agent that evaluate outcomes and carry improvements across generations.

Use SIA: Self Improving AI with Harness & Weight Updates to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2605.27276; inspect its method and evaluation before treating results as production evidence.

[2605.27276] SIA: Self Improving AI with Harness & Weight Updates

Humans are the bottleneck in building and improving AI. Both the models and the agents that wrap them are written, tuned, and corrected by people. The long-horizon goal of an AI that can figure out how to improve itself remains open. Two largely disjoint research lines attack this bottleneck. The harness-update school has a meta-agent rewrite the scaffold of a task-specific agent (its tools, prompts, retry logic, and search procedure) while the model weights are held fixed. The test-time training school uses hand-written RL pipelines to update the model's own weights on task feedback while the harness is held fixed. These two silos operate in isolation. We propose SIA, a self-improving loop in which a language-model agent (the Feedback-Agent) updates both the harness and the weights of a task-specific agent. We evaluate across three contrasting domains: Chinese legal charge classification, low-level GPU kernel optimisation, and single-cell RNA denoising. Combining both levers outperforms scaffold iteration alone on all three benchmarks. SIA-W+H achieves 25.1% over prior SOTA on LawBench, 12.4% faster GPU kernels than prior SOTA (1,017 vs 1,161 {\mu}s), and 20.4% over prior SOTA on denoising. Harness updates make the model agentic, shaping how it searches and acts, while weight updates build the domain intuition that no prompt or scaffold can instil.

Prannay Hebbar; Yogendra Manawat; Samuel Verboomen; Alesia Ivanova; Selvam Palanimalai; Kunal Bhatia; Vignesh Baskaran


Persistent memory is treated as an external runtime artifact. Survey from the Zhuge/Schmidhuber group that frames a modern agent as a foundation model coupled to an operational scaffold (prompts, memory, tools, control logic) and organizes self-improvement research by which scaffold component gets updated and what signal drives the update.

Use Self-Improvements in Modern Agentic Systems: A Survey to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.13104; inspect its method and evaluation before treating results as production evidence.

[2607.13104] Self-Improvements in Modern Agentic Systems: A Survey

Self-improving autonomous agents are moving from research prototypes to deployed systems. The primary goal is controllable evolution, or adaptation, from experience with minimal or even no human input. This survey frames modern self-improving agents as adaptive systems that convert experience into accumulated capability gains. We offer a system-level framework that represents a modern agent as a configuration coupling a foundation model with an operational scaffold of prompts, memory, tools, and control logic. Within this framework, self-improvement is formalized as a self-induced update operator that obtains and commits updates to model parameters or scaffold components. We organize prior work by update target and by the signals that drive change, then review applications and discuss evaluation, before closing with open problems and future directions. For convenience, we track technical updates on https://github.com/selfimproving-agent/awesome-Self-Improving-Agents.

Zhe Ren; Yimeng Chen; Dandan Guo; Guowei Rong; Tonghui Li; R. B. Xiong; Qingfeng Lan; Wenyi Wang; Li Nanbo; Yibo Yang; Mingchen Zhuge; Jürgen Schmidhuber

97 pages, 12 figures. Project page: https://selfimproving-agent.github.io/ Repository: https://github.com/selfimproving-agent/awesome-Self-Improving-Agents


Connects Loop Engineering to prior agent-loop and feedback-loop research. Evolves the agent harness (prompts, injected knowledge, runtime config) with model weights frozen: an LLM diagnoses failures and proposes patches while deterministic code owns all sampling and significance testing, and accepted patches populate a pathology-keyed quality-diversity archive, with sealed-test gains of 9-15.5 points across seven domains.

Use Self-Evolving Agent Harnesses via Gated Semantic Quality-Diversity to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.13683; inspect its method and evaluation before treating results as production evidence.

[2607.13683] HarnessBank: Semantic Gene-Bank Search with Gated Verification for Agent-Harness Self-Evolution

Large Language Models (LLMs) have enabled capable agents across diverse applications. Beyond the foundation model, the performance of an agent is governed by the surrounding agent harness, including prompts, tools, control loops, etc. Automatically evolving this harness offers a promising pathway to agent improvement, yet existing approaches typically rely on greedy candidate selection and noisy self-generated feedback, rendering their gains susceptible to search collapse, task-specific overfitting, and poor verifiability. To tackle these challenges, we introduce HarnessBank, a trustworthy agent-harness self-evolution framework that pairs a task agent with a separate evolver agent for iterative failure diagnosis, harness generation, and evolution verification. HarnessBank maintains a Harness Gene Bank composed of high-performing harnesses of different semantic coordinates. Those harnesses are reinvented, recombined, screened, and selected during the self-evolution procedure. Moreover, we propose a Gated Harness Screening mechanism to efficiently filter high-quality harnesses and reduce the cost of evaluating numerous offspring harnesses. Across seven agent benchmarks, HarnessBank produces consistent performance improvements from 5.1% to 15.4%. Cross-model experiments further verify that the improvements come from the model-specific self-evolving process, instead of a universally optimal harness. Our code will be publicly available upon acceptance.

Xiaotian Luo; Dizhan Xue; Fengxingyu Wang; Chuanrui Hu; Yafeng Deng


Checkpointed state makes long-running agent work recoverable across failures. Treats long-running autonomous research as a continuous observable pipeline: exploration recorded as portable checkpointed artifacts with code, outputs, and evidence links, plus repair loops and human-in-the-loop quality gates, so failed branches stay inspectable. Single-author work, content is on-target for long-horizon loop infrastructure but adoption is unproven.

Use XScientist: A Git-Like Research Protocol for Long-Running Autonomous Scientific Discovery to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.12301; inspect its method and evaluation before treating results as production evidence.

[2607.12301] XScientist: A Git-Like Research Protocol for Long-Running Autonomous Scientific Discovery

Autonomous research systems are often evaluated as one-shot paper generators: given a topic, they produce a manuscript and a small set of experiment logs. This framing hides the operational problem that makes such systems difficult to trust: research is long-running, branching, failure-prone, and dependent on auditable handoffs between agents and humans. XScientist is a git-like research protocol and operating system for this setting. It orchestrates idea generation, experiment execution, manuscript drafting, self-review, repair, quality gating, daemon scheduling, and reproducibility artifacts as one continuously observable pipeline. The central design choice is to treat each run as a portable research artifact rather than only as a PDF. XScientist exports an Agent-Native Research Artifact (ARA), a protocol that records an exploration DAG, per-node code and outputs, claim-to-evidence anchors, content hashes, provenance, and re-execution hooks. This makes each generated paper inspectable as a science exploration tree: failed branches, repaired experiments, ablations, and manuscript claims remain connected to the nodes that produced them. The system also includes deterministic integrity forensics, sample gates, truth contracts, reviewer-oriented repair loops, and long-running daemon controls. This paper describes the current XScientist architecture, the ARA protocol surface, and the practical safeguards needed to move autonomous science from single-run demos toward reproducible, reviewable, and forkable research infrastructure. The implementation and manuscript source are maintained in the public GitHub repository at https://github.com/smileformylove/XScientist.


Knowledge-Centric Self-Improvement

State persistence is explicit enough for repeated runs and handoff. Proposes a knowledge-centric alternative to agent-centric self-improvement: agents stay generic and disposable while the persistent, improving object is a curated shared knowledge base that agents write evidence-grounded insights into and then distill, reporting higher solve rates at lower cost with knowledge that transfers across tasks and model families.

Use Knowledge-Centric Self-Improvement to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.19592; inspect its method and evaluation before treating results as production evidence.

Self-improving AI systems typically treat the agent as the object that improves, by optimizing prompts, workflows, harnesses, or even the agent's own code. This agent-centric view can make improvements expensive to maintain and difficult to transfer, because gains become tied to a particular agent design, task distribution, or adaptation run. We study a complementary paradigm: knowledge-centric self-improvement, in which agents remain generic and disposable while the persistent object is a curated knowledge base that agents can leverage for future tasks. We conduct controlled case studies to operationalize this idea via a simple protocol. Agents attempt one task, then contribute evidence-grounded insights to a shared knowledge base via task-level and cross-task forums, followed by knowledge distillation. Because self-improvement is contained in the knowledge rather than the agent, improvement can be more inspectable, transferable, and portable. Across abstract reasoning, coding, and terminal benchmarks, this protocol improves solve rates while reducing dollar cost relative to agent-centric baselines. The resulting distilled knowledge also transfers to held-out tasks and across LLM families, indicating that the improvement is not merely an LLM- or run-specific behavior. These results support a new view of self-improving agentic systems: progress can be driven primarily by the curated persistent knowledge. Code is available at https://github.com/recursive-knowledge/KSI.

Xuefei Julie Wang; Lauren Hyoseo Yoon; Chengrui Qu; Amanda Zichang Wang; Atharva Sehgal; Eric Mazumdar; Yisong Yue


Orchestration and control flow are made explicit and inspectable. Open-source framework for end-to-end SFT/RL training of agents that run inside real inference harnesses (Claude Code, OpenClaw) via a lightweight proxy that serves the harness's model calls while recording them as training data, with Kubernetes-orchestrated distributed rollouts. Directly attacks the gap between harness engineering and open training stacks that cannot express stateful multi-process harness inference.

Use OpenForgeRL: Train Harness-native Agents in Any Environment to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.21557; inspect its method and evaluation before treating results as production evidence.

[2607.21557] OpenForgeRL: Train Harness-native Agents in Any Environment

Modern AI agents rely on elaborate inference harnesses such as Claude Code, Codex, and OpenClaw to drive multi-turn reasoning, tool use, and access to external systems. While powerful, these complex harnesses also make agents hard to train end-to-end with open infrastructure, whose SFT/RL stacks cannot natively express stateful, multi-process harness inference. To address this, we present OpenForgeRL, an open-source framework for training harness-based agents end-to-end in diverse environments. OpenForgeRL achieves this with a lightweight proxy that serves the harness's model calls while recording them as training data for a standard RL codebase (e.g., veRL), and a Kubernetes orchestrator that runs each rollout in its own remote container, together enabling training on any harness in any environment at scale. By decoupling training and inference, OpenForgeRL allows researchers to easily train, study, and improve agents directly in the real harnesses and environments they are deployed with. We validate our framework across diverse, complex harnesses and environments, spanning tool/claw-based agents and multimodal GUI browser- and computer-use agents. Using only hundreds to a few thousand tasks, OpenForgeClaw reaches 31.7 pass^3 and 55.9 pass@3 on ClawEval and 33.7 on QwenClawBench. OpenForgeGUI reaches 37.7 on OSWorld-Verified, 63.0 on Online-Mind2Web, and 72.3 on WebVoyager. Both outperform open baselines of similar size on nearly all benchmarks, and in the GUI setting match or surpass models several times larger. Beyond benchmarks, we analyze how harness choice (e.g., ZeroClaw, OpenClaw, Codex) and RL shape agent behavior. We find that some harnesses are substantially harder to learn than others, and that RL improves agentic reliability, such as self-verification, tool coverage, and completing multi-step plans, though critical abilities such as error recovery remain weak.

Xiao Yu; Baolin Peng; Ruize Xu; Hao Zou; Qianhui Wu; Hao Cheng; Wenlin Yao; Nikhil Singh; Zhou Yu; Jianfeng Gao

updated the paper header to show ICLR2027 instead of ICLR2026 (already past)


Verification is promoted from a final check to a loop-control signal. Family of recursively self-improving deep-research agents built on the discovery-verification asymmetry: an inner research loop gathers evidence while an outer loop audits the answer constraint-wise, flags unresolved claims, and launches targeted follow-up research, with autonomously learned compression of interaction history. Self-improvement plus verification-in-the-loop, both core list themes.

Use AREX: Towards a Recursively Self-Improving Agent for Deep Research to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.21461; inspect its method and evaluation before treating results as production evidence.

[2607.21461] AREX: Towards a Recursively Self-Improving Agent for Deep Research

Deep research requires agents to find answers that jointly satisfy multiple constraints. Discovering such answers is costly, whereas verifying a candidate can often be decomposed into tractable constraint-wise checks. This discovery--verification asymmetry suggests that a research agent should do more than simply search longer: it should recursively improve its current answer by verifying intermediate results and using the partially verified state to guide subsequent refinement. We introduce AREX, a family of Recursively Self-Improving (RSI) deep research agents. AREX alternates between an inner research loop that gathers evidence and constructs a provisional answer, and an outer self-improvement loop that audits the answer constraint-wise, identifies unresolved claims, and launches targeted follow-up research. To sustain RSI over long horizons, AREX learns an autonomous context-update tool that compresses growing interaction history into a compact improvement state preserving verified evidence and unresolved constraints, without relying on an external model. We train AREX on verified synthetic tasks and high-quality trajectories through agentic mid-training and long-horizon reinforcement learning. To mitigate sparse final rewards during long horizon learning, we emphasize key steps where decisive evidence is acquired or erroneous research directions are corrected. We instantiate a dense 4B model and a 122B-A10B Mixture-of-Experts model. Across BrowseComp, WideSearch, DeepSearchQA, Humanity's Last Exam (HLE), and other reasoning and tool-use benchmarks, AREX substantially outperforms comparable-scale baselines and remains competitive with models using substantially more activated parameters.

Shuqi Lu; Chaofan Li; Kun Luo; Zhang Zhang; Hui Wang; Hongwang Xiao; Lei Xiong; Jiahao Wang; Sen Wang; Xiyan Jiang; Wanli Li; Yuyang Hu; Hongjin Qian; Bingyu Yan; Jianlyu Chen; Ziyi Xia; Yingxia Shao; Kang Liu; Zhicheng Dou; Di He; Chaozhuo Li; Qiwei Ye; Zhongyuan Wang; Zheng Liu


Connects Loop Engineering to prior agent-loop and feedback-loop research. Optimizes Agent Skills (reusable procedural knowledge for frozen-model agents) by jointly resolving where a workflow failed, which mechanism caused it, and which third-party Skill knowledge to reuse locally: node-mechanism attribution pinpoints the failed node and smallest valid edit target, bounded patches apply the repair, and provenance- and scope-aware selection imports external knowledge, reaching 90.3 hard accuracy on SpreadsheetBench with skills that transfer to WikiTableQuestions.

Use Workflow-Localized Mechanism Learning: Attribution-Guided Repair and Knowledge Reuse for Structured Agent Skills to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.20999; inspect its method and evaluation before treating results as production evidence.

[2607.20999] Workflow-Localized Mechanism Learning: Attribution-Guided Repair and Knowledge Reuse for Structured Agent Skills

Agent Skills package reusable procedural knowledge as external artifacts for frozen language-model agents, yet existing optimizers do not jointly resolve where a failure occurs in a workflow, which mechanism caused it, and how relevant knowledge from third-party Skills should be reused locally. We introduce Workflow-Localized Mechanism Learning (WML). Its Node--Mechanism Attribution identifies the failed workflow node, implicated mechanisms, and smallest valid edit target, routing single-mechanism defects to L3 resources and relational defects across mechanisms to L2 composition protocols. A six-module Workflow-Guided Skill Optimization (WGSO) loop then selects provenance- and scope-aware third-party knowledge, applies bounded patches, evaluates candidates, and stores verified outcomes in optimizer-side memory. On SpreadsheetBench, WML reaches 90.33 +/- 1.53 and 74.67 +/- 3.51 Hard Accuracy with DeepSeek and Qwen3.6-Flash, respectively; without additional optimization, the learned Skills transfer to WikiTableQuestions with 84.00 +/- 2.00 and 83.00 +/- 2.00 Denotation Accuracy. On Compiler-Supported50, WML attains both the highest hard-PASS rate and the lowest cost per successful task; compiled execution sharply reduces tokens and calls relative to a direct SkillAgent while retaining most of its successful tasks. Code and artifacts are available at https://github.com/xiaolin9595/workflow-localized-mechanism-learning.

Zibin Lin; Shengli Zhang; Taotao Wang; Yihan Xia; Deen Ma; Guofu Liao


Context is managed as durable loop state rather than a single prompt payload. Uses experience distillation to internalize an agent's own interaction histories into model weights, so in-context experience gains persist after the experience leaves the context window, matching RL-style improvement with far fewer environment interactions.

Use Sample-Efficient Learning from Agent Experience to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.21051; inspect its method and evaluation before treating results as production evidence.

[2607.21051] Sample-Efficient Learning from Agent Experience

Real-world agent learning is often constrained by costly environment interactions, such as running time-consuming experiments or obtaining human feedback. In-context learning offers a highly sample-efficient way for agents to learn from their own interaction histories, but its gains disappear once that experience is removed from the context. Separately, context distillation provides a mechanism for internalizing contextual information into model weights. However, applying it to agents' interaction histories without sacrificing environment sample efficiency remains underexplored. We term this problem Experience Distillation and develop an implementation that requires no further environment interaction beyond the collected experience. Experiments on 749 curated software-engineering tasks and six text-adventure games show that it retains at least 64.8\% of the gains from in-context learning across both domains, whereas direct supervised fine-tuning on the collected experience recovers only 3.8\%. Compared with classical reinforcement-learning baselines, in-context learning from trial-and-error experience followed by Experience Distillation matches their performance with at least \(9.6\times\) fewer environment samples.

Chenhui Gou; Haoqin Tu; Yunhao Fang; Jianfei Cai; Hamid Rezatofighi


Evaluation data is used as the feedback signal for improving loop behavior. Policy-centric training paradigm that reframes skills as a dynamic training-time scaffold for long-horizon agent RL: rollout groups from the latest policy become evidence cards, task-specific evaluation adjusts the context for subsequent rollouts, and guidance is pruned as the policy strengthens before the scaffold is discarded at deployment, improving over strong baselines by up to 18.6% on ALFWorld and WebShop while using 32.1% fewer prompt tokens on search-augmented QA.

Use PATS: Policy-Aware Training Scaffolding for Agentic Reinforcement Learning to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.21419; inspect its method and evaluation before treating results as production evidence.

[2607.21419] PATS: Policy-Aware Training Scaffolding for Agentic Reinforcement Learning

In long-horizon LLM agent reinforcement learning, weak policies often repeat similar failures, producing uninformative rollout trajectories and limiting effective policy optimization. Existing skill-centric methods improve exploration by optimizing, filtering, or internalizing reusable skills. However, they remain centered on the skills themselves rather than being designed as adaptive training-time support for the evolving policy. To address this, we propose a policy-centric training paradigm that reframes skills as a dynamic training scaffold. Our framework, PATS, converts rollout groups from the latest policy into evidence cards and uses task-specific evaluation to adjust the context used in subsequent rollouts. Concrete guidance helps weak policies to complete challenging tasks. As policy improves, redundant context is revised or removed to reduce reliance on explicit guidance while preserving useful rollout variation. The policy is optimized with environmental rewards using standard RLVR, and the training scaffold is discarded at deployment. Across ALFWorld, WebShop, and seven search-augmented QA benchmarks, PATS achieves performance competitive with SOTA baselines while using 25%-50% fewer tokens.

Yipeng Shi; Zhipeng Ma; Yue Wang; Qitai Tan; Yang Li; Peng Chen; Zhengzhou Zhu


Connects Loop Engineering to prior agent-loop and feedback-loop research. Applies TextGrad-style natural-language policy learning to agent failure trajectories and separates two abilities usually conflated: following a useful text policy versus learning one from experience. Finds a clear gap, human-written policies lift frozen 7B agents by about 5 points, while policies auto-learned from the agent's own failure traces fail to consistently beat baseline prompting even with richer traces, counterfactual reasoning, or iterative search, a cautionary result for anyone building failure-to-lesson loops.

Use From Agent Failures to Text Policies: What Works and What Breaks to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.20668; inspect its method and evaluation before treating results as production evidence.

[2607.20668] From Agent Failures to Text Policies: What Works and What Breaks

TextGrad improves language-model systems by revising text from feedback. Its core thesis is that natural-language feedback can act as a gradient for optimizing text components without changing model weights. Applying it to agents is harder because feedback arrives only after a sequence of actions, making it difficult to identify which decision caused failure. We study this problem by separating the ability to follow a useful policy from the ability to learn that policy from experience. Our main finding is a clear gap between these two abilities. Human-written policies improve two frozen 7B agents on TextWorldExpress by 5.0 success points, showing that useful policy text exists. However, policies generated from agent trajectories do not reliably outperform fixed prompting, even with richer traces, counterfactual evidence, or iterative GEPA search. The main challenge for agent-level TextGrad is therefore not executing textual policy updates, but reliably generating and selecting them from experience.


The work turns loop quality into a measurable task or score. Nearly 6,000 runs across two office-automation benchmarks and three model harness stacks measuring what average success rate hides: skills also cause regressions (tasks solved without the skill, failed with it). Finding that should change how people ship Skills -- the best-performing skills win primarily by regressing less, not by gaining more. Names three mechanisms including 'skill description osmosis' (a skill changes behavior merely by sitting in context, never invoked) and grounding displacement. Essential reading for anyone adding a skills directory to an agent harness.

Use The Regression Tax: Decomposing Why Skills Help and Hurt LLM Agents to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.22520; inspect its method and evaluation before treating results as production evidence.

[2607.22520] The Regression Tax: Decomposing Why Skills Help and Hurt LLM Agents

Adding procedural skills to an LLM agent is typically evaluated by average improvement in task success. However, this metric hides an important cost: skills can also make agents worse. We measure both sides by comparing agents with and without skills across nearly 6,000 runs spanning two office automation benchmarks and three model harness stacks. This allows us to distinguish two outcomes. A regression is a task solved without skills but failed after skills are added. A residual failure is a task that fails both with and without skills. We find that regressions are substantial enough that the best performing skills outperform others primarily by regressing less, not by gaining more. We identify three causes of regression: (i) skill description osmosis, a skill changes an agent's behavior simply by being present in context, even when it is never invoked; (ii) grounding displacement, a skill's prescribed procedure overrides how the agent interprets its inputs; and (iii) verification displacement, where the procedure suppresses checks the agent would otherwise perform on its outputs. Analysing persistent failures reveals the same underlying pattern. Existing skills overemphasize procedural guidance the stage least often responsible for failure while under supporting grounding and verification, the dominant sources of remaining errors. After correcting evaluation artifacts and studying traces, we find many regressions and persistent failures recoverable through better grounding and verification. Procedural skills should be evaluated by decomposing their net effect into gains and regressions, not by aggregate improvement alone. We identify three regression modes skills should avoid, and find that reliability depends more on grounding and verification than on procedural skill choice.


Persistent memory is treated as an external runtime artifact. Shows ordinary production feedback -- one-bit outcome verdicts and after-the-fact corrections -- is a sufficient signal for continual learning when a frozen model is paired with external memory distilling each episode into retrievable natural-language rules. On tau-bench banking, against a static-RAG control over the full policy corpus, outcome verdicts alone lift single-trial success to 1.6x baseline and corrections to 2.6x, solving 22 of 84 tasks the baseline never solves. The canonical 'the loop learns, the weights don't' result.

Use Learning on the Job: Continual Learning from Deployment Feedback for Frozen-Weights Agents to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.22157; inspect its method and evaluation before treating results as production evidence.

[2607.22157] Learning on the Job: Continual Learning from Deployment Feedback for Frozen-Weights Agents

AI agents encounter learning opportunities in every episode they run, and discard nearly all of them: the underlying models are frozen at deployment, so an agent that resolves a difficult request today starts from zero when it recurs tomorrow. Yet ordinary operation already produces feedback, in the form of outcome verdicts and after-the-fact corrections. We show that this feedback is a sufficient signal for continual learning when the frozen model is paired with an external memory that distils each episode into retrievable natural-language rules. On the banking domain of $\tau$-bench, against a static-RAG control retrieving over the complete policy corpus, learning from the one-bit outcome verdict lifts single-trial success to 1.6$\times$ the baseline, and learning from corrections to 2.6$\times$, converting 22 of the 84 tasks the baseline never solves. The result spans the deployment spectrum, measured on Mistral Large, an open-weights model that organisations with data sovereignty requirements can self-host, and replicated on a frontier model, Claude Sonnet 5. The accumulated memory also transfers: each model, reading the store built by the other, rises above its own no-memory baseline. The harness, protocol, and data are released.

Valentin Tablan; Scott Taylor; Kristoffer Bernhem


Verification is promoted from a final check to a loop-control signal. Names the central dilemma of self-evolving training loops: environment-bound methods get precise feedback but stay narrow, while open-ended self-generation broadens the task space and loses reliable verification, letting misleading rewards pollute the training loop. Positions agent skills as the middle ground -- each skill guarantees deep verifiable execution in a scenario while dynamic routing across skills preserves open-endedness. Skill-SP couples a proposer, a solver, and a dynamic skill controller in an RL loop.

Use Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.22529; inspect its method and evaluation before treating results as production evidence.

[2607.22529] Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills

LLM training is shifting from manual design and annotation to interaction-driven self-evolution. However, existing self-evolutionary methods face a fundamental dilemma between task diversity and verification reliability: environment-bound methods obtain precise feedback but confine learning to narrow domains, while open-ended self-generation broadens the task space but lacks reliable verification, allowing misleading rewards to pollute the training loop. We identify agent skills as a powerful middle ground to reconcile this tension: each skill ensures deep, verifiable execution in a specific scenario, while dynamic routing across skills maintains open-ended task variety. Leveraging this insight, we introduce Skill Self-Play (Skill-SP), a co-evolutionary framework comprising a proposer, a solver, and a dynamic skill controller. Orchestrated via a reinforcement learning loop, these components co-evolve in a continuous self-play loop: the proposer generates challenging tasks conditioned on dynamically sampled skills; the solver explores candidate solutions to push its capability boundaries; and the skill controller collects execution feedback to update and expand the skill library. This interactive co-evolution effectively bridges the gap between structured verification and open-ended exploration. Empirical evaluations on tool-use and reasoning benchmarks demonstrate that Skill-SP, serving as a robust evolution engine, consistently pushes the performance ceiling of competent backbones while catalyzing striking turnarounds for initially misaligned models. Our code is available at https://github.com/Qwen-Applications/skill-self-play.

Siyuan Huang; Pengyu Cheng; Haotian Liu; Tao Chen; Yihao Liu; Jingwei Ni; Shijie Zhou; Ziyi Yang; Gangwei Jiang; Mengyu Zhou; Yu Cheng; Xiaoxi Jiang; Guanjun Jiang


Connects Loop Engineering to prior agent-loop and feedback-loop research. Hypothesizes that AlphaEvolve-style test-time self-evolution succeeds because of meta-skills -- notably self-reflection against environment feedback -- that conventional post-training neglects entirely. MetaEvolve cultivates them through a data-synthesis pipeline, evolution-aware RL, and inference-time evolutionary search, grounded in coding where execution yields continuous reward beyond binary correctness. Training the loop behavior rather than the task behavior is a distinct and underexplored lever.

Use Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.21971; inspect its method and evaluation before treating results as production evidence.

[2607.21971] Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning

Test-time scaling through iterative self-evolution with environment feedback, as demonstrated by AlphaEvolve, shows remarkable performance gains. We hypothesize that the success of such evolution frameworks hinges on meta-skills, such as self-reflection with environment feedback, that enable effective multi-round refinement, yet are largely neglected by traditional post-training. To bridge this gap, we present MetaEvolve, a framework designed to develop these meta-skills via a data synthesis pipeline, evolution-aware reinforcement learning (RL), and inference-time evolutionary search. Concretely, we ground MetaEvolve in coding, where program execution provides natural, continuous reward signals beyond binary correctness. Building on these signals, we synthesize evolution trajectories as training data, each containing a current program, its fitness score (combining correctness and efficiency), and a history of prior attempts, and train the model via RL with verifiable rewards derived from test case execution. By training on large-scale code data, we aim to inspire generalizable domain-agnostic meta-skills that can transfer broadly to open-ended problems where such rich training signals are scarce. Across seven coding benchmarks, MetaEvolve outperforms the strongest baseline by 10.01% absolute on in-distribution tasks and 24.12% on out-of-distribution tasks. On open-ended algorithm optimization problems entirely outside the training domain, it further achieves a 46.9% relative improvement. These results demonstrate that explicitly cultivating self-evolution meta-skills offers a principled path toward more capable and autonomously self-evolving AI.

Shujin Wu; Cheng Qian; Xiusi Chen; Heng Ji


Durable execution and replay are treated as first-class loop infrastructure. Studies why executable feedback on one task rarely becomes durable capability on the next, and names the two obstacles precisely: trajectory-derived artifacts often encode source-specific repairs rather than cross-task mechanisms, and a weaker target model may not be able to operationalize a valid abstract procedure (the abstraction-execution gap). SciConsolidate contrasts verified successes against failures to induce cross-task procedures, gates them through development-validation, and uses failure-informed answer-free query synthesis to expand consolidation data without reference solutions.

Use From Execution to Capability: Scientific Experience Consolidation via Procedural Knowledge Synthesis to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.24459; inspect its method and evaluation before treating results as production evidence.

[2607.24459] From Execution to Capability: Scientific Experience Consolidation via Procedural Knowledge Synthesis

Large language models increasingly solve scientific-computing tasks, but executable feedback from one problem rarely becomes durable capability on subsequent problems. We study scientific-computing experience consolidation: converting verified runtime experience into transferable procedural knowledge and persistent model improvement. This setting presents two challenges: trajectory-derived artifacts may encode source-specific repairs rather than cross-task computational mechanisms; and a weaker target model may be unable to operationalize an otherwise valid abstract procedure - an abstraction-execution gap. We introduce SciConsolidate, which contrasts verified successes and failures to induce cross-task procedures, selects them through a development-validation gate, and uses failure-informed, answer-free query synthesis to expand the consolidation data without requiring pre-existing reference answers. Because the target model may not directly execute these abstractions, a stronger model concretizes them into executable code supervision for standard, procedure-free SFT; a matched no-procedure teacher branch isolates the value of procedural guidance. On SciCode, runtime procedure injection improves Qwen3.6-27B by +3.85/+6.26 sub-step/main-problem points, but yields almost no aggregate main-problem gain for Qwen3.5-9B, providing operational evidence of the abstraction-execution gap. After procedure-guided concretization, the 9B student improves under procedure-free deployment by +3.89/+6.25 points over the no-procedure SFT control and by +5.62/+11.25 over the original 9B model. These results establish an experience-to-capability pathway for scientific computing and provide a practical starting point for scaling self-improving scientific assistance.


Control flow is represented as an inspectable graph rather than an opaque prompt loop. Scaffold optimization as tree search: extends UCT with a reuse-aware regularization term derived from a bipartite prefix reuse graph, biasing selection toward branches that reuse already-materialized configuration prefixes so the search stops re-executing shared pipeline stages under tight evaluation budgets. The RAGSpace framework unifies heterogeneous components from LongRAG, LightRAG and others into one searchable space. Useful for teams hand-tuning agentic pipelines by intuition.

Use Agent-UCT: Upper Confidence Bounds Applied to Trees for Agentic Workflow Optimization with Cost-Awareness to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.24162; inspect its method and evaluation before treating results as production evidence.

[2607.24162] Agent-UCT: Upper Confidence Bounds Applied to Trees for Agentic Workflow Optimization with Cost-Awareness

Optimizing agentic workflows, such as retrieval-augmented generation (RAG) pipelines, requires navigating a combinatorial space of discrete component choices under tight evaluation budgets. Existing approaches - heuristic search, black-box optimization, and standard tree search methods - do not explicitly exploit the compositional structure of these workflows, leading to redundant computation and inefficient budget allocation. We introduce Agent-UCT (Agent-based Cost-Aware Upper Confidence Bounds Applied to Trees), a tree search algorithm that extends UCT with a reuse-aware regularization term derived from a bipartite prefix reuse graph. Agent-UCT biases selection toward branches that leverage previously materialized configuration prefixes, reducing redundant execution while maintaining effective exploration. Our framework, RAGSpace, unifies heterogeneous RAG components from LongRAG, LightRAG, and Self-RAG into a five-dimensional configuration space, enabling systematic cross-framework recombination. WTB (Workflow Test Bench) provides deterministic replay, content-addressable caching, and transactional consistency, ensuring that intermediate states are materialized once and reused across the search. Experiments on HotpotQA and UltraDomain demonstrate that Agent-UCT identifies configurations with the highest out-of-sample performance among the evaluated fixed framework presets. Under full-pool evaluation, bipartite prefix reuse reduces logical search cost by 73.6% relative to the no-prefix-sharing cost upper bound. Compared with full-pool evaluation, sampling-based evaluation further achieves a 4.2x wall-clock speedup. Agent-UCT, RAGSpace, and WTB together provide a unified framework for cost-aware, reproducible, and compositionally efficient agentic workflow optimization.

Yang Li; Hai Liu; Dian Shao; Yu Wang; Xiyu Chen; Sergey Volkov; Bozhi Wang; Ziyu Sun; Sihang Liu; Ye Luo; Xiaowei Zhang


The work targets tasks that exceed a single context window or prompt session. Builds a unified controlled multi-turn environment to study where long-horizon planning ability actually comes from, which opaque internet-scale training makes impossible to isolate. Findings usable by loop designers: explicit world-model construction via chain-of-thought state-transition modeling yields stronger long-horizon generalization, atomic skills alone do not compose, a little long-horizon data goes a long way, and suboptimal trajectories are severely harmful because errors amplify over turns. Also covers single- and multi-teacher on-policy agentic distillation.

Use The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.24720; inspect its method and evaluation before treating results as production evidence.

[2607.24720] The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation

Multi-turn long-horizon planning is critical for foundation model agents, yet how to fundamentally improve it remains unclear. Existing models are trained on uncontrollable and opaque Internet data, making it difficult to identify how planning ability is acquired, shaped, and integrated. To address this challenge, we introduce a unified and controlled multi-turn environment that enables precise control. It allows systematically study long-horizon planning across three stages. (1) Planning ability acquisition during pre-training. We study data format, distribution, and quality. Explicit world model construction through CoT state transition modeling yields stronger long-horizon generalization. Atomic skills alone are insufficient for compositional generalization, whereas a litte long-horizon data works. Moreover, suboptimal trajectories severely impair performance because errors amplify over long horizons. (2) Planning ability shaping via GRPO and OPD post-training. Through mutual information, we distinguish general planning patterns from task-specific planning knowledge. For planning patterns, we identify three application regions of post-training: unnecessary, effective, and unsupported. OPD has a broader effective region than GRPO under low-quality and long-horizon settings, as it provides more consistent update directions. For planning knowledge, distilling unseen procedures from a teacher with different knowledge may impair student's prior world modeling without fully establishing new knowledge. (3) Planning ability integration through MOPD post-training. We show that multi-teacher on-policy distillation (MOPD) integrates capabilities by converging to shared planning-pattern across environments. Compatible patterns enable cross-environment generalization, partially shared patterns support continual learning, while completely conflicting patterns cause severe interference.

Tianyi Men; Zhuoran Jin; Kang Liu; Jun Zhao


Context is managed as durable loop state rather than a single prompt payload. Formalizes the loop as a frozen inner model wrapped by an outer context policy, making prompt templates, demonstrations, and retrieved context the controlled variable, with stability guarantees and uncertainty calibration for online updates. A rare attempt to give harness engineering actual control-theoretic foundations.

Use Context Assembly as the Controlled Variable: A Control-Theoretic View of Harness Policies for Frozen LLM Agents to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.25408; inspect its method and evaluation before treating results as production evidence.

[2607.25408] Context Assembly as the Controlled Variable: A Control-Theoretic View of Harness Policies for Frozen LLM Agents

A growing body of 2026 work applies control theory to LLM agents: Lyapunov-certified stability for tool-mediated controllers (Prinos et al., "Stable Agentic Control", 2026), sample-complexity bounds for sparse policies over massive discrete tool universes (Majumdar, "Sparse Agentic Control", 2026), and regulatory-control decompositions of multi-agent systems into auditable feedback loops (Nogueira and Skogestad, 2026). We do not claim to introduce control theory to LLM agents -- that ship has sailed. Our narrower claim is about what the controlled variable is. Prior work controls tool selection, inter-agent message routing, or the agent's raw action stream. We instead treat context assembly itself -- which prompt template, which few-shot demonstrations, how much retrieved context, how many planning/verification passes -- as the controlled variable, learned online by a contextual bandit or REINFORCE policy sitting outside a frozen model. This paper develops the formal decomposition (inner frozen policy $\pi_\theta$, outer context policy $\pi_\phi$), gives a stability argument for the online controller in the sense used by Zhang et al. (2026) (non-decreasing expected reward under bounded policy change), and reports an uncertainty-calibration analysis of the controller's own confidence against realized task outcomes. The applied counterpart to this paper instantiates the same controller across three domains and two model providers and releases the dataset, trajectory logs, and a deployment recipe; here we focus on the formal framing and the stability/uncertainty evidence a control-theoretic claim requires.

6 pages, 2 figures, 1 table. Code and companion paper's data: https://github.com/dpaul0501/context-optimization-rl


Connects Loop Engineering to prior agent-loop and feedback-loop research. Argues agentic systems are pre-standardization and proposes deriving abstractions by extending classical OS and cloud primitives to stochastic, natural-language-mediated execution, the way POSIX and Kubernetes consolidated their eras. A useful frame for where loop infrastructure is heading.

Use Towards an Agent Operating System - Lessons from Classical and Cloud OS to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Research source arXiv:2607.25076; inspect its method and evaluation before treating results as production evidence.

[2607.25076] Towards an Agent Operating System - Lessons from Classical and Cloud OS

Every major wave of platform software follows the same arc: an initial period of experimentation with competing frameworks and ad-hoc implementations, followed by the articulation of a small set of stable abstractions with well-defined semantics, and finally consolidation around those abstractions into a platform that applications can portably target. POSIX did this for classical operating systems; Kubernetes did it for the cloud. Agentic AI systems - autonomous, LLM-driven agents that plan, use tools, maintain memory, and collaborate - are currently in the experimentation phase of the third such wave. dozens of frameworks and protocols have emerged, but no community consensus exists on what the core abstractions are or what guarantees they carry. Without that consensus, agentic applications cannot be written portably, platforms cannot compose reliably, and the field cannot advance beyond prototype deployments. We argue that the path forward is to follow the prior-wave methodology: derive new agentic abstractions by extending classical OS and cloud OS primitives to stochastic, natural-language-mediated execution, specify their semantics precisely, and consolidate around them - just as POSIX and Kubernetes consolidated their respective waves.


Connects Loop Engineering to prior agent-loop and feedback-loop research. Anthropic research report (2026-07-28) that doubles as one of the most detailed first-party accounts of a multi-day autonomous agent loop.

Use Discovering Cryptographic Weaknesses with Claude to understand the evidence, vocabulary, and lineage behind recurring agent systems.

Contextual source from www.anthropic.com; useful for practice signals or boundary conditions, not independent validation.

Discovering cryptographic weaknesses with Claude \ Anthropic

Anthropic researchers find weaknesses in cryptographic algorithms with Claude Mythos Preview