deer-workflow

Control flow is represented as an inspectable graph rather than an opaque prompt loop. Created 2026-07-26. Code-first runtime that keeps orchestration and control flow in TypeScript while delegating only the semantic work to replaceable agent runtimes (Codex CLI and Claude supported out of the box). Emits an observable JSONL event stream for downstream automation and ships an interactive TUI for watching execution. The design position, deterministic graph in host code, swappable model-side workers, structured event log for auditability, is the practical answer to prompt-defined orchestration, and it is a distinct project from the already-listed bytedance/deer-flow. MIT.

Use deer-workflow to turn a recurring-agent idea into an explicit loop contract.

Inspectable GitHub source (403 stars; 36 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - deerwork-ai/deer-workflow: An open-source graph engineering runtime that keeps orchestration in TypeScript and delegates semantic work to replaceable Agent runtimes. · GitHub

An open-source graph engineering runtime that keeps orchestration in TypeScript and delegates semantic work to replaceable Agent runtimes. - deerwork-ai/deer-workflow


Control flow is represented as an inspectable graph rather than an opaque prompt loop. Converts each completed trajectory and its evaluator signals into posterior evidence for bounded harness updates, so the scaffold itself evolves via episodic memory and state graphs instead of staying static. Directly operationalizes the self-improving-loop premise: the harness is the learned artifact, not the weights.

Use Living-Harness Is an Interactive-Agent Evolver to turn a recurring-agent idea into an explicit loop contract.

Research source arXiv:2607.26598; inspect its method and evaluation before treating results as production evidence.

Specify a loop contract and operating pattern.

[2607.26598] Living-Harness Is an Interactive-Agent Evolver

Large language model (LLM) agents may recover from a failure within an episode or after a retry, yet the same execution failure can recur in later tasks because post-episode feedback rarely revises the persistent harness that guides future interactions. Static harnesses improve reliability through fixed tools, context, memory, and workflow structures, but remain unchanged after deployment. We propose $\textbf{Living-Harness}$, a self-evolving agent harness that converts each completed trajectory and its evaluator signals into posterior evidence for bounded harness updates. Guided by a domain-level $\textbf{Evolution-SOP}$ ($\textbf{S}$tandard $\textbf{O}$perating $\textbf{P}$rocedure), Living-Harness extracts an episode abstraction and structured update evidence, and writes two complementary forms of procedural knowledge: episodic memory that records trigger conditions, failure patterns, and recovery actions, and a state graph that records state nodes, repair edges, and transition rules. The updated harness state is retrieved to guide future interactions, while tools and base context remain frozen, allowing procedural repairs to accumulate across evolution cycles. On eight interactive environments derived from $\tau^2$-Bench and MultiWOZ-2.4, Living-Harness improves average Pass@1 over the strongest interactive baseline by 10.07 and 9.91 percentage points, respectively, and supports retrieval-only reuse of the evolved harness state across model backbones.

Yuetian Du; Yucheng Wang; He Xu; Jiexu Xu; Shanwen Tan; Bing Zhao; Boyu Yang; Zhijie Xu; Ming Kong; Hu Wei; Jie Liu; Qiang Zhu


Orchestration and control flow are made explicit and inspectable. Uses counterfactual causal learning to let an agent's harness adapt orchestration per task and environment, cutting long-horizon compute while holding success rate. One of the first papers to treat harness configuration as a learnable causal policy rather than a hand-tuned constant.

Use CHILL-Harness: Counterfactual Harness Learning for Efficient Reasoning in Long-Horizon Agents to turn a recurring-agent idea into an explicit loop contract.

Research source arXiv:2607.25825; inspect its method and evaluation before treating results as production evidence.

[2607.25825] CHILL-Harness: Counterfactual Harness Learning for Efficient Reasoning in Long-Horizon Agents

Agent harnesses have become the operational infrastructure of modern large language model agents, coordinating context, tools, verification, and execution control to translate latent model capability into reliable long-horizon behavior. However, reliable long-horizon behavior requires harness control to adapt to task demands, execution environments, and evolving execution states, whereas current harnesses predominantly rely on hand-crafted or globally fixed policies; this mismatch manifests as unnecessary computational overhead and, in adverse cases, reduced task success. To address this limitation, we formulate the task of enabling adaptive orchestration in harness systems as a causal learning problem and propose Counterfactual Harness Intervention Learning for Long-Horizon Agents (CHILL-Harness). CHILL-Harness intervenes at the orchestration layer to enable advantage-guided workflow adaptation, thereby improving reasoning and execution efficiency while preserving task performance. Specifically, we develop causal intervention effect learning as the effect-estimation component of CHILL-Harness to estimate intervention-relative workflow advantage from confidence-weighted execution evidence and identify advantageous workflow adaptations. We further introduce advantage-realizing causal orchestration as its realization component to adaptively allocate counterfactual reasoning and realize only workflow adjustments supported by sufficient expected advantage. Finally, we incorporate a success-preserving objective and advantage-margin authorization constraints into CHILL-Harness to promote reliable adaptation. Extensive experiments on heterogeneous long-horizon tasks spanning information seeking, software engineering, and terminal interaction show that CHILL-Harness consistently preserves or improves task success while substantially reducing token consumption and execution time.

Jiarun Fu; Lizhong Ding; Sida Chen; Honglei Xin; Chunhui Zhang; Pengqi Li; Qiuning Wei; Ye Yuan; Guoren Wang


Packages the evidence as queryable CSV and JSONL rather than only a rendered page. RL-optimizes the harness around a frozen model by treating it as a fixed action space rather than allowing unconstrained self-modification, evaluated across tool-use workflows, code generation, and retrieval QA with released code and data. The bounded-action-space framing is the safety-relevant counterpoint to open-ended self-editing agents.

Use A Control System, a Dataset, and a Recipe for Making Frozen LLM Agents Learn a Domain to turn a recurring-agent idea into an explicit loop contract.

Research source arXiv:2607.25415; inspect its method and evaluation before treating results as production evidence.

[2607.25415] A Control System, a Dataset, and a Recipe for Making Frozen LLM Agents Learn a Domain

Production LLM agents are increasingly assembled from a frozen model wrapped in a harness: a prompt template, a tool set, a memory/retrieval layer, a planning strategy, and a verification policy. Two 2026 systems, Meta-Harness (Lee et al., 2026) and HyperAgents (Meta AI, 2026), show that this harness can itself be optimized or even self-rewritten by an agentic proposer -- at the cost of either an expensive code-search loop or unconstrained self-modifying code, neither of which is auditable or usable with a fully black-box model API. We take a narrower, more constrained position: treat the harness as a small, fixed, human-legible action space and learn a policy over it online with classic sample-efficient reinforcement learning (an $\epsilon$-greedy contextual bandit and REINFORCE), scored against a multi-objective reward (task success, verifier score, policy compliance, cost, latency, and an unsupported-claim penalty). We instantiate this control system with DSPy (Khattab et al., 2024) as both the context assembler and the source of the strongest non-adaptive baseline (a DSPy BootstrapFewShot static prompt), and evaluate it across three verifiable task domains -- tool-use workflows, code generation (HumanEval), and multi-hop retrieval QA (HotpotQA) -- and two model providers (a local Ollama model and AWS Bedrock). We release the harness-control-system code, the cross-domain verifiable task suite, the full trajectory/reward-decomposition logs from training, and a provider-agnostic deployment recipe for applying this to a new organization's domain and verification setup.

8 pages, 1 figure, 3 tables. Code and dataset: https://github.com/dpaul0501/context-optimization-rl


Distills reusable agent-control patterns that are not tied to a single vendor implementation. Frames how many tools an agent should acquire as an optimal-stopping problem, training on the gap between stopping now and optimal continuation and proving the objective aligns with the stopping target under heterogeneous costs. Directly addresses when to halt, one of the least-formalized parts of loop design.

Use Scores Are Not Decisions: Cost-Aware Stopping for Tool Acquisition in LLM Agents to turn a recurring-agent idea into an explicit loop contract.

Research source arXiv:2607.27083; inspect its method and evaluation before treating results as production evidence.

[2607.27083] Scores Are Not Decisions: Cost-Aware Stopping for Tool Acquisition in LLM Agents

As LLM agents increasingly depend on diverse external services such as search engines, databases, and connectors, agent harnesses face a fundamental tool-selection challenge: acquiring too few tools leaves the task under-informed, while too many adds cost, context load, and privacy exposure. Routers and retrievers can rank candidate tools by relevance, but a ranking alone does not determine how many are worth selecting. Existing approaches leave acquisition under heterogeneous costs unaddressed. We formulate this decision as cost-aware marginal decision-focused stopping (CAM-DF) over ranked tool prefixes, with CAM-DF-lite as a compact interpretable variant. We train directly on the offline gap between stopping now and the best continuation: its sign labels the decision, its magnitude weights each error by the payoff at stake. We prove this objective is Bayes-aligned with the stopping target and that score-only rules are suboptimal under heterogeneous costs. We evaluate on 1,343 tasks across five tool-use domains. On $\tau$-bench Retail, CAM-DF attains the highest payoff among deployable methods, with gains over a predict-then-threshold baseline across all five ranking sources and two cost regimes. Our approach is state-of-the-art under heterogeneous costs and high cost pressure, with larger gains under weaker rankings. In live execution, CAM-DF exposes the agent to 37\% fewer tools than full access while maintaining comparable task success. The CAM-DF family is a lightweight pre-execution plugin that turns existing tool rankings into lower-cost acquisition decisions without fine-tuning the underlying LLM.

Yicheng Feng; Yan Zhang; Yan Cheng; Wei Qi


Skill Recorder

Distills reusable agent-control patterns that are not tied to a single vendor implementation. Microsoft's tool for authoring loops from demonstration instead of from prose. It captures a real work session locally, clicks, app and window switches, pages visited, optional spoken narration, then uses GitHub Copilot CLI to reconstruct what you actually did as one intent plus an ordered step list you review and edit.

Use Skill Recorder to turn a recurring-agent idea into an explicit loop contract.

Inspectable GitHub source (2,234 stars; 236 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - microsoft/skill-recorder: Desktop app that records your on-screen work session and uses the GitHub Copilot CLI to reconstruct it as an intent + ordered steps, then builds a reusable Skill or Automation for Microsoft Scout, Microsoft Copilot Cowork, or Copilot Studio. · GitHub

Desktop app that records your on-screen work session and uses the GitHub Copilot CLI to reconstruct it as an intent + ordered steps, then builds a reusable Skill or Automation for Microsoft Scout, Microsoft Copilot Cowork, or Copilot Studio. - microsoft/skill-recorder


Distills reusable agent-control patterns that are not tied to a single vendor implementation. Identifies that prose skill files force an agent to re-derive control flow on every run, then introduces 'skill compilation', lowering prose skills into executable harnesses through a typed IR (AG-IR). Reports 86% vs 56% of mandated steps executed, 2.3x more completed procedures, and 42% fewer tokens, stable across model generations.

Use SIGIL: Compiling Agent Skills into Typed Harnesses to turn a recurring-agent idea into an explicit loop contract.

Research source arXiv:2607.27309; inspect its method and evaluation before treating results as production evidence.

[2607.27309] SIGIL: Compiling Agent Skills into Typed Harnesses

AI-Integrated agents increasingly acquire capability from skills: prose procedure files loaded into a model's context and run by a tool-calling loop. A skill is described to the runtime but never encoded in it, so the model re-derives its control flow on every run and may skip mandated verification. Across 30 skills and two model generations, a prose agent performs only 56% of the steps its own skill mandates, while producing artifacts that pass output checks. The remedy is known: write a harness, in which the procedure is program structure. However, hand-writing harnesses is tedious and discards the authoring surface that made skills succeed. To address this limitation, we introduce Skill Compilation, realized in SIGIL, which compiles a prose skill into an executable harness. At its center is AG-IR, a typed agentic intermediate representation separating model-owned cognition from code-owned mechanism. Compiled harnesses perform 86% of mandated steps, complete the full procedure 2.3x as often, and require 0.58x the tokens. Notably, the guarantee is model-independent: the harness holds at 86% across two model generations while prose swings from 56% to 68%.

Jayanaka Dantanarayana; Savini Kashmira; Lingjia Tang; Jason Mars


Distills reusable agent-control patterns that are not tied to a single vendor implementation. Identifies a concrete instability in self-improving loops: when the LLM both picks the direction of a modification and generates the concrete hypothesis, search becomes unstable under limited experiment budgets.

Use RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems to turn a recurring-agent idea into an explicit loop contract.

Research source arXiv:2607.29241; inspect its method and evaluation before treating results as production evidence.

[2607.29241] RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems

Optimizing modern recommender models still depends heavily on engineers manually iterating over architectural, objective, and training-strategy changes. While LLM-based agents can automate this trial-and-error process, allowing the LLM to both select modification directions and generate concrete hypotheses often leads to unstable search under limited experiment budgets. Inspired by the above challenge, we propose RecHarness, a Bandit-Routed Agentic Harness for automated recommender model optimization. RecHarness separates the optimization process into two steps: a bandit router selects the next modification direction according to historical validation feedback, while the LLM generates a concrete optimization hypothesis and executable code edit within the selected direction. To sustain long-horizon exploration, RecHarness uses a jump-basin mechanism to activate a structural-jump arm when local edits stagnate. Across multiple recommendation tasks, datasets, and model backbones, RecHarness achieves more stable performance improvements and uses limited trial budgets more effectively than LLM-reasoning search. During a 7-day online A/B test on a large-scale short-video advertising platform, the selected candidate improves ADVV by 2.084%, Revenue by 0.534%, and Exposure by 0.559%. Code is available at https://github.com/6lyc/RecHarness.

Haoran Ling; Yuecheng Li; Zeyu Song; Jing Yao; Shuwen Kang; Chi Lu; Wenjin Wu; Peng Jiang


State persistence is explicit enough for repeated runs and handoff. Attacks the instability of harness self-evolution by anchoring evidence at function level, recalibrating which historical insights still apply given current state, and distilling role-conditioned search intent. Answers the practical question of when a past lesson should still steer the loop rather than mislead it.

Use DREvo: Distilling Recalibrated Historical Experience for Harness Self-Evolution to turn a recurring-agent idea into an explicit loop contract.

Research source arXiv:2607.26722; inspect its method and evaluation before treating results as production evidence.

[2607.26722] DREvo: Distilling Recalibrated Historical Experience for Harness Self-Evolution

Harness plays a critical role in large language model agent performance, and building a high-performing harness requires substantial expert effort. Therefore, recent research has increasingly explored harness self-evolution, which iteratively proposes, evaluates, and improves harnesses using historical trial experience. However, accumulated historical experience does not always translate into stable search guidance, and performance often fluctuates substantially across evolution iterations, making it difficult to reliably discover high-performing harnesses under a limited evolution budget. We identify two limitations in how existing harness self-evolution methods leverage historical experience: (1) Lack of dynamic reassessment of whether historical experience remains valid for the current harness, and (2) Lack of explicit mechanisms for translating valid historical experience into actionable search directions. To address these limitations, we propose a new harness self-evolution method, named DREvo, which integrates function-level evidence anchoring, state-dependent evidence recalibration, and role-conditioned search intent distillation to determine which historical evidence remains valid and where the harness should evolve next. Under limited evolution budgets, DREvo exhibits smoother evolution trajectories, achieves the highest accuracy on all five benchmarks, and delivers average gains of 16.2% and 14.2% over the evaluated baselines on domain reasoning and agentic tasks, respectively.

Hanghui Guo; Weijie Shi; Zhangze Chen; Shengxiang Xu; Yishu Wang; Yimei Zhang; Wangze Ni; Jia Zhu; Shimin Di


Distills reusable agent-control patterns that are not tied to a single vendor implementation. Treats blind-spot discovery as a learnable capability rather than assuming failures are already surfaced: an RL-trained Mentor policy generates diagnostic probe tasks, clusters recurring failure patterns, and converts them into reusable corrective skills without touching the executor or needing human labels.

Use SkillMentor: LLM Agent Self-Evolution via Learning Blind-Spot Diagnosis to turn a recurring-agent idea into an explicit loop contract.

Research source arXiv:2607.27360; inspect its method and evaluation before treating results as production evidence.

[2607.27360] SkillMentor: LLM Agent Self-Evolution via Learning Blind-Spot Diagnosis

Agent self-evolution has primarily focused on learning how to act, while overlooking an equally important capability: learning to discover what an agent does not know. Existing approaches typically assume that failure discovery is given, focusing on how to repair failures once they are identified. We ask whether blind-spot diagnosis itself can be learned. We thus study diagnosis as an agent capability separate from execution, and exclude two alternative sources of progress: executor adaptation and human supervision. Under these constraints, performance cannot improve through executor updates or annotated examples, forcing all improvements to originate from the learned diagnostic capability. We propose SkillMentor, which trains a Mentor policy via reinforcement learning to generate diagnostic tasks, identify recurrent failure modes, and curate them into reusable corrective skills. Across AppWorld and BFCLv3, SkillMentor improves executor performance by an average of 44.2%. These results suggest that blind-spot diagnosis is a learnable capability, enabling self-evolution without updating executor weights or relying on human-curated data.

Xiaoyi Bao; Yuanzhen Xie; Yunzhi Tan; Jinghang Gu; Zhongqing Wang; Chu-Ren Huang; Bo Hu; Zang Li


SWE-agent

Uses real automated software-engineering systems as evidence for practical loop architectures. Agent-computer interface and autonomous software engineering agent for repository tasks.

Use SWE-agent to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (20,018 stars; 2,186 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

Choose runtimes, tools, and delegation surfaces.

GitHub - SWE-agent/SWE-agent: SWE-agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges. [NeurIPS 2024] · GitHub

SWE-agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges. [NeurIPS 2024] - GitHub - SWE-agent/SWE-agent: SWE-agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges. [NeurIPS 2024]


Uses real automated software-engineering systems as evidence for practical loop architectures. Paper behind SWE-agent and its interface design.

Use SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering to choose an implementation surface for repeatable agent work.

Research source arXiv:2405.15793; inspect its method and evaluation before treating results as production evidence.

[2405.15793] SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering

Language model (LM) agents are increasingly being used to automate complicated tasks in digital environments. Just as humans benefit from powerful software applications, such as integrated development environments, for complex tasks like software engineering, we posit that LM agents represent a new category of end users with their own needs and abilities, and would benefit from specially-built interfaces to the software they use. We investigate how interface design affects the performance of language model agents. As a result of this exploration, we introduce SWE-agent: a system that facilitates LM agents to autonomously use computers to solve software engineering tasks. SWE-agent's custom agent-computer interface (ACI) significantly enhances an agent's ability to create and edit code files, navigate entire repositories, and execute tests and other programs. We evaluate SWE-agent on SWE-bench and HumanEvalFix, achieving state-of-the-art performance on both with a pass@1 rate of 12.5% and 87.7%, respectively, far exceeding the previous state-of-the-art achieved with non-interactive LMs. Finally, we provide insight on how the design of the ACI can impact agents' behavior and performance.

John Yang; Carlos E. Jimenez; Alexander Wettig; Kilian Lieret; Shunyu Yao; Karthik Narasimhan; Ofir Press

Neural Information Processing Systems Foundation

Published in Advances in Neural Information Processing Systems 37 (NeurIPS); the linked arXiv record remains available for open access.


mini-SWE-agent

Uses real automated software-engineering systems as evidence for practical loop architectures. Minimal coding agent that is useful for understanding the core loop without a large framework.

Use mini-SWE-agent to choose an implementation surface for repeatable agent work.

Working implementation, framework, runtime, or repository; signal comes from usable code and ecosystem adoption.


OpenHands

Uses real automated software-engineering systems as evidence for practical loop architectures. Open platform for AI software developers as generalist agents.

Use OpenHands to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (83,363 stars; 10,760 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - OpenHands/OpenHands: 🙌 OpenHands: AI-Driven Development · GitHub

🙌 OpenHands: AI-Driven Development. Contribute to OpenHands/OpenHands development by creating an account on GitHub.


Evaluation data is used as the feedback signal for improving loop behavior. Paper describing OpenHands, CodeActAgent, benchmarks, and generalist agent evaluation.

Use OpenHands: An Open Platform for AI Software Developers as Generalist Agents to choose an implementation surface for repeatable agent work.

Research source arXiv:2407.16741; inspect its method and evaluation before treating results as production evidence.

[2407.16741] OpenHands: An Open Platform for AI Software Developers as Generalist Agents

Software is one of the most powerful tools that we humans have at our disposal; it allows a skilled programmer to interact with the world in complex and profound ways. At the same time, thanks to improvements in large language models (LLMs), there has also been a rapid development in AI agents that interact with and affect change in their surrounding environments. In this paper, we introduce OpenHands (f.k.a. OpenDevin), a platform for the development of powerful and flexible AI agents that interact with the world in similar ways to those of a human developer: by writing code, interacting with a command line, and browsing the web. We describe how the platform allows for the implementation of new agents, safe interaction with sandboxed environments for code execution, coordination between multiple agents, and incorporation of evaluation benchmarks. Based on our currently incorporated benchmarks, we perform an evaluation of agents over 15 challenging tasks, including software engineering (e.g., SWE-BENCH) and web browsing (e.g., WEBARENA), among others. Released under the permissive MIT license, OpenHands is a community project spanning academia and industry with more than 2.1K contributions from over 188 contributors.

Xingyao Wang; Boxuan Li; Yufan Song; Frank F. Xu; Xiangru Tang; Mingchen Zhuge; Jiayi Pan; Yueqi Song; Bowen Li; Jaskirat Singh; Hoang H. Tran; Fuqiang Li; Ren Ma; Mingzhang Zheng; Bill Qian; Yanjun Shao; Niklas Muennighoff; Yizhe Zhang; Binyuan Hui; Junyang Lin; Robert Brennan; Hao Peng; Heng Ji; Graham Neubig

Published in International Conference on Learning Representations (ICLR); the linked arXiv record remains available for open access.


Agentless

Uses real automated software-engineering systems as evidence for practical loop architectures. Workflow-based approach for software issue resolution using localization, repair, and patch validation.

Use Agentless to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (2,091 stars; 237 forks; MIT license; updated 2026-08-06); popularity is context, not proof of reliability.

GitHub - OpenAutoCoder/Agentless: Agentless🐱: an agentless approach to automatically solve software development problems · GitHub

Agentless🐱: an agentless approach to automatically solve software development problems - OpenAutoCoder/Agentless


Uses real automated software-engineering systems as evidence for practical loop architectures. Useful contrast case: strong results through structured workflow rather than a fully open-ended agent.

Use Agentless: Demystifying LLM-based Software Engineering Agents to choose an implementation surface for repeatable agent work.

Research source arXiv:2407.01489; inspect its method and evaluation before treating results as production evidence.

[2407.01489] Agentless: Demystifying LLM-based Software Engineering Agents

Recent advancements in large language models (LLMs) have significantly advanced the automation of software development tasks, including code synthesis, program repair, and test generation. More recently, researchers and industry practitioners have developed various autonomous LLM agents to perform end-to-end software development tasks. These agents are equipped with the ability to use tools, run commands, observe feedback from the environment, and plan for future actions. However, the complexity of these agent-based approaches, together with the limited abilities of current LLMs, raises the following question: Do we really have to employ complex autonomous software agents? To attempt to answer this question, we build Agentless -- an agentless approach to automatically solve software development problems. Compared to the verbose and complex setup of agent-based approaches, Agentless employs a simplistic three-phase process of localization, repair, and patch validation, without letting the LLM decide future actions or operate with complex tools. Our results on the popular SWE-bench Lite benchmark show that surprisingly the simplistic Agentless is able to achieve both the highest performance (32.00%, 96 correct fixes) and low cost ($0.70) compared with all existing open-source software agents! Furthermore, we manually classified the problems in SWE-bench Lite and found problems with exact ground truth patch or insufficient/misleading issue descriptions. As such, we construct SWE-bench Lite-S by excluding such problematic issues to perform more rigorous evaluation and comparison. Our work highlights the current overlooked potential of a simple, interpretable technique in autonomous software development. We hope Agentless will help reset the baseline, starting point, and horizon for autonomous software agents, and inspire future work along this crucial direction.

Chunqiu Steven Xia; Yinlin Deng; Soren Dunn; Lingming Zhang


AutoCodeRover

Uses real automated software-engineering systems as evidence for practical loop architectures. Autonomous program improvement system for issue localization, patch generation, and validation.

Use AutoCodeRover to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (3,099 stars; 333 forks; NOASSERTION license; updated 2026-08-05); popularity is context, not proof of reliability.

GitHub - AutoCodeRoverSG/auto-code-rover: A project structure aware autonomous software engineer aiming for autonomous program improvement. Resolved 37.3% tasks (pass@1) in SWE-bench lite and 46.2% tasks (pass@1) in SWE-bench verified with each task costs less than $0.7. · GitHub

A project structure aware autonomous software engineer aiming for autonomous program improvement. Resolved 37.3% tasks (pass@1) in SWE-bench lite and 46.2% tasks (pass@1) in SWE-bench verified with each task costs less than $0.7. - AutoCodeRoverSG/auto-code-rover


Uses real automated software-engineering systems as evidence for practical loop architectures. Paper on autonomous code repair loops over real repositories.

Use AutoCodeRover: Autonomous Program Improvement to choose an implementation surface for repeatable agent work.

Research source arXiv:2404.05427; inspect its method and evaluation before treating results as production evidence.

[2404.05427] AutoCodeRover: Autonomous Program Improvement

Researchers have made significant progress in automating the software development process in the past decades. Recent progress in Large Language Models (LLMs) has significantly impacted the development process, where developers can use LLM-based programming assistants to achieve automated coding. Nevertheless, software engineering involves the process of program improvement apart from coding, specifically to enable software maintenance (e.g. bug fixing) and software evolution (e.g. feature additions). In this paper, we propose an automated approach for solving GitHub issues to autonomously achieve program improvement. In our approach called AutoCodeRover, LLMs are combined with sophisticated code search capabilities, ultimately leading to a program modification or patch. In contrast to recent LLM agent approaches from AI researchers and practitioners, our outlook is more software engineering oriented. We work on a program representation (abstract syntax tree) as opposed to viewing a software project as a mere collection of files. Our code search exploits the program structure in the form of classes/methods to enhance LLM's understanding of the issue's root cause, and effectively retrieve a context via iterative search. The use of spectrum-based fault localization using tests, further sharpens the context, as long as a test-suite is available. Experiments on SWE-bench-lite (300 real-life GitHub issues) show increased efficacy in solving GitHub issues (19% on SWE-bench-lite), which is higher than the efficacy of the recently reported SWE-agent. In addition, AutoCodeRover achieved this efficacy with significantly lower cost (on average, $0.43 USD), compared to other baselines. We posit that our workflow enables autonomous software engineering, where, in future, auto-generated code from LLMs can be autonomously improved.

Yuntong Zhang; Haifeng Ruan; Zhiyu Fan; Abhik Roychoudhury

Published in Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA); the linked arXiv record remains available for open access.


SWE-bench reading list

Uses real automated software-engineering systems as evidence for practical loop architectures. Maintained map of software engineering agent systems and related papers.

Use SWE-bench reading list to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (15 stars; 4 forks; updated 2026-06-30); popularity is context, not proof of reliability.

GitHub - SWE-bench/reading-list: Academic papers and works related to SWE-bench and SWE-agents · GitHub

Academic papers and works related to SWE-bench and SWE-agents - SWE-bench/reading-list


The work separates roles across agents, verifiers, or orchestration layers. ICSE'26 observe-analyze-repair loop with instrumentation, analysis, and repair agents, a history-learning mechanism, and a rollback to the last good state; iteration alone drives most of the gain.

Use TraceCoder: A Trace-Driven Multi-Agent Framework for Automated Debugging of LLM-Generated Code to choose an implementation surface for repeatable agent work.

Research source arXiv:2602.06875; inspect its method and evaluation before treating results as production evidence.

[2602.06875] TraceCoder: A Trace-Driven Multi-Agent Framework for Automated Debugging of LLM-Generated Code

Large Language Models (LLMs) often generate code with subtle but critical bugs, especially for complex tasks. Existing automated repair methods typically rely on superficial pass/fail signals, offering limited visibility into program behavior and hindering precise error localization. In addition, without a way to learn from prior failures, repair processes often fall into repetitive and inefficient cycles. To overcome these challenges, we present TraceCoder, a collaborative multi-agent framework that emulates the observe-analyze-repair process of human experts. The framework first instruments the code with diagnostic probes to capture fine-grained runtime traces, enabling deep insight into its internal execution. It then conducts causal analysis on these traces to accurately identify the root cause of the failure. This process is further enhanced by a novel Historical Lesson Learning Mechanism (HLLM), which distills insights from prior failed repair attempts to inform subsequent correction strategies and prevent recurrence of similar mistakes. To ensure stable convergence, a Rollback Mechanism enforces that each repair iteration constitutes a strict improvement toward the correct solution. Comprehensive experiments across multiple benchmarks show that TraceCoder achieves up to a 34.43\% relative improvement in Pass@1 accuracy over existing advanced baselines. Ablation studies verify the significance of each system component, with the iterative repair process alone contributing a 65.61\% relative gain in accuracy. Furthermore, TraceCoder significantly outperforms leading iterative methods in terms of both accuracy and cost-efficiency.

Jiangping Huang; Wenguang Ye; Weisong Sun; Jian Zhang; Mingyue Zhang; Yang Liu

Published in Proceedings of the 48th IEEE/ACM International Conference on Software Engineering (ICSE); the linked arXiv record remains available for open access.


Uses real automated software-engineering systems as evidence for practical loop architectures. Production loop where an agent exercises a spec surface as a synthetic power user behind ground-truth tests and quality gates, reporting 285+ self-correcting iterations and 1,000+ merged PRs with zero detected regressions.

Use The Kitchen Loop: User-Spec-Driven Development for a Self-Evolving Codebase to choose an implementation surface for repeatable agent work.

Research source arXiv:2603.25697; inspect its method and evaluation before treating results as production evidence.

[2603.25697] The Kitchen Loop: User-Spec-Driven Development for a Self-Evolving Codebase

Code production is now a commodity; the bottleneck is knowing what to build and proving it works. We present the Kitchen Loop, a framework for autonomous, self-evolving software built on a unified trust model: (1) a specification surface enumerating what the product claims to support; (2) 'As a User x 1000', where an LLM agent exercises that surface as a synthetic power user at 1,000x human cadence; (3) Unbeatable Tests, ground-truth verification the code author cannot fake; and (4) Drift Control, continuous quality measurement with automated pause gates. We validate across two production systems over 285+ iterations, producing 1,094+ merged pull requests with zero regressions detected by the regression oracle (methodology in Section 6.1). We observe emergent properties at scale: multi-iteration self-correction chains, autonomous infrastructure healing, and monotonically improving quality gates. The primitives are not new; our contribution is their composition into a production-tested system with the operational discipline that makes long-running autonomous evolution safe.


State persistence is explicit enough for repeated runs and handoff. Dissects 13 open-source coding-agent scaffolds and identifies five composable loop primitives (ReAct, generate-test-repair, plan-execute, retry, tree search) that real agents layer, mapping how control loop, tools, and state combine.

Use Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures to choose an implementation surface for repeatable agent work.

Research source arXiv:2604.03515; inspect its method and evaluation before treating results as production evidence.

[2604.03515] Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures

LLM-based coding agents can localize bugs, generate patches, and run tests with diminishing human oversight, yet the scaffolding code that surrounds the language model (the control loop, tool definitions, state management, and context strategy) remains poorly understood. Existing surveys classify agents by abstract capabilities (tool use, planning, reflection) that cannot distinguish between architecturally distinct systems, and trajectory studies observe what agents do without examining the scaffold code that determines why. This paper presents a source-code-level architectural taxonomy derived from analysis of 13 open-source coding agent scaffolds at pinned commit hashes. Each agent is characterized across 12 dimensions organized into three layers: control architecture, tool and environment interface, and resource management. The analysis reveals that scaffold architectures resist discrete classification: control strategies range from fixed pipelines to Monte Carlo Tree Search, tool counts range from 0 to 37, and context compaction spans seven distinct strategies. Five loop primitives (ReAct, generate-test-repair, plan-execute, multi-attempt retry, tree search) function as composable building blocks that agents layer in different combinations; 11 of 13 agents compose multiple primitives rather than relying on a single control structure. Dimensions converge where external constraints dominate (tool capability categories, edit formats, execution isolation) and diverge where open design questions remain (context compaction, state management, multi-model routing). All taxonomic claims are grounded in file paths and line numbers, providing a reusable reference for researchers studying agent behavior and practitioners designing new scaffolds.


A Self-Improving Coding Agent

Verification is promoted from a final check to a loop-control signal. An agent that edits its own code and tools and re-runs against a benchmark, lifting itself from 17% to 53% on a SWE-bench Verified subset, a concrete self-modifying improvement loop.

Use A Self-Improving Coding Agent to choose an implementation surface for repeatable agent work.

Research source arXiv:2504.15228; inspect its method and evaluation before treating results as production evidence.

[2504.15228] A Self-Improving Coding Agent

Recent advancements in Large Language Models (LLMs) have spurred interest in deploying LLM agents to undertake tasks in the world. LLMs are often deployed in agent systems: code that orchestrates LLM calls and provides them with tools. We demonstrate that an agent system, equipped with basic coding tools, can autonomously edit itself, and thereby improve its performance on benchmark tasks. We find performance gains from 17% to 53% on a random subset of SWE Bench Verified, with additional performance gains on LiveCodeBench, as well as synthetically generated agent benchmarks. Our work represents an advancement in the automated and open-ended design of agentic systems, and demonstrates a data-efficient, non gradient-based learning mechanism driven by LLM reflection and code updates.

Maxime Robeyns; Martin Szummer; Laurence Aitchison


Context is managed as durable loop state rather than a single prompt payload. Longitudinal study of 35 Qwen Code CLI releases with the model held constant, tracing coding-agent shifts to specific scaffolding changes in system prompts, tools, context management, and reasoning loops, and separating scaffolding regressions from model regressions.

Use Don't Blame the Large Language Model: How Scaffolding Evolution Shapes Coding Agent Quality to choose an implementation surface for repeatable agent work.

Research source arXiv:2607.03691; inspect its method and evaluation before treating results as production evidence.

[2607.03691] Don't Blame the Large Language Model: How Agent Harness Evolution Shapes Coding Agent Quality

Coding agents, autonomous systems that use large language models (LLMs) to resolve software engineering tasks, rely on agent harness: a middleware layer in between a developer and a large language model that orchestrates system prompts, tool execution, context management, and iterative reasoning loops. While these agent harnesses evolve at extreme velocities, no study has examined how this evolution affects agent quality (i.e., effectiveness and efficiency) over time. Practitioners regularly report quality regressions after agent harness updates, yet consistently attribute them to the underlying model rather than the harness itself. In this paper, we address this gap by conducting the first controlled longitudinal study that isolates the agent harness contribution. Unlike prior work that fixes the agent harness and varies the model, we fix the model and vary only the agent harness, evaluating 35 sequential releases to measure their impact on agent effectiveness and efficiency. We first empirically study the development and release evolution of five major open-source agent harnesses (i.e., Codex, Qwen Code, Gemini, OpenCode, and OpenHands), revealing extreme release velocities exceeding two releases per day and thousands of issues within months. We then perform a controlled deep dive into 35 sequential releases of the Qwen Code CLI, evaluating each against 50 stratified SWE-bench Verified tasks while holding the underlying LLM constant. We trace the resulting quality fluctuations to specific development patterns and architectural components, and illustrate our findings with concrete qualitative evidence linking individual pull requests to measured quality shifts.

Oussama Ben Sghaier; Hao Li; Bram Adams; Ahmed E. Hassan


Orchestration and control flow are made explicit and inspectable. MIT-licensed white-box agent harness built on the thesis that agent behavior is set by the orchestration code around the model as much as the model itself, letting researchers inspect, modify, and evaluate its orchestration logic with reported token-efficiency gains over existing harnesses.

Use ToFu: A White-Box, Token-Efficient Agent Harness for Researchers to choose an implementation surface for repeatable agent work.

Research source arXiv:2607.11423; inspect its method and evaluation before treating results as production evidence.

[2607.11423] ToFu: A White-Box, Token-Efficient Agent Harness for Researchers

Agentic coding tools present new opportunities to transform research workflows. The performance of agent systems built depends on both large language models (LLMs) and the harness around LLMs, which is the orchestration code that determines an agent's behavior. We present ToFu, an agentic harness for researchers that reads your codebase, edits files, runs commands, and integrates with your development tools. ToFu plays a dual role in research. As a research assistant, it supports practical research workflows with superior token efficiency, lower cost, and multilingual capability compared with existing agentic harnesses. Its release under the MIT License further enables local deployment for privacy-sensitive users. As a research object, ToFu provides a white-box agentic harness that allows researchers to inspect, modify, and evaluate its orchestration logic, tool-use behavior, and harness design, while retaining strong benchmark performance and an application-level user experience.

Junhao Ruan; Yuan Ge; Bei Li; Yongjing Yin; Yuchun Fan; Xin Chen; Jingang Wang; Chenglong Wang; Jingbo Zhu; Tong Xiao


Uses real automated software-engineering systems as evidence for practical loop architectures. Regime-by-design ablation measuring when constraining a coding agent to a single execute_code action helps or hurts, separating task regime from agent design so harness choices can be made from evidence rather than intuition.

Use When Does Restricting a Coding Agent to execute_code Help? to choose an implementation surface for repeatable agent work.

Research source arXiv:2607.10569; inspect its method and evaluation before treating results as production evidence.

[2607.10569] When Does Restricting a Coding Agent to execute_code Help? A Regime $\times$ Agent-Design Ablation

Modern coding agents expose multiple tool surfaces -- IDE primitives, bash, and Model Context Protocol (MCP) code-execution -- and the field has shipped three contradictory claims about which one matters. We run the missing crossed comparison: an integrity-clean three-arm ablation (baseline / bash_only / code_only) on synthetic computation tasks and SWE-bench Mini modification tasks, holding model, harness, and prompts fixed, with two agents (Claude Code, OpenAI Codex CLI) so the comparison spans both regime and agent-design axes. Across the four resulting (regime, agent) cells, restricting the agent to a single execute_code MCP tool is cheaper than -- or statistically tied with -- its cheapest tool-rich rival in three cells (significantly on Artifact/Claude and SWE-bench/Codex; directionally on Artifact/Codex), with pass rates statistically tied within each cell. The lone exception is SWE-bench/Claude, where code_only is directionally costlier (+14.4%, not significant); a conditional-cost analysis localizes that gap to failure-cost on doomed-run trajectories, not a per-edit tax on successful runs. Two implications: the cheapest tool surface is jointly determined by task regime and agent design rather than by either axis alone, and the headline cost signal lives in cache-adjusted cost -- not pass rate, which is invariant across surfaces at the model sizes we evaluate. The benchmark harness, task suite, and analysis code are available at https://github.com/hyang0129/onlycodes.

Accepted at KDD Workshop on Agentic Software Engineering (SE 3.0); the linked arXiv record is the available paper version.

Current arXiv acceptance note and official non-archival workshop page


Durable execution and replay are treated as first-class loop infrastructure. Turns human-approved failures into durable policy: a coding agent revises a code-shaped sketch, regenerates code and prompt surfaces, preserves provenance, and must pass a regression gate before seeing the next case. In one captured run, the evolved-sketch rebuild passes 19 of 21 withheld cases versus 15 of 21 for replaying accepted examples; the single-model, single-order study does not establish general superiority.

Use Agentic Synthesis against Counterexample-Supplemented Sketches to choose an implementation surface for repeatable agent work.

Research source arXiv:2607.15854; inspect its method and evaluation before treating results as production evidence.

[2607.15854] Agentic Synthesis against Counterexample-Supplemented Sketches

Coding agents can fix a failing example without preserving the domain rule that made it fail. We present agentic synthesis against counterexample-supplemented sketches, a repository-native method for systems whose policy is discovered during implementation. A human starts with a partial sketch, and a coding agent compiles a replaceable projection. When simulation exposes missing or mistaken policy, an operator approves the corrected behavior and the minimum general rule the case authorizes. Every Developer call names its change authority and the rules, holes, anchors, and approved behavior that must survive. Conflict or ambiguous permission leaves the files unchanged and produces a clarification question. A complete archive preserves provenance; a curated regression set gates distinct boundaries. Before another candidate is revealed, the active case and curated regressions must pass both deterministic approved-output comparison and a separate review against the current sketch. Periodic clean regeneration tests whether the sketch carries the learned policy. We demonstrate the method with CatSynth, a captured synthetic application. In one open-world run with GPT-5.4-mini, 8 of 14 frozen candidates became counterexamples. Under the corrected protocol, replay-all, evolved-sketch rebuild, and retained Sketch-CE each passed all 8 accepted cases. They passed 14, 17, and 16 of 21 withheld cases, respectively. Sketch review rejected premature empty-input and tag policies and restored dropped anchors; adjudicated reviewer errors did not become policy. One model and one reveal order cannot establish general correctness or superiority. On this suite, the second check exposed drift hidden by deterministic replay, and the reviewed sketch passed three more withheld cases than raw example replay.

32 pages, 5 displayed figures (4 distinct screenshots). Includes the CatSynth artifact supplement. Code and captured experiment artifacts: https://github.com/open-horizon-labs/counterexample-supplemented-sketches Clarifies the two-check CESS method and Developer change authority; adds the protocol-correct CatSynth rerun and replaces the prior withheld-case headline


Verification is promoted from a final check to a loop-control signal. Defines a repository-hosted governance contract connecting contributor evidence with maintainer verification and decision authority; after an audit of 50 repositories, a 15-reviewer study improves exact risk-label recovery from 15/37 to 37/38 with the manifest-supported materials.

Use Making Agent-Mediated Contributions Governable: A Project-Level Governance Manifest for Open-Source AI Collaboration to choose an implementation surface for repeatable agent work.

Research source arXiv:2607.15769; inspect its method and evaluation before treating results as production evidence.

[2607.15769] Making Agent-Mediated Contributions Governable: A Project-Level Governance Manifest for Open-Source AI Collaboration

Generative AI and coding agents are intensifying a central governance tension in open-source software (OSS): they scale contribution generation faster than maintainers can assess risk, evidence, and accountability. Existing responses improve agent-readability and traceability, but project rules must also organize contribution-specific risk, evidence, accountability, and review-gate states. We theorize this organizational arrangement as project-side governability infrastructure. A diagnostic audit of 50 GitHub repositories finds widespread general governance artifacts, observable agent-readability, and fragmented AI-governance cues, but no project-wide arrangement that coordinates shared rules, preparation obligations, verification rights, and maintainer decision authority across AI-mediated contribution workflows. We develop the Agent Governance Manifest (AGM) as a repository-hosted boundary resource and bidirectional governance contract linking contributor-side evidence preparation with maintainer-side verification. In a controlled reviewer-side evaluation with 15 participants and 75 task-level outputs, AGM-supported materials improved exact risk-label recovery (37/38 vs. 15/37) and perceived review support (6.14 vs. 3.27 on a 1-7 scale). In a contributor-side feasibility check, 15 participants completed 45 tasks; all final packages represented the core governance state correctly, and 41 passed strict structural validation. The study develops a three-layer framework of agent-readability, traceability, and governability, theorizes agent-mediated contributions as governable boundary objects, and advances compliance-enabling digital innovation governance while preserving maintainer decision authority.

Jinjin Gao; Luyang Li; Shufen Guo; Ligang He; Xiaoning Sun


Ralph

Persistent memory is treated as an external runtime artifact. Geoffrey Huntley's original Ralph technique: run one agent in a bare loop with fresh context per iteration and the filesystem plus specs as memory.

Use Ralph to choose an implementation surface for repeatable agent work.

Operational pattern or playbook; signal comes from reusable loop structure and practical transferability.

How Ralph Wiggum went from 'The Simpsons' to the biggest name in AI right now - Venture Beat 😎Here's a cool little field report from a Y Combinator hackathon event where they put Ralph Wiggum to the test. "We Put a Coding Agent in a While Loop and It Shipped


everything is a ralph loop

Durable execution and replay are treated as first-class loop infrastructure. Follow-up essay arguing the loop, not the agent, is the durable engineering unit: one task per iteration, deterministic context, and verification inside the loop.

Use everything is a ralph loop to choose an implementation surface for repeatable agent work.

I’ve been thinking about how I build software is so very very different how I used to do it three years ago. No, I’m not talking about acceleration through usage of AI but instead at a more fundamental level of approach, techniques and best practices. Standard software practices


how-to-ralph-wiggum

Uses real automated software-engineering systems as evidence for practical loop architectures. Reference repository documenting the Ralph Wiggum technique end to end, from the bare loop script to guardrails and conventions.

Use how-to-ralph-wiggum to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (1,732 stars; 146 forks; updated 2026-08-06); popularity is context, not proof of reliability.

GitHub - ghuntley/how-to-ralph-wiggum: The Ralph Wiggum Technique—the AI development methodology that reduces software costs to less than a fast food worker's wage. · GitHub

The Ralph Wiggum Technique—the AI development methodology that reduces software costs to less than a fast food worker's wage. - ghuntley/how-to-ralph-wiggum


A Brief History of Ralph

Uses real automated software-engineering systems as evidence for practical loop architectures. Traces how the bare-loop technique spread from a provocation to a production practice among early adopters.

Use A Brief History of Ralph to choose an implementation surface for repeatable agent work.

Contextual source from www.humanlayer.dev; useful for practice signals or boundary conditions, not independent validation.

A Brief History of Ralph | HumanLayer Blog

The Ralph Wiggum Technique went viral in the last week of 2025. Here's the story of ralph since the first time I met Geoff in June of 2025.


Ralph Copilot

Persistent memory is treated as an external runtime artifact. Language-agnostic Ralph loop implementation using fresh context, filesystem memory, `PRD.md`, and `PROGRESS.md`.

Use Ralph Copilot to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (138 stars; 16 forks; MIT license; updated 2026-07-25); popularity is context, not proof of reliability.

GitHub - giocaizzi/ralph-copilot at e5b2813cc876c73a8c9d3398c0115da0d15f63cf · GitHub

Copilot implementation of Ralph loop. Contribute to giocaizzi/ralph-copilot development by creating an account on GitHub.


Ralph (snarktank)

State persistence is explicit enough for repeated runs and handoff. Ryan Carson's PRD-driven Ralph implementation that re-runs Amp or Claude Code with a fresh instance per iteration, gates each story on typecheck and tests, and persists state in prd.json, progress.txt, and Git history until every story passes.

Use Ralph (snarktank) to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (21,402 stars; 2,062 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - snarktank/ralph: Ralph is an autonomous AI agent loop that runs repeatedly until all PRD items are complete. · GitHub

Ralph is an autonomous AI agent loop that runs repeatedly until all PRD items are complete. - GitHub - snarktank/ralph: Ralph is an autonomous AI agent loop that runs repeatedly until all PRD items are complete.


ralph-claude-code

Uses real automated software-engineering systems as evidence for practical loop architectures. Loop runner that repeatedly re-executes Claude Code against project requirements, using dual-condition exit detection, rate limiting, and a circuit breaker to decide when the loop should stop.

Use ralph-claude-code to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (9,584 stars; 724 forks; MIT license; updated 2026-08-04); popularity is context, not proof of reliability.

GitHub - frankbria/ralph-claude-code: Autonomous AI development loop for Claude Code with intelligent exit detection · GitHub

Autonomous AI development loop for Claude Code with intelligent exit detection - frankbria/ralph-claude-code


ralph-orchestrator

Orchestration and control flow are made explicit and inspectable. Multi-backend implementation of the Ralph Wiggum technique that keeps a coding agent looping until task completion, using role-scoped hat personas that coordinate through events, with human-in-the-loop controls and a monitoring dashboard.

Use ralph-orchestrator to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (3,093 stars; 286 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - mikeyobrien/ralph-orchestrator: An improved implementation of the Ralph Wiggum technique for autonomous AI agent orchestration · GitHub

An improved implementation of the Ralph Wiggum technique for autonomous AI agent orchestration - mikeyobrien/ralph-orchestrator


ralphex

Uses real automated software-engineering systems as evidence for practical loop architectures. Extended Ralph loop runner that creates a Git branch per plan, executes tasks in fresh sessions with a commit after each, runs a multi-phase review pipeline with parallel review agents, and archives the completed plan.

Use ralphex to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (1,418 stars; 118 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - umputun/ralphex: Extended Ralph loop for autonomous AI-driven plan execution · GitHub

Extended Ralph loop for autonomous AI-driven plan execution - umputun/ralphex


ralph (iannuttall)

Persistent memory is treated as an external runtime artifact. File-based Ralph-style agent loop that executes one JSON PRD story per iteration with fresh model context, using Git and on-disk state as memory across Claude, Codex, Droid, and OpenCode backends.

Use ralph (iannuttall) to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (938 stars; 90 forks; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - iannuttall/ralph: A minimal, file‑based agent loop for autonomous coding. · GitHub

A minimal, file‑based agent loop for autonomous coding. - iannuttall/ralph


ralph-loop-agent

Verification is promoted from a final check to a loop-control signal. Vercel Labs implementation of the Ralph loop for the AI SDK: an outer loop re-runs the agent with verifier feedback until a verifyCompletion check passes or iteration, token, or cost stop conditions trigger.

Use ralph-loop-agent to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (823 stars; 85 forks; Apache-2.0 license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - vercel-labs/ralph-loop-agent: Continuous Autonomy for the AI SDK · GitHub

Continuous Autonomy for the AI SDK. Contribute to vercel-labs/ralph-loop-agent development by creating an account on GitHub.


Open Ralph Wiggum

Context is managed as durable loop state rather than a single prompt payload. Agent-agnostic CLI that runs the Ralph Wiggum loop by feeding the same prompt to a fresh agent instance each iteration, with task tracking, live status monitoring, and mid-loop context injection across six coding-agent backends.

Use Open Ralph Wiggum to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (1,873 stars; 142 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - Th0rgal/open-ralph-wiggum: Type `ralph "prompt"` to start open code in a ralph loop. Also supports a prompt file & status check. Open Code, Claude Code, Codex, Copilot · GitHub

Type `ralph "prompt"` to start open code in a ralph loop. Also supports a prompt file & status check. Open Code, Claude Code, Codex, Copilot - Th0rgal/open-ralph-wiggum


Compound Engineering

Persistent memory is treated as an external runtime artifact. Every's named plan-work-review-compound loop, where each run feeds lessons back into `AGENTS.md`-style memory so the next loop is easier; the self-improving counterpart to Ralph.

Use Compound Engineering to choose an implementation surface for repeatable agent work.


Gas Town

The work separates roles across agents, verifiers, or orchestration layers. Steve Yegge's multi-agent orchestrator that runs 20-30 parallel coding agents with coordinator, worker, and merge-queue roles; the structured-orchestration end of the spectrum that Ralph anchors with bare iteration.

Use Gas Town to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (17,491 stars; 1,603 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - gastownhall/gastown: Gas Town - multi-agent workspace manager · GitHub

Gas Town - multi-agent workspace manager. Contribute to gastownhall/gastown development by creating an account on GitHub.


Amp

The work separates roles across agents, verifiers, or orchestration layers. Agentic coding tool built around threads, subagents, and an opinionated harness, with an owner's manual that documents loop-style operating practices.

Use Amp to choose an implementation surface for repeatable agent work.

Working implementation, framework, runtime, or repository; signal comes from usable code and ecosystem adoption.

Amp is the frontier agent that lets you wield the full power of leading models.


karl

The work separates roles across agents, verifiers, or orchestration layers. Autonomous multi-agent development loop with planner, reviewer, architect, tester, developer, deployment, and retry phases.

Use karl to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (0 stars; 0 forks; MIT license; updated 2026-04-08); popularity is context, not proof of reliability.

GitHub - kayoslab/karl: Autonomous multi-agent development loop · GitHub

Autonomous multi-agent development loop. Contribute to kayoslab/karl development by creating an account on GitHub.


joelclaw agent-loop skill

Durable execution and replay are treated as first-class loop infrastructure. Durable Planner-Implementor-Reviewer-Judge coding loops via Inngest events and progress files.

Use joelclaw agent-loop skill to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (61 stars; 3 forks; updated 2026-08-07); popularity is context, not proof of reliability.

joelclaw/skills/agent-loop/SKILL.md at main · joelhooks/joelclaw · GitHub

Personal AI operating system — blog, architecture decisions, and the journey from zero to a composable agent system. - joelclaw/skills/agent-loop/SKILL.md at main · joelhooks/joelclaw


ARIS (Auto-Research-In-Sleep)

Verification is promoted from a final check to a loop-control signal. Markdown-only skills that run autonomous overnight ML research loops on Claude Code, Codex, or other LLM agents, iterating idea discovery and experiments with cross-model review as the verification gate.

Use ARIS (Auto-Research-In-Sleep) to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (14,363 stars; 1,273 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - wanshuiyin/Auto-claude-code-research-in-sleep: ARIS ⚔ (Auto-Research-In-Sleep) — Lightweight Markdown-only skills for autonomous ML research: cross-model review loops, idea discovery, and experiment automation. No framework, no lock-in — works with Claude Code, Codex, OpenClaw, or any LLM agent. · GitHub

ARIS ⚔ (Auto-Research-In-Sleep) — Lightweight Markdown-only skills for autonomous ML research: cross-model review loops, idea discovery, and experiment automation. No framework, no lock-in — works with Claude Code, Codex, OpenClaw, or any LLM agent. - wanshuiyin/Auto-claude-code-research-in-sleep


AutoAgent

The work turns loop quality into a measurable task or score. Meta-agent that autonomously edits its own harness (system prompt, tools, orchestration), re-runs the benchmark, and keeps or discards each change by score, with an author-reported top SpreadsheetBench result from a 24-hour unattended run.

Use AutoAgent to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (4,561 stars; 498 forks; updated 2026-08-06); popularity is context, not proof of reliability.

GitHub - kevinrgu/autoagent: autonomous harness engineering · GitHub

autonomous harness engineering. Contribute to kevinrgu/autoagent development by creating an account on GitHub.


zeroshot

Verification is promoted from a final check to a loop-control signal. CLI that runs a planner, an implementer, and independent validators in isolated environments, looping until a change is verified or rejected with reproducible failures.

Use zeroshot to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (1,695 stars; 149 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - the-open-engine/zeroshot: Independent executor–verifier orchestration for software changes. · GitHub

Independent executor–verifier orchestration for software changes. - the-open-engine/zeroshot


Loki Mode

Uses real automated software-engineering systems as evidence for practical loop architectures. Autonomous spec-to-app loop that runs Reason-Act-Reflect-Verify cycles behind quality gates, with completion gated by a blind three-reviewer council and a deterministic evidence receipt that rejects empty diffs and failing tests.

Use Loki Mode to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (1,031 stars; 201 forks; NOASSERTION license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - asklokesh/loki-mode: Multi-agent autonomous SDLC framework. Spec to deployed app. PRD, GitHub issue, OpenAPI/JSON/YAML, or one-line brief. 5 AI providers, 8 quality gates. · GitHub

Multi-agent autonomous SDLC framework. Spec to deployed app. PRD, GitHub issue, OpenAPI/JSON/YAML, or one-line brief. 5 AI providers, 8 quality gates. - asklokesh/loki-mode


Looper

Verification is promoted from a final check to a loop-control signal. Claude Code skill for designing review-gated agent loops before running them, coaching the user into a portable loop.yaml spec with explicit goals, typed verification, iteration caps, and budget limits, then emitting artifacts runnable in-session or via an external Python runner.

Use Looper to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (700 stars; 64 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - ksimback/looper: Design visual, review-gated agent loops for Claude Code before you run them. · GitHub

Design visual, review-gated agent loops for Claude Code before you run them. - ksimback/looper


Agent Apprenticeship

Packages the evidence as queryable CSV and JSONL rather than only a rendered page. Multi-backend ecosystem where apprentice agents complete tasks through workflow loops, mentors or humans verify results, and execution traces are compiled into a published dataset that feeds future agent improvement.

Use Agent Apprenticeship to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (1,342 stars; 58 forks; MIT license; updated 2026-08-06); popularity is context, not proof of reliability.

GitHub - ray-r-ren/agent-apprenticeship: The living ecosystem where AI agents complete tasks through workflow loops, improve through iterative execution, are evaluated by mentor agents or humans in the loop, and turn completed work into reusable work experience and data to improve future agents. · GitHub

The living ecosystem where AI agents complete tasks through workflow loops, improve through iterative execution, are evaluated by mentor agents or humans in the loop, and turn completed work into reusable work experience and data to improve future agents. - ray-r-ren/agent-apprenticeship


Scholar Loop

The work separates roles across agents, verifiers, or orchestration layers. Autonomous multi-agent research loop from literature to hypothesis to real ML experiments to write-up, scoring every checkable agent claim against frozen ground-truth metrics and shipping an adversarial cheater engine that probes the loop for reward-hacking gaps.

Use Scholar Loop to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (466 stars; 35 forks; MIT license; updated 2026-08-03); popularity is context, not proof of reliability.

GitHub - renee-jia/scholar-loop: An autonomous AI scientist: a multi-agent loop over literature, experiments, self-critique and write-up, with deterministic guards against reward-hacking and hallucination. · GitHub

An autonomous AI scientist: a multi-agent loop over literature, experiments, self-critique and write-up, with deterministic guards against reward-hacking and hallucination. - renee-jia/scholar-loop


loop-engineering (Cobus Greyling)

Workspace isolation is part of the loop design, not an afterthought. Patterns-and-tooling repo shipping seven npm CLIs (loop-init, loop-audit, loop-cost, loop-sync, loop-context, loop-mcp-server, loop-worktree), starter kits, and production loop patterns; scaffolds skills/state/budget files, scores a repo's "Loop Ready" readiness, detects state drift, and estimates token spend per cadence for Claude Code, Codex, OpenCode, and Grok loops.

Use loop-engineering (Cobus Greyling) to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (9,944 stars; 1,343 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - cobusgreyling/loop-engineering: Practical patterns, starters & CLI tools for loop engineering with AI coding agents. Design systems that prompt and orchestrate agents (inspired by Addy Osmani and Boris Cherny). Includes loop-audit, loop-init, loop-cost. · GitHub

Practical patterns, starters & CLI tools for loop engineering with AI coding agents. Design systems that prompt and orchestrate agents (inspired by Addy Osmani and Boris Cherny). Includes loop-audit, loop-init, loop-cost. - cobusgreyling/loop-engineering


AutoCVE

Verification is promoted from a final check to a loop-control signal. Open-source agent-driven CVE discovery platform whose orchestrator coordinates Recon, Scan, Triage, Finding, and Verification agents through ReAct loops with correction nudges and structured FinalizeFinding termination, running the full discover, source-audit, dynamic-verify, dedup, and report loop on a self-hosted FastAPI/React/PostgreSQL stack with agent-tree observability.

Use AutoCVE to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (1,281 stars; 94 forks; AGPL-3.0 license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - larlarua/AutoCVE: Agent-driven automated CVE discovery platform for source code auditing, vulnerability verification, and report generation. · GitHub

Agent-driven automated CVE discovery platform for source code auditing, vulnerability verification, and report generation. - larlarua/AutoCVE


LoongFlow (Baidu)

Persistent memory is treated as an external runtime artifact. Baidu's open-source agent framework built explicitly for Loop Engineering: a Plan-Execute-Summary loop with structured experiential memory lets agents plan, execute, reflect, and evolve across software-engineering, math, and ML tasks (Apache-2.0, on PyPI, paper: arXiv 2512.24077).

Use LoongFlow (Baidu) to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (468 stars; 53 forks; Apache-2.0 license; updated 2026-08-05); popularity is context, not proof of reliability.

GitHub - baidu-baige/LoongFlow: LoongFlow is an expert-grade Agent framework for Loop Engineering. Through a Plan-Execute-Summary loop and structured experiential memory, it enables AI to continuously think, execute, reflect, and evolve across complex software engineering, mathematical, and machine learning tasks. · GitHub

LoongFlow is an expert-grade Agent framework for Loop Engineering. Through a Plan-Execute-Summary loop and structured experiential memory, it enables AI to continuously think, execute, reflect, and evolve across complex software engineering, mathematical, and machine learning tasks. - baidu-baige/LoongFlow


cc10x

Durable execution and replay are treated as first-class loop infrastructure. Claude Code plugin that routes work through one router, nine specialist agents, sixteen skills, and four workflows, enforcing fail-closed verification and test-honesty gates and writing each workflow's intent, evidence, and verdicts to durable .cc10x/ disk artifacts so resume and review survive context compaction.

Use cc10x to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (161 stars; 25 forks; MIT license; updated 2026-08-04); popularity is context, not proof of reliability.

GitHub - romiluz13/cc10x: The Loop Engine for Claude Code — engineer the loop, not the prompt. 1 router · 9 agents · 16 skills · 4 workflows. Fail-closed gates, test honesty, anti-anchored review. · GitHub

The Loop Engine for Claude Code — engineer the loop, not the prompt. 1 router · 9 agents · 16 skills · 4 workflows. Fail-closed gates, test honesty, anti-anchored review. - romiluz13/cc10x


RigorLoop

Uses real automated software-engineering systems as evidence for practical loop architectures. Statistically-grounded loop-engineering framework in which a strategy agent directs concurrent executor agents that iteratively build and refine a solution against gold-standard examples.

Use RigorLoop to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (93 stars; 0 forks; MIT license; updated 2026-08-05); popularity is context, not proof of reliability.

GitHub - ronikobrosly/RigorLoop: A statistically-sound agentic build framework that employs agentic loops to create code artifacts (whether a script, a skill markdown file, etc). Crucially, it splits verification data into the classic data science-like dev, validation, and final test sets to avoid overfitting. · GitHub

A statistically-sound agentic build framework that employs agentic loops to create code artifacts (whether a script, a skill markdown file, etc). Crucially, it splits verification data into the classic data science-like dev, validation, and final test sets to avoid overfitting. - GitHub - ronikobrosly/RigorLoop: A statistically-sound agentic build framework that employs agentic loops to create code artifacts (whether a script, a skill markdown file, etc). Crucially, it splits verification data into the classic data science-like dev, validation, and final test sets to avoid overfitting.


Open-Inspect

Execution isolation and permission boundaries are part of the design. Open-source background coding-agent system inspired by Ramp's Inspect: hosted agents in full dev-environment sandboxes, reachable from a web UI, Slack, GitHub PRs, Linear, or webhooks.

Use Open-Inspect to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (2,637 stars; 377 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - ColeMurray/background-agents: An open-source background agents coding system · GitHub

An open-source background agents coding system. Contribute to ColeMurray/background-agents development by creating an account on GitHub.


T3MP3ST

The work separates roles across agents, verifiers, or orchestration layers. Multi-agent offensive-security meta-harness that turns an existing coding agent into an autonomous vulnerability-research loop, with a verify-claims receipt step that separates confirmed findings from speculation.

Use T3MP3ST to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (5,460 stars; 1,133 forks; AGPL-3.0 license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - elder-plinius/T3MP3ST: autonomous red teaming platform; multi-agent offensive-security meta-harness · GitHub

autonomous red teaming platform; multi-agent offensive-security meta-harness - elder-plinius/T3MP3ST


Loom

Durable execution and replay are treated as first-class loop infrastructure. Open-source delivery harness for existing coding agents that treats delivery as a durable loop: route, execute, verify, record evidence, repair, and continue from saved state.

Use Loom to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (751 stars; 88 forks; Apache-2.0 license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - valkor-ai/loom: Loop engineering for agentic software delivery. · GitHub

Loop engineering for agentic software delivery. Contribute to valkor-ai/loom development by creating an account on GitHub.


Inferoa

Uses real automated software-engineering systems as evidence for practical loop architectures. Inference-native agent harness for loop engineering that treats every loop as an inference workload, shaping each turn to preserve cacheable prefixes and bound stale evidence.

Use Inferoa to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (488 stars; 85 forks; Apache-2.0 license; updated 2026-08-05); popularity is context, not proof of reliability.

GitHub - agentic-in/inferoa: Inference-native Tokenmaxxing Agent Harness for Loop Engineering · GitHub

Inference-native Tokenmaxxing Agent Harness for Loop Engineering - agentic-in/inferoa


PlanWeave

Control flow is represented as an inspectable graph rather than an opaque prompt loop. File-backed loop-engineering system for long-running coding agents that turns fuzzy plans into a claimable task graph of nodes and block documents routed through implementation and review.

Use PlanWeave to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (319 stars; 19 forks; MIT license; updated 2026-08-06); popularity is context, not proof of reliability.

GitHub - GaosCode/PlanWeave: PlanWeave is a file-backed loop engineering system for long-running coding agents. It turns fuzzy plans into claimable tasks, routes them through implementation and review agents, records every run, and keeps the loop recoverable. · GitHub

PlanWeave is a file-backed loop engineering system for long-running coding agents. It turns fuzzy plans into claimable tasks, routes them through implementation and review agents, records every run, and keeps the loop recoverable. - GaosCode/PlanWeave


loop.js

Verification is promoted from a final check to a loop-control signal. TypeScript loop-engineering framework that runs an agent in rounds against a stated goal until a skeptical, read-only verifier agent accepts the result or a budget is exhausted.

Use loop.js to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (149 stars; 2 forks; Apache-2.0 license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - loop-js/loop.js: A loop engineering framework — state a Goal; Rounds run until a skeptical, read-only Verify agent settles it. · GitHub

A loop engineering framework — state a Goal; Rounds run until a skeptical, read-only Verify agent settles it. - loop-js/loop.js


ai-trains-ai

Uses real automated software-engineering systems as evidence for practical loop architectures. Recursive training loop where a trainer agent autonomously writes complete reinforcement-learning jobs (environments, rewards, configs), runs them, and iterates on the results.

Use ai-trains-ai to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (232 stars; 18 forks; MIT license; updated 2026-08-06); popularity is context, not proof of reliability.

GitHub - Danau5tin/ai-trains-ai: RL-training an AI agent to RL-train AI agents. · GitHub

RL-training an AI agent to RL-train AI agents. Contribute to Danau5tin/ai-trains-ai development by creating an account on GitHub.


Persistent memory is treated as an external runtime artifact. Factory's software-factory pattern, where Automations coordinate recurring workflows with shared objectives and memory, Missions run multi-agent execution over hours or days, and Droid Computers give agents persistent remote execution across the SDLC.

Use Factory 2.0: From Coding Agents to Software Factories to choose an implementation surface for repeatable agent work.

Contextual source from factory.ai; useful for practice signals or boundary conditions, not independent validation.

Factory 2.0: From coding agents to software factories | Factory.ai Factory.ai Logo Arrow Right Icon

In 2023, we launched Factory with the mission to bring autonomy to software engineering. While others were using models...


Superpowers 6

Evaluation data is used as the feedback signal for improving loop behavior. Release notes doubling as a case study of an unattended overnight autoresearch loop that ran 25 harness experiments against the project's own eval suite, roughly halving orchestration runtime and cutting token spend about 60%.

Use Superpowers 6 to choose an implementation surface for repeatable agent work.

Contextual source from blog.fsck.com; useful for practice signals or boundary conditions, not independent validation.

Superpowers 6 — Massively Parallel Procrastination

I'm Jesse. I make stuff. Software, hardware. Very occasionally, trouble.


Introducing Devin Security Swarm

The trigger or cadence is explicit, making the workflow recurring rather than one-off. Cognition's agent swarm runs a continuous discover-verify-fix security loop: parallel agents hunt vulnerabilities, reproduce each in an isolated sandbox to confirm exploitability before reporting, and open remediation PRs, re-running on a schedule after the backlog clears.

Use Introducing Devin Security Swarm to choose an implementation surface for repeatable agent work.

Contextual source from cognition.com; useful for practice signals or boundary conditions, not independent validation.

Introducing Devin Security Swarm | Cognition

Devin Security Swarm finds vulnerabilities across the codebase, validates exploitability at runtime, and ships remediation PRs.


Towards Self-Driving Codebases

Uses real automated software-engineering systems as evidence for practical loop architectures. Cursor research on running thousands of coding agents as a recursive planner-subplanner-worker hierarchy sustaining roughly 1,000 commits per hour, finding that tolerating small error rates that peer agents later fix beats enforcing per-step correctness.

Use Towards Self-Driving Codebases to choose an implementation surface for repeatable agent work.

Contextual source from cursor.com; useful for practice signals or boundary conditions, not independent validation.

We're making a part of our multi-agent research harness available to try today in preview.


Factory: Incident Response Automation

State persistence is explicit enough for repeated runs and handoff. Factory's July 10, 2026 launch where a Droid triggered by Slack alerts (Sentry, Datadog, Rootly, Axiom) autonomously investigates each incident on a dedicated computer, triages, prepares fixes, and reports back in the thread, recording what it learns in a persistent runbook that Factory says improves its incident response over time.

Use Factory: Incident Response Automation to choose an implementation surface for repeatable agent work.

Incident Response | Factory.ai Factory.ai Logo Arrow Right Icon

On-call alerts have always been stomach-dropping moments. Someone's dinner, weekend, or launch review gets hijacked for ...


The work separates roles across agents, verifiers, or orchestration layers. Matt Shumer's demonstration of a single-prompt run in which a frontier model worked autonomously for almost a week with subagent fan-out to build a navigable voxel Manhattan.

Use A Week-Long Autonomous Voxel Manhattan Build to choose an implementation surface for repeatable agent work.

Contextual source from x.com; useful for practice signals or boundary conditions, not independent validation.

Matt Shumer on X: "GPT-5.6-Sol one-shotted this voxel-based Manhattan. Just look at the precision... it's insane. It ran for almost a week, completely autonomously, to get the job done. https://t.co/LZgthaBnqL" / X


Uses real automated software-engineering systems as evidence for practical loop architectures. Converts accepted review feedback into versioned behavioral rules that future coding sessions can apply; across 11 reported production sessions, error classes covered by a rule did not recur, a promising but deliberately small-scope result.

Use Self-Improving AI Coding Agents Through Accumulated Behavioral Rules: A Closed-Loop Framework to choose an implementation surface for repeatable agent work.

Research source arXiv:2607.13091; inspect its method and evaluation before treating results as production evidence.

[2607.13091] Self-Improving AI Coding Agents Through Accumulated Behavioral Rules: A Closed-Loop Framework

LLM-based coding agents repeat the same classes of mistakes across sessions because they lack a mechanism to retain corrections from human review feedback. We present a closed-loop framework in which every accepted review comment is codified as a persistent behavioral rule, progressively expanding the set of error classes the agent can self-detect. The framework combines an accumulating rule set in a version-controlled instruction file, a self-review checklist executed before code submission, and automated validation that ensures rule set integrity as it grows. In deployment across a 35+ service microservices platform, the rule set grew from 5 to 18 behavioral rules, 15+ language-specific standards, and a 15-item self-review checklist, all derived from real review feedback. We present empirical results from 11 recorded working sessions spanning code generation, PR review, incident investigation, and cross service refactoring. We observe that accumulated rules shift review effort from low-level correctness toward design-level validation, achieve a measured 0% recurrence rate for ruled-against error classes, and transfer across heterogeneous agent interfaces. We compare our approach against related work in experiential LLM learning (Reflexion, ExpeL, Voyager) and automated code review (CodeReviewer, SWE-bench agents), showing that our framework achieves persistent cross-session learning without weight updates, operates on production codebases rather than synthetic benchmarks, and addresses an orthogonal dimension (behavioral consistency over time) that existing benchmarks do not measure. The result is a coding agent that improves with every review cycle, accumulating the engineering wisdom of its human collaborators without changing a single model weight.

Accepted at 32nd IEEE International Conference on Engineering Technology and Innovation (ICE/ITMC); the linked arXiv record is the available paper version.

Current arXiv acceptance note and official conference page


Webwright

Durable execution and replay are treated as first-class loop infrastructure. Treats workspace code, screenshots, and run artifacts as durable state while browsers remain disposable, producing a readable write-run-inspect-repair loop for long-horizon web tasks.

Use Webwright to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (5,893 stars; 372 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - microsoft/Webwright: A simple SWE style browser agent framework that achieves SOTA results on long horizon web tasks. · GitHub

A simple SWE style browser agent framework that achieves SOTA results on long horizon web tasks. - GitHub - microsoft/Webwright: A simple SWE style browser agent framework that achieves SOTA results on long horizon web tasks.


Verification is promoted from a final check to a loop-control signal. Geoffrey Huntley, originator of the Ralph technique, on porting entire codebases by pointing a looped coding agent at the source and target and letting verified iterations do the work.

Use Porting Software Has Been Trivial for a While Now to choose an implementation surface for repeatable agent work.

Contextual source from ghuntley.com; useful for practice signals or boundary conditions, not independent validation.

porting software has been trivial for a while now. here’s how you do it.

This one is short and sweet. if you want to port a codebase from one language to another here’s the approach: 1. Run a ralph loop which compresses all tests into /specs/.md which looks similar to “study every file in tests/* using separate subagents and document in /specs/


DeerFlow

The work targets tasks that exceed a single context window or prompt session. ByteDance's open-source long-horizon SuperAgent harness that researches, codes, and creates across extended multi-step runs, one of the most widely adopted open agent harnesses.

Use DeerFlow to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (79,495 stars; 10,860 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - bytedance/deer-flow: An open-source long-horizon SuperAgent harness that researches, codes, and creates. With the help of sandboxes, memories, tools, skill, subagents and message gateway, it handles different levels of tasks that could take minutes to hours. · GitHub

An open-source long-horizon SuperAgent harness that researches, codes, and creates. With the help of sandboxes, memories, tools, skill, subagents and message gateway, it handles different levels of tasks that could take minutes to hours. - bytedance/deer-flow


Grok Build

Primary-source operational guidance rather than commentary. xAI's official agentic coding CLI that plans, edits, and verifies changes in a repository, joining the major vendor coding-agent runtimes.

Use Grok Build to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (24,352 stars; 4,626 forks; Apache-2.0 license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - xai-org/grok-build: SpaceXAI's coding agent harness and TUI. Fullscreen, mouse interactive, extensible. · GitHub

SpaceXAI's coding agent harness and TUI. Fullscreen, mouse interactive, extensible. - xai-org/grok-build


BlitzOS

Context is managed as durable loop state rather than a single prompt payload. Open-source pattern for booting cloud agents pre-loaded with your work context so recurring sessions resume instantly instead of cold-starting.

Use BlitzOS to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (174 stars; 20 forks; MIT license; updated 2026-08-06); popularity is context, not proof of reliability.

GitHub - blitzdotdev/blitzos: BlitzOS allows cloud agents to boot already knowing your work, and keep working with your laptop closed · GitHub

BlitzOS allows cloud agents to boot already knowing your work, and keep working with your laptop closed - blitzdotdev/blitzos


Orchestration and control flow are made explicit and inspectable. Field report on running twenty parallel Codex harnesses against open Erdős problems, a live demonstration of fleet-scale loop orchestration applied to mathematics research.

Use Star Fleet Math: Solving Erdős Problems with 20 Parallel Codex Harnesses to choose an implementation surface for repeatable agent work.

Contextual source from www.starfleetmath.com; useful for practice signals or boundary conditions, not independent validation.

A Mac app controlling up to 20 starship agentic harnesses running GPT-5.6 Sol max in parallel on open mathematics problems.


Finn-loop

Uses real automated software-engineering systems as evidence for practical loop architectures. Three Claude Code skills from Alex Finn that turn Linear + GitHub into a small, human-gated software factory: /finn-spec interviews you until the behavior is unambiguous and files a Linear issue with acceptance criteria and non-goals, /finn-build claims the next agent-ready issue and opens a PR (run repeatedly via the /loop primitive), and /finn-review posts loop-approved or changes-requested verdicts against required CI checks, one approval label, one rule: humans merge.

Use Finn-loop to choose an implementation surface for repeatable agent work.

Inspectable GitHub source (293 stars; 53 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - finna/Finn-loop: The Finn-loop: a 3-skill AI software factory for Claude Code — spec, build, review. Humans merge. · GitHub

The Finn-loop: a 3-skill AI software factory for Claude Code — spec, build, review. Humans merge. - finna/Finn-loop


Packages the evidence as queryable CSV and JSONL rather than only a rendered page. Mazloomzadeh, Morovati, and Khomh run a longitudinal analysis over the AIDev dataset covering 9,428 agentic PRs from five agents across 489 Python repositories.

Use How Do AI Coding Agents Contribute to Software Development? An Empirical Study of Agentic Pull Requests to choose an implementation surface for repeatable agent work.

Research source arXiv:2607.21832; inspect its method and evaluation before treating results as production evidence.

[2607.21832] How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests

Recent advances in large language models and their rapid adoption across software engineering tasks have made Artificial Intelligence (AI) coding agents an integral component of modern software development workflows. While developers increasingly benefit from these coding agents, their impact on software quality remains insufficiently understood. In particular, how agentic contributions evolve across the software development lifecycle has not been thoroughly investigated. This study aims to characterize agentic pull requests (PR) in comparison to human generated PRs and to examine how their properties change across different stages of the development lifecycle. Using the AIDev dataset, we first analyze how differences in merge rates between agentic and human generated PRs vary over time. We then identify the types of development tasks where AI coding agents are predominantly applied and investigate how these task distributions evolve across development quarters. Finally, we compare a set of key characteristics of agentic and human generated PRs, focusing on their implications for software quality and their temporal dynamics. Overall, our findings provide an empirical and longitudinal perspective on the role of AI coding agents in software development, offering a more nuanced understanding of their benefits and limitations in real-world practices.

Iren Mazloomzadeh; Mohammad Mehdi Morovati; Foutse Khomh