Abstract
Coding agents often find correct patches early but continue working, sometimes undoing correct solutions, with termination lag varying by model and benchmark.
Coding agents can solve a task well before they stop working on it. We characterize termination lag--the number of steps between an agent's first correct patch and the end of its run--across five models, two agent frameworks, and three SWE-bench datasets by recording and testing every distinct repository patch generated during execution. Our analysis reveals that 38--82% of successful runs achieve a correct solution on their initial source-code edit. However, agents continue to expend substantial effort after correct patches, with post-solution steps accounting for a median of 35--37% for Qwen3-Coder and 54--60% for Laguna-XS.2 in successful runs on SWE-bench Verified and Multilingual. Agents spend most of these additional steps verifying their work and exploring the repository. In a small fraction of cases (1--8%), this continued activity leads agents to undo their earlier correct patch, leaving the final solution incorrect.
Community
What if coding agents already solved the problem but don't know when to stop?
We studied 5 LLMs, 2 agent frameworks, and 3 SWE-bench datasets, tracking and evaluating intermediate code patches throughout agent execution.
๐ Key findings:
- 38โ82% of runs that reach a correct patch get it right on their first source-code edit.
- A median of 35โ60% of steps in successful runs occur AFTER the correct patch is produced in our core settings.
- Agents spend most of these extra steps verifying and exploring.
- In 1โ8% of cases, agents even break an already-correct solution and finish with an incorrect patch.
These findings suggest an important opportunity: making coding agents recognize when to stop could significantly reduce unnecessary computation.
๐ฌ How should coding agents decide when a solution is good enough to submit?
Github:
https://github.com/musacim/termination-lag
HF:
https://huggingface.co/datasets/musacim/termination-lag-trajectories
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- PAUSE: Search-Free Steering of Coding Agents Before Their First Source-Code Edit (2026)
- StopLoss: Teaching Agents to Predict Failure and Stop Early (2026)
- WideSWE: Can Coding Agents Coordinate Changes Across Repositories? (2026)
- Knowing When to Stop: The Evidence-Action Gap in Coding Agents (2026)
- Before They Can Solve: Predicting Post-Training Coding-Agent Performance from Base Models (2026)
- SWEPatchTest: Evaluating Software Tests Beyond Reproduction (2026)
- Analyzing and Mitigating Cost-Inefficient Behaviors in Coding Agents (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Thanks for separating time-to-first-correct-patch from the end of the run. Recording and testing every distinct repository patch makes the lag visible, instead of folding it into a single pass/fail at termination.
Two numbers in the abstract are especially useful together: 38โ82% of successful runs are already correct on the first source-code edit, yet post-solution steps still account for a median 35โ37% (Qwen3-Coder) and 54โ60% (Laguna-XS.2) of those runs on SWE-bench Verified and Multilingual. Most of that lag is spent verifying and exploring the repository, while a small fraction (1โ8%) undo an earlier correct patch.
A question I would like to understand better (I have not run experiments): if a stop rule accepted the first patch that passed the agent's own checks, how would that change (i) wasted post-solution steps and (ii) the 1โ8% regression rate? My concern is that the verification steps may be what keeps the regression rate small, so first-correct is not automatically a free saving. The limitation that each task is run once also makes it hard to tell how stable the lag and undo rates are.
I would be glad to hear if you already measured a stop-on-first-self-test policy, or whether the recorded traces would support that counterfactual without new rollouts.
โ Jack Tan (@Jacktan007 )
Finding the right patch early and then undoing it is the crew that keeps sanding after the finish is already good. Knowing when to stop is a skill of its own. On a job the punch list tells you you're done. I'd be curious whether a cheap check after each patch would cut most of that lag.
Get this paper in your agent:
hf papers read 2611.14960 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper