Treats feedback, telemetry, and deterministic artifacts as loop-control gates. Leonardo de Moura, Lean's creator, dissects a kernel soundness bug surfaced when Ramana Kumar published an AI-assisted "disproof" of the Collatz conjecture on Jul 25; Kiran Gopinathan reduced it to a minimal exploit on Jul 28 and it was patched within an hour.

Use Postmortem for Kernel Soundness Bug #14576 to measure progress and gate completion with repeatable evidence.

Contextual source from leodemoura.github.io; useful for practice signals or boundary conditions, not independent validation.

Postmortem for Kernel Soundness Bug #14576 — Leonardo de Moura

Leonardo de Moura — Creator of Lean and Z3


ratchet

Treats feedback, telemetry, and deterministic artifacts as loop-control gates. A PostToolUse-hook harness that stops trusting the agent to police itself against its own rules file and instead measures every edit as it lands, reporting violations back into the same live session so the agent can revise before the turn ends.

Use ratchet to measure progress and gate completion with repeatable evidence.

Inspectable GitHub source (438 stars; 85 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

Gate progress with tests, evals, and evidence.

GitHub - 0xwilliamortiz/ratchet: Your agent reads the rules. This checks whether it followed them. · GitHub

Your agent reads the rules. This checks whether it followed them. - 0xwilliamortiz/ratchet


Verification is promoted from a final check to a loop-control signal. A methodological negative result aimed squarely at teams building detector-based verification gates. The claim: a high probing AUC on a clean-vs-attack split is not, on its own, evidence that the probe detects malicious content, it may simply be recognizing surface artifacts of how the attack was constructed.

Use When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents to measure progress and gate completion with repeatable evidence.

Research source arXiv:2606.22864; inspect its method and evaluation before treating results as production evidence.

[2606.22864] When AUC 0.998 Is Not Enough: A Candidate Evaluation Protocol for Hidden-State Probes of Indirect Prompt Injection in Multimodal Computer-Use Agents

Hidden-state probing -- a linear classifier on a frozen vision-language model's internal activations -- has emerged as an attractive evaluation tool for flagging indirect prompt injection (IPI) in multimodal computer-use agents before the agent emits a corrupted action. We argue, on a single-backbone cautionary case study (Qwen2.5-VL-7B on Mind2Web, teacher-forced replay), that a high probing AUC on a clean-vs-attack split is not, on its own, evidence of malicious-content detection. Two post-hoc diagnostics -- a paired-construction scalar baseline on text-side injections, and same-step nuisance-matched visual controls on the overlay surface -- do not license an unqualified malicious-content interpretation of the headline while leaving room for partly-semantic readings. We package the diagnostics as a candidate control set with reporting heuristics for what a high clean-vs-attack AUC does and does not license. Labels are injection-surface-present, not attack success; generalisation beyond this backbone and benchmark is a conjecture.

17 pages, 3 figures. Camera-ready version for EvalMG '26, The 2nd Workshop on Evaluation for Multimodal Generation, co-located with SIGIR 2026


Verification is promoted from a final check to a loop-control signal. Closes the loop between a verifier's rejection and the agent's skill library: verification failures are attributed to specific skill deficiencies and distilled into reusable lessons, so the next attempt is informed by a diagnosis rather than a raw error dump.

Use VeriSkill: A Self-Evolution Framework for Program Verification Skills to measure progress and gate completion with repeatable evidence.

Research source arXiv:2607.27733; inspect its method and evaluation before treating results as production evidence.

[2607.27733] VeriSkill: A Self-Evolution Framework for Program Verification Skills

Automating program verification with LLM agents requires generating specifications, annotations, auxiliary lemmas, and tool invocations, all of which depend on reusable skills. A natural remedy is skill self-evolution: distilling skills from trajectories and refining them through feedback. However, existing evolution methods struggle with program verification tasks because they cannot reliably identify skill-specific failures or extract actionable signals from opaque verifier feedback. In this paper, we propose VeriSkill, a self-evolution framework built for program verification. It attributes verification failures to skill deficiencies, distills diagnostic signatures into reusable lessons, and iteratively refines candidate skills, admitting only revisions that improve verification performance while preserving program semantics. Experiments show that VeriSkill consistently outperforms all baselines across multiple verification tools, agent frameworks, and LLM backends.

Changguo Jia; Tianqi Zhao; Zhiyou Xiao; Weiming Zhang; Minghui Zhou


Verification is promoted from a final check to a loop-control signal. Models the agent trajectory as a provenance-constrained state machine: tool outputs normalize into a ledger and every reasoning claim must cite a ledger entry, verified at entity and numeric level.

Use LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger to measure progress and gate completion with repeatable evidence.

Research source arXiv:2607.28374; inspect its method and evaluation before treating results as production evidence.

[2607.28374] LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger

Multimodal agents for visual question answering increasingly operate as multi-step trajectories that interleave perception, retrieval, and reasoning, yet evaluation still largely reduces to final-answer accuracy. This aggregate signal cannot tell whether a correct answer was reached through grounded evidence, language priors, or accidental error cancellation. We propose to treat a multimodal agent trajectory as a provenance-constrained state machine: tool outputs are normalized into a Structured Evidence Ledger that serves as the trajectory state, downstream reasoning and decision claims may cite only active ledger entries, grounding is checked at the entity and numeric level, and repair is realized as typed state transitions that cannot introduce content without tool-produced provenance. We instantiate this design as LedgerMind (Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger), augmented by a Three-Layer Grounding Protocol, an Adaptive Dual-Path Dispatcher that matches reasoning depth to question complexity, and an Event-Triggered Verification-and-Repair engine with a formal provenance non-amplification guarantee. We use LedgerMind to target four recurring failure patterns that final-answer accuracy tends to obscure: unsupported intermediate reasoning, citation-backed entity hallucination (Phantom Grounding), over-reasoning on simple queries, and repair-time amplification. Experiments across multiple multimodal reasoning benchmarks and backbone MLLMs show that LedgerMind improves both answer accuracy and trajectory-level faithfulness.

Enjun Du; Hange Zhou; Chenxu Du; Siyi Liu; Zirong Chen; Ziyu Zheng; Yongqi Zhang


Control flow is represented as an inspectable graph rather than an opaque prompt loop. Shifts the gate earlier than the usual post-hoc check: a GNN trained on historical trajectory graphs flags action sequences likely to fail before they execute, letting the agent self-correct pre-emptively. A cheap pre-execution veto is exactly the primitive unattended loops need to avoid burning steps on doomed branches.

Use Leveraging Trajectory Graphs for Pre-Execution Error Diagnosis in Agentic LLM Systems to measure progress and gate completion with repeatable evidence.

Research source arXiv:2607.27443; inspect its method and evaluation before treating results as production evidence.

[2607.27443] Leveraging Trajectory Graphs for Pre-Execution Error Diagnosis in Agentic LLM Systems

Large Language Model~(LLM)-based agents have demonstrated exceptional performance across a wide range of complex interactive tasks. However, they often struggle with long-horizon interactive tasks common in domains, such as embodied AI. The complexity and vast action spaces in these settings lead to compounding errors, where a single suboptimal action can derail an entire trajectory, causing the agent to exhaust its limited step budget on inefficient or unrecoverable paths. To overcome this without costly fine-tuning, we draw inspiration from software debugging, where execution logs are analyzed to preemptively catch errors. We propose \textit{Trajectory Graph Copilot}, a novel framework that acts as a ``copilot'' for LLM agents by diagnosing potential action errors before they are executed. At its core,\textit{Graph Debugger} models historical trajectories as a probabilistic graph and uses a Graph Neural Network to identify sequential action patterns that frequently lead to failure. Functioning as a proactive diagnostic sandbox, our method provides early warnings on potentially flawed actions, prompting the agent to self-correct. This pre-action error diagnosis prevents costly mistakes, significantly enhancing the agent's ability to complete long-horizon tasks successfully. The extensive experiments on four benchmarks with three LLM agents demonstrate a $14.69\%$ pass ratio improvement on average.

Xu Zheng; Zhuomin Chen; Chaohao Lin; Hua Wei; Haifeng Chen; Wei Cheng; Dongsheng Luo


Verification is promoted from a final check to a loop-control signal. Benchmarks the judge rather than the agent, showing that even frontier VLM judges share a systematic leniency bias that marks failed computer-use runs as successes, the failure mode that quietly corrupts any loop gated on an LLM verifier. Ships OS-Shepherd (9B/35B) open reward models matching commercial judges at 30-60% lower cost.

Use OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models to measure progress and gate completion with repeatable evidence.

Research source arXiv:2607.28609; inspect its method and evaluation before treating results as production evidence.

[2607.28609] OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models

Computer-using agents (CUAs) are advancing rapidly across the digital world. A CUA trajectory records the agent's actions, states, and reasoning. Verifying whether it fulfilled the task instruction is central to CUA evaluation, data curation, and reinforcement learning. Neither human-written verifiers nor human annotators can provide such verification at scale, so the field increasingly turns to vision-language models (VLMs) as judges of CUA trajectories. But a fundamental question has long gone unexamined: are these VLM judges reliable enough? To study it systematically, we introduce OSReward, a realistic, high-quality benchmark that evaluates VLM judges on CUA trajectories. The trajectories come from diverse agent backbones executing human-verified instructions across platforms, and are then rigorously labeled with ground-truth verdicts through multi-stage human annotation. Building on it, we derive OSReward-Hard, a challenge set concentrating genuinely hard cases, and OSReward-Multi for fine-grained efficiency and alignment scoring. The most comprehensive evaluation of VLM judges to date finds even state-of-the-art models fall short of an ideal judge, sharing a systematic leniency bias that mislabels failed runs as successes. The few reliable enough to trust are too expensive to run at scale, while affordable open models trail far behind. To close this gap, we construct and release OS-Shepherd-100K, an open corpus of reasoning-annotated trajectory judgments for the CUA community. On it, we train OS-Shepherd (9B and 35B), open reward models that supply low-cost, stable, and reliable reward signals, matching commercial judges at 30-60x lower cost than the frontier. Extensive analyses further inform the design of reliable CUA reward at scale. Our code, benchmark, dataset, and model checkpoints are available at https://os-copilot.github.io/OSReward-Home/.

Qiushi Sun; Kanzhi Cheng; Yian Wang; Bowen Yang; Hang Yan; Liheng Chen; Fangzhi Xu; Zichen Ding; Nuo Chen; Jialin Cao; Xingdong Gong; Zehao Li; Kaiming Jin; Xinfeng Yuan; Zhoumianze Liu; Jingyang Gong; Zhangyue Yin; Jiahui Gao; Zhiyong Wu; Tianbao Xie; Jianbing Zhang; Ben Kao; Lingpeng Kong


Treats feedback, telemetry, and deterministic artifacts as loop-control gates. The 'Locksmith Loop' instruments both source and target programs and iteratively searches for inputs that penetrate unexplored branches, identifying what blocks coverage rather than accepting an LLM's claim of equivalence.

Use Agentic Method for Deterministic Validation of Legacy Code Migration to measure progress and gate completion with repeatable evidence.

Research source arXiv:2607.28271; inspect its method and evaluation before treating results as production evidence.

[2607.28271] Agentic Method for Deterministic Validation of Legacy Code Migration

Migration of legacy COBOL programs to Java requires extensive testing to ensure correct functionality. This effort is often complicated by the lack of test data and the difficulty of validating all corner cases. In this paper we propose a novel agentic test-synthesis method, the "Locksmith Loop," which is initiated by preparing two runtime environments: the COBOL source and the generated Java target are each instrumented with mocks and executed off-mainframe on commodity hardware, then an iterative agentic loop performs Witness Search over input mocks to penetrate program branches, followed by parity-preserving mutations. When routing boundaries are reached, an analyzer identifies a Locked Paragraph: a condition preventing deeper exploration. Across three COBOL-Java case studies, spanning two open-source programs and one internal production-like COBOL program and ranging from 430 to 4,114 source lines, Locksmith consistently improved coverage beyond input-search plateaus, reaching nearly complete coverage on the two open-source programs and 91.90% branch coverage on the internal production-like COBOL program. The generated Java matched the COBOL reference under deterministic parity checks in all accepted test cases. Through these findings we demonstrate, to the best of our knowledge, a novel approach for validating agentic coding output using a deterministic oracle.

Andras Ferenczi; Jordan Docherty; Mariya Bessonov; Matthew Findlay; Krishna Lingamneni


The work turns loop quality into a measurable task or score. A rule-based auditable gate sits between an LLM supervisory controller and the regulatory layer, blocking unsafe setpoints.

Use Safety-Gated Agentic Supervisory Control on a Coupled Distillation Benchmark: Regime Map, Auditable Gate, and Co-Design Findings to measure progress and gate completion with repeatable evidence.

Research source arXiv:2607.27849; inspect its method and evaluation before treating results as production evidence.

[2607.27849] Safety-Gated Agentic Supervisory Control on a Coupled Distillation Benchmark: Regime Map, Auditable Gate, and Co-Design Findings

An open-weight LLM can write composition setpoints every five minutes. What a plant still needs is a hard check: named constraints, logged margins, and an admit/block decision before the regulatory layer moves. This paper puts that check in a rule-based forked-twin counterfactual gate (nine pinned constraints) and leaves the regulatory layer unchanged. On Skogestad's Column A the ladder is PID-only (C0), linear MPC (C1), ungated agent (C2), and gated agent (C3) under one contract: identical level closure (M_D, M_B), scenarios, and seeds; C2/C3 share the linear-MPC backend. The split is not subtle. Off-nominal target acquisition: the agent beats Pareto-tuned linear MPC in the strong band (C2/C1 IAE ratio 0.361 at the upper CI). Disturbance rejection on the same 16-point grid inverts by 16.03 at the upper CI (10.18 at the point estimate), where an ungated LLM supervisor does not belong. The gate compresses a specification-abandonment attractor into a bounded offset (d approx. -1.4; P95 cell IAE 11.5 to 0.77). A one-line prompt fix removes the attractor at source (6/10 to 0/10; sensitivity only, not a new headline). In a 250-cell statistical pass, 534 of 590 gate interventions are spec-on-bound geometry: the operating specification sits on a safety limit, so a well-behaved OP becomes inoperable while misbehaving ones are only contained; 318 blocks still correct actively harmful proposals. Headlines are single-column and model-conditional on DeepSeek-V4-Flash. A second-family sweep (NVIDIA Nemotron-3-Super) keeps the disturbance-rejection fails band and plant-side failure geography; magnitudes and protocol operability stay model-conditional, and Super target-acquisition strong cells are survivors only (not confirmation). Transfer means twin, constraint envelope, and setpoint interface, not a second plant class measured here.

31 pages, 8 figures. Code and data: https://github.com/cgncro-cyber/IndustrialAI. Sole author; independent research


The lethal trifecta for AI agents

Untrusted intake is treated as a loop-level security boundary. Simon Willison's rule of thumb: private data, untrusted content, and an exfiltration channel must never meet inside one unattended agent.

Use The lethal trifecta for AI agents to bound risk before recurring or unattended execution.

Contextual source from simonwillison.net; useful for practice signals or boundary conditions, not independent validation.

Bound permissions, cost, failure, and escalation.

The lethal trifecta for AI agents: private data, untrusted content, and external communication

If you are a user of LLM systems that use tools (you can call them “AI agents” if you like) it is critically important that you understand the risk of …


Prompt injection series

Untrusted intake is treated as a loop-level security boundary. Ongoing series on the core unsolved vulnerability for loops whose intake includes content written by strangers.

Use Prompt injection series to bound risk before recurring or unattended execution.


Agentic AI - Threats and Mitigations

Persistent memory is treated as an external runtime artifact. OWASP threat model for agentic systems, useful when reviewing intake, memory, tool, and delegation boundaries.

Use Agentic AI - Threats and Mitigations to bound risk before recurring or unattended execution.

Primary documentation from a platform, SDK, standard, or framework; strong implementation signal.

Agentic AI - OWASP Lists Threats and Mitigations

Explore key threats and mitigation strategies for agentic AI, focusing on security measures to address vulnerabilities in AI applications and their potential risks.


Primary-source operational guidance rather than commentary. OpenAI's official defense-in-depth guidance: least privilege, sandboxed tools, output verification, and human confirmation for the high-impact actions an unattended loop might take.

Use Designing AI agents to resist prompt injection to bound risk before recurring or unattended execution.

Designing AI agents to resist prompt injection | OpenAI

How ChatGPT defends against prompt injection and social engineering by constraining risky actions and protecting sensitive data in agent workflows.


sandbox-runtime

Execution isolation and permission boundaries are part of the design. Anthropic's OS-level filesystem and network sandboxing for arbitrary processes without requiring a container.

Use sandbox-runtime to bound risk before recurring or unattended execution.

Inspectable GitHub source (4,881 stars; 394 forks; Apache-2.0 license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - anthropic-experimental/sandbox-runtime: A lightweight sandboxing tool for enforcing filesystem and network restrictions on arbitrary processes at the OS level, without requiring a container. · GitHub

A lightweight sandboxing tool for enforcing filesystem and network restrictions on arbitrary processes at the OS level, without requiring a container. - anthropic-experimental/sandbox-runtime


E2B

Execution isolation and permission boundaries are part of the design. Open-source isolated cloud sandboxes for running untrusted, AI-generated code inside agent loops.

Use E2B to bound risk before recurring or unattended execution.

Inspectable GitHub source (13,285 stars; 981 forks; Apache-2.0 license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - e2b-dev/E2B: Open-source, secure environment with real-world tools for enterprise-grade agents. · GitHub

Open-source, secure environment with real-world tools for enterprise-grade agents. - e2b-dev/E2B


Modal Sandboxes

Execution isolation and permission boundaries are part of the design. Secure sandboxed execution for agent-driven code with resource limits and network controls.

Use Modal Sandboxes to bound risk before recurring or unattended execution.

This page is a high-level guide to Sandboxes, secure containers for executing untrusted user or agent code on Modal.


Daytona

Execution isolation and permission boundaries are part of the design. Infrastructure for running AI-generated code in fast, isolated sandboxes.

Use Daytona to bound risk before recurring or unattended execution.

Working implementation, framework, runtime, or repository; signal comes from usable code and ecosystem adoption.

Daytona - Secure Infrastructure for Running AI-Generated Code

Deploy Al code with confidence using Daytona's lightning-fast infrastructure. 90ms environment creation, stateful operations, and enterprise-grade security.


peerd

Orchestration and control flow are made explicit and inspectable. Browser-extension harness that runs the agent loop entirely client-side with user-supplied keys, sandboxed compute, and per-environment actor agents that hold only their tools and no API keys, isolating the orchestrator from untrusted content as a prompt-injection boundary.

Use peerd to bound risk before recurring or unattended execution.

Inspectable GitHub source (376 stars; 36 forks; Apache-2.0 license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - NotASithLord/peerd: The first AI agent harness native to the browser. A browser extension that runs a full agent loop where you already work: it drives your tabs, spins up sandboxed compute (JS notebooks, WASM Linux VMs, client-side apps), and shares what it builds peer-to-peer. BYOK, no backend, no telemetry. · GitHub

The first AI agent harness native to the browser. A browser extension that runs a full agent loop where you already work: it drives your tabs, spins up sandboxed compute (JS notebooks, WASM Linux VMs, client-side apps), and shares what it builds peer-to-peer. BYOK, no backend, no telemetry. - NotASithLord/peerd


Evaluation data is used as the feedback signal for improving loop behavior. Shows one poisoned email can write hidden entries into a persistent personal agent's long-term memory that silently alter future unattended runs, introducing the 108-case WhisperBench evaluation and the MemGhost attack that reaches 87.5% success.

Use When Claws Remember but Do Not Tell: Stealthy Memory Injection in Persistent Personal Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.05189; inspect its method and evaluation before treating results as production evidence.

[2607.05189] When Claws Remember but Do Not Tell: Stealthy Memory Injection in Persistent Personal Agents

Persistent personal agents combine long-term memory with access to users' external environments, enabling personalized foreground assistance and proactive background execution. This integration also creates a new path to compromise: untrusted external content can be silently written into persistent memory and later reused as trusted state. We study this threat as stealth memory injection, in which a remote black-box adversary delivers a single email payload that must induce the agent to write poisoned memory, stay hidden in the agent's response to the user, and affect future behavior. We introduce WhisperBench, a 108-case benchmark spanning five risk categories and both fact and preference poisoning. Built on a real IMAP/SMTP workflow and an authentic email agent skill, it enables full-cycle evaluation of stealth memory injection attacks. To enable this black-box attack under single-email delivery and without runtime feedback, we propose MemGhost, a one-shot payload generation framework. MemGhost uses an environment proxy to emulate persistent-agent execution and an objective proxy to convert memory adoption and conversational stealth into dense rubric-based rewards, then trains the attacker policy with supervised fine-tuning and reinforcement learning. Across 56 held-out test cases, MemGhost achieves 87.5% end-to-end success on OpenClaw with GPT-5.4 and 71.4% on Claude Code SDK with Sonnet 4.6. It also transfers across personal-agent architectures (NanoClaw and Hermes Agent) and memory backends (filesystem and vector-based Mem0), and remains effective against input-level, model-level, and system-level defenses. These results suggest that persistent memory can turn ordinary external processing into a practical pathway for long-term agent compromise.

Yechao Zhang; Shiqian Zhao; Jiawen Zhang; Jie Zhang; Gelei Deng; Xiaogeng Liu; Chaowei Xiao; Tianwei Zhang


Evaluation data is used as the feedback signal for improving loop behavior. Introduces FARMA, an attack that plants forged reasoning traces in an agent's persistent memory so poisoned rationales carry into future runs, and SENTINEL, a reasoning-guard defense that cut attack success from up to 100% to zero in evaluation.

Use Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses to bound risk before recurring or unattended execution.

Research source arXiv:2607.05029; inspect its method and evaluation before treating results as production evidence.

[2607.05029] Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses

Persistent memory has enabled large language model (LLM) agents to store factual knowledge, prior decisions, reasoning histories, tool usage information, and context. While this has improved the agent's functionality and continuity across tasks, it has also introduced a new attack surface: the agent's own reasoning history. In this paper, we introduce the Forged Amplifying Rationale Memory Attack (FARMA), which poisons an agent's remembered reasoning rather than its factual knowledge. It inserts forged reasoning traces using evasive language that bypasses keyword-based defenses, then amplifies them through self-referential reinforcement that defeats consensus-based defenses. To address FARMA, we introduce SENTINEL, a layered defense pipeline to detect forged reasoning entries. Its central component is the Reasoning Guard that structurally analyzes candidate entries for forgery using five weighted signals. We evaluate FARMA and SENTINEL across multiple agents and different LLM models with 50 trials and show that FARMA achieves an attack success rate of up to 100% under baseline conditions and is capable of defeating defense mechanisms like keyword filter and A-MemGuard. Our evaluation also shows that SENTINEL reduces FARMA's attack success rate to as low as 0% with no false positives observed across 326 benign agent traces. Our work demonstrates the need to protect not only an agent's retrieved content but also the integrity of its reasoning history.

Neeraj Karamchandani; Piyush Nagasubramaniam; Sencun Zhu; Dinghao Wu


Evaluation data is used as the feedback signal for improving loop behavior. Extends AI-control evaluation to coding agents shipping code that persists across sessions, showing a misaligned agent can spread an attack across successive PRs to evade per-transcript monitors, and adds a stateful link-tracker monitor that cuts evasion from 93% to 47%.

Use Distributed Attacks in Persistent-State AI Control to bound risk before recurring or unattended execution.

Research source arXiv:2607.02514; inspect its method and evaluation before treating results as production evidence.

[2607.02514] Distributed Attacks in Persistent-State AI Control

As AI coding agents become more autonomous, they increasingly ship code iteratively, with the codebase persisting across sessions. This persistence creates a new attack surface: a misaligned or prompt-injected agent can distribute attacks across pull requests (PRs) and time its payload for the PR with the best natural cover. To study the resulting dynamics, we introduce Iterative VibeCoding, a setting for AI control, the study of safely deploying capable but potentially untrusted AI. In Iterative VibeCoding, a coding agent builds software over a sequence of PRs in a persistent codebase while pursuing a covert side task. Our benchmark includes two task families: CLI tools and Flask web services, across 20 total task variations. We use Claude Sonnet 4.5 as the attack agent and GPT-4o as the monitor. We compare gradual attacks, which distribute the side task across PRs, against non-gradual attacks concentrated in a single PR. No single monitor is robust to both: which strategy evades best (success while evading the monitor) depends on the monitor type, so a defender cannot close off both gradual and non-gradual attacks with any one monitor. High evasion (>= 65%) generalizes across model attack agent backends (Sonnet 4.5, Gemini 3.1 Pro, Kimi K2.5), confirming this is a property of the persistent-state attack surface rather than a single model's capability. Evasion also remains high across state-of-the-art monitor models and the gap between gradual and non-gradual evasion widens for more capable models. We introduce a stateful link-tracker monitor that tracks suspicious buildup across PRs. On both task families, it detects gradual attacks substantially better than diff monitors that merely see more accumulated history. Combining this stronger monitor with trajectory monitors in a four-monitor ensemble reduces gradual-attack evasion from 93% under the weakest standard diff monitor to 47%.

Josh Hills; Ida Caspary; Asa Cooper Stickland


Verification is promoted from a final check to a loop-control signal. Verification protocol that recomputes state digests before each query and logs authorized changes to a trusted-hardware ledger, so an agent's persistent memory and tool descriptions cannot be covertly poisoned between runs and can be rolled back to the last verified state.

Use ElephantAgent: Contextual State Continuity in Agentic Systems to bound risk before recurring or unattended execution.

Research source arXiv:2607.01919; inspect its method and evaluation before treating results as production evidence.

[2607.01919] ElephantAgent: Contextual State Continuity in Agentic Systems

Agentic systems enhance their capabilities by invoking external tools and maintaining persistent memory. However, these external dependencies introduce novel attack surfaces. Recent tool and memory poisoning attacks show that maliciously crafted tool descriptors and poisoned memory can covertly bias agent behavior. These threats reflect a deeper issue: the lack of verifiable continuity in the agent's contextual state for planning and execution. We present ElephantAgent, a protocol that enforces Contextual State Continuity to defend against contextual state poisoning. Inspired by prior state-continuity mechanisms (e.g., Nimble), ElephantAgent extends this protection to the evolving contextual state of agentic systems. We define the contextual state as the bounded, security-critical subset of the agent's entire context (e.g., tool state and memory). Before processing each query, ElephantAgent recomputes the digest of the local contextual state and verifies it against the latest authorized digest. Using replicated trusted hardware, ElephantAgent maintains a linearizable ledger of authorized contextual state transitions and detects out-of-band state tampering. To handle in-band semantic abuse, ElephantAgent additionally provides Historical Traceability, enabling conditional post-hoc audit and recovery to a known-good prior state.

Jiankai Jin; Xiangzheng Zhang; Zhao Liu; Wenzhuo Xu; Dongdong Yang; Deyue Zhang; Quanchen Zou


Cloudflare security-audit-skill

The contribution is machine-readable and validation-friendly. Cloudflare's open-sourced six-phase audit pipeline in which separate validation agents try to disprove each finding and fresh agents independently verify every claim against source code, emitting schema-validated findings that accumulate across repeated runs.

Use Cloudflare security-audit-skill to bound risk before recurring or unattended execution.

Inspectable GitHub source (2,772 stars; 203 forks; MIT license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - cloudflare/security-audit-skill: A coding-agent skill for multi-phase security audits with independently verified, machine-readable findings · GitHub

A coding-agent skill for multi-phase security audits with independently verified, machine-readable findings - cloudflare/security-audit-skill


Verification is promoted from a final check to a loop-control signal. Systematizes 39 papers on the execution layer around coding agents, spanning sandbox isolation, capability control, TOCTOU races, MCP threats, and egress control, surfacing five cross-cutting gaps and four verified CVEs in production agent harnesses.

Use The Balkanization of Execution-Security Research for AI Coding Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.05743; inspect its method and evaluation before treating results as production evidence.

[2607.05743] The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilities

AI coding agents now read repositories, call tools, and execute shell commands with limited human oversight, and a fast-growing body of work studies whether the execution layer around them is actually safe. That literature is scattered. Papers on sandbox isolation, capability and access control, policy enforcement, time-of-check-to-time-of-use (TOCTOU) races, Model Context Protocol (MCP) threats, identity delegation, execution provenance, network egress control, and static analysis of agent-generated code are published independently and rarely cite one another. We systematize 39 papers published between 2023 and 2026 into 17 categories, each verified directly against its source. The same verification protocol also confirms four disclosed, patched CVEs directly affecting production agent harnesses. Reading across categories surfaces five cross-cutting gaps that no single paper addresses. (1) Isolation architectures and capability models are almost never evaluated against one another on a shared benchmark. (2) Policy-enforcement studies report failure rates from 69% to 98% of real denylists, yet no isolation paper re-evaluates its own defense under that adversarial setting. (3) TOCTOU and MCP threats are analyzed as separate literatures despite both being instances of the same state-validation problem. (4) Every enforcement mechanism assumes an honest policy author, leaving policy-authoring error itself unaddressed. (5) Benign but out-of-scope agent actions occurring at rates up to 17.1% under realistic prompting are addressed by no access-control or capability paper in the corpus. Existing broader surveys of agentic AI security discuss sandboxing only as one item among many defenses, leaving execution security without a dedicated systematization. This paper is written to fill that gap. We conclude with a research agenda directed at the five gaps.

18 pages, 15 figures, 6 tables. Systematizes 39 execution-security papers (2023-2026) into 17 verified categories. Machine-readable corpus and verification script released as a supplementary artifact


Context is managed as durable loop state rather than a single prompt payload. Execution-boundary system where a deterministic gate admits a tool call only after field authority, exact-effect authorization, and invocation authority all bind to the same action manifest, protecting loops that read attacker-writable context.

Use Context-to-Execution Integrity for LLM Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.06000; inspect its method and evaluation before treating results as production evidence.

[2607.06000] Context-to-Execution Integrity for LLM Agents

Language-model agents read attacker-writable context to solve tasks. Tool execution needs a separate authority check for protected sink fields, sink-interpreted payloads, and the invocation event. Context-to-Execution Integrity (CXI) is an execution-boundary system for this setting. Policies mark protected sink fields, typed releases carry narrow validated values from writable context to specific destinations, opaque data slots keep evidence as data, and a deterministic gate admits a call only after field authority, exact-effect authorization, and invocation authority all bind to the same action manifest. We evaluate CXI on open-weight field-projection runs, AgentDojo live episodes, a code-agent exact-effect benchmark, manifest-bound ledger faults, proposal-pressure controls, and hosted/API compatibility traces. AgentDojo covers 720 live episodes and 1,739 LLM calls; the code-agent benchmark covers 400 repository episodes with exact-effect authorization and lease-bound execution, yielding 231 safe task completions and zero observed field, effect, or invocation escapes. The accounting reports parser outcomes, authorization outcomes, and task-quality outcomes together with the admission-integrity result. Across the evaluated sinks, CXI admits execution only when field, effect, and invocation authority bind to the same action manifest.


Persistent memory is treated as an external runtime artifact. GhostWriter is a two-phase attack that poisons the long-term memory store of tool-using personal agents so injected content persists across runs and activates in later tasks.

Use When Agents Remember Too Much: Memory Poisoning Attacks on Large Language Model Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.06595; inspect its method and evaluation before treating results as production evidence.

[2607.06595] When Agents Remember Too Much: Memory Poisoning Attacks on Large Language Model Agents

Personal AI agents powered by large language models can reason and act using available tools to access emails, manage calendars, and push code to remote repositories, all with minimal oversight. When augmented with long-term memory, an agent can recall specific details relevant to the current task, reducing the need for large context windows. Currently, long-term memory agents tend to fall into two distinct domains: conversational and action-planning agents. Personal assistant agents sit at the convergence of these two domains and handle sensitive information while interacting with untrusted information sources, creating previously unaccounted security vulnerabilities. In this work, we introduce the novel attack vector, GhostWriter, which exploits current memory subsystems in tool-using personal agents to poison their memory store. GhostWriter operates in two phases: injection, where an adversary sends a hidden attack payload to the target agent; and activation, in which the poisoned memory is retrieved. We show that GhostWriter achieves near-universal injection rates of approximately 98% and a high average activation rate of approximately 60% against state-of-the-art agents. This attack is possible due to the lack of security-focused memory governance. In response, we propose Agentic Memory Sentry (AM-Sentry), which leverages two mitigation techniques: a memory-saving policy and a memory-retrieval screen. Our experiments show that AM-Sentry dramatically reduces GhostWriter's success rate while preserving agent utility.

George Torres; Sharad Shrestha; Satyajayant Misra


Persistent memory is treated as an external runtime artifact. Proposes TokenWall, a runtime firewall that audits a long-lived agent's semantic flows (memory updates, tool arguments, inter-component messages) before they reach privileged sinks, reporting attack success reduced to 12.5% with a 97.4% benign pass rate and 0.69s added latency.

Use Token-Flow Firewall: Semantic Runtime Auditing for Persistent AI Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.08395; inspect its method and evaluation before treating results as production evidence.

[2607.08395] Token-Flow Firewall: Semantic Runtime Auditing for Persistent AI Agents

Persistent AI agents extend large language models (LLMs) beyond single-turn interaction into long-lived software systems. Unlike traditional chat assistants, unsafe content in these agents can propagate through persistent state, reusable skills, and tool-mediated interactions, creating a substantially larger semantic attack surface. We observe that most security-critical interactions in such agents are transmitted through natural-language token flows, including memory updates, tool arguments, retrieved files, and inter-component communications. This observation enables a new security formulation: unsafe behavior can be intercepted as risky semantic flows before reaching privileged runtime sinks. Based on this insight, we propose TokenWall, a runtime defense framework that acts as a semantic firewall over agent token flows. TokenWall performs boundary-aware semantic auditing over these flows, constructing structured source-sink audit records, applying lightweight local inspection before execution, and selectively escalating ambiguous high-risk cases to stronger arbitration modules. Unlike prior approaches that rely on sparse auditing or remote large-model oversight, TokenWall enables full-coverage pre-execution mediation while reducing remote arbitration and latency. Experiments on CIK-Bench show that TokenWall reduces attack success rate to 12.5% while maintaining a 97.4% benign executable pass rate without human confirmation. TokenWall further introduces only 0.69 seconds of additional latency on benign cases, demonstrating that semantic runtime containment can achieve a practical security-utility trade-off for persistent AI agents.

Puji Wang; Yingchen Zhang; Ruqing Zhang; Jiafeng Guo; Xueqi Cheng


Untrusted intake is treated as a loop-level security boundary. Applies contextual least privilege to web agents by dynamically labeling page content with trust levels and mechanically confining what the agent can see and do, cutting cross-site prompt-injection attack success on benign pages without requiring developer annotations.

Use Prismata: Confining Cross-Site Prompt Injection in Web Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.08147; inspect its method and evaluation before treating results as production evidence.

[2607.08147] Prismata: Confining Cross-Site Prompt Injection in Web Agents

Autonomous web agents promise to automate everyday browsing tasks, but inherit one of the web's oldest attack surfaces. Cross-Site Scripting proved that mixing trusted and untrusted content is dangerous, even on benign pages. Agents resurface this risk by interpreting natural language as instructions, allowing third-party and user-generated content to hijack the agent via prompt injection. The core challenge is that deriving a task-specific security policy requires reasoning over page structure that is entangled with the attacker's content. We present Prismata, a defense enforcing contextual least privilege for web agents, constraining both what the agent sees and what it can do. Prismata's dynamic trust derivation produces permission labels for page content, with structural confinement guarantees, inspired by classical integrity models, that bound any labeling errors so that labels can only decrease in privilege and mislabelings are bounded. Prismata's mechanical confinement enforces these labels by redacting content and restricting agent capabilities. Importantly, these mechanisms require no developer annotations, so Prismata supports the long tail of websites. Across recent published web agent attacks, including adaptive variants, Prismata substantially reduces attack success while preserving benign task utility.

Corban Villa; Alp Eren Ozdarendeli; Sijun Tan; Raluca Ada Popa


The work targets tasks that exceed a single context window or prompt session. Embeds a two-channel attribution watermark in LLM-agent trajectory logs (one channel keyed on content for deletion resistance, one on log structure for rewrite resistance) so provenance survives an adversary with full read/write access, reporting detection scores near z = 100 on long-horizon ToolBench and ALFWorld trajectories with no loss of agent performance.

Use TRACE: A Two-Channel Robust Attribution Watermark via Complementary Embeddings for LLM-Agent Trajectories to bound risk before recurring or unattended execution.

Research source arXiv:2607.08400; inspect its method and evaluation before treating results as production evidence.

[2607.08400] TRACE: A Two-Channel Robust Attribution Watermark via Complementary Embeddings for LLM-Agent Trajectories

LLM agents reach users through resellers, who may rebrand a developer's agent or substitute a cheaper model. When provenance is disputed, attribution rests on the trajectory log (the record of tool calls, observations, and executed actions, not the model's reasoning), which the reseller stores and processes to meter usage. A watermark must therefore survive an adversary with full read/write access to the very evidence it is detected from; existing agent watermarks do not, as their attribution is read straight off that log. We present TRACE, to our knowledge the first agent watermark that is distortion-free in its action choices, self-synchronizing under deletion, and unconditionally invariant under rewriting. Deletion desynchronizes a position-derived key and rewriting alters content, so a deletion-robust key must come from content and a rewrite-robust key from position, and no single key serves both. A trajectory, however, has room for two watermarks. TRACE superposes a selection channel that sets which action is chosen, keyed on local content with a distortion-free sampler, so the agent's distribution is provably unchanged and detection resynchronizes after deletions, and a tally channel that sets how many records each decision group holds, keyed on the log's skeleton alone, which no rewriting can touch. We prove this behavioral watermark's signal is bought with decision entropy, each decision paying at least half its entropy and deterministic decisions nothing, and that erasing both channels forces the reseller to corrupt the trajectories it resells. On ToolBench and ALFWorld, TRACE matches the unwatermarked agent's success rate while its selection channel reaches detection scores near z = 100 on long-horizon trajectories, stays detectable under 70% step deletion, and keeps a tally channel exactly unchanged under LLM rewriting of any strength.

Zheng Gao; Xiaoyu Li; Xiaoyan Feng; Jiaojiao Jiang; Yang Song; Yulei Sui; Zhenchang Xing; Liming Zhu


Frames security as a recurring-loop boundary rather than a one-time prompt hygiene issue. Replaces binary attack-success red-teaming metrics with a seven-level ordinal severity rubric (L0-L6) that grades harm along the agent's tool-call trajectory by action reversibility, scope expansion, and privilege escalation, validated with deterministic analysis and frontier-model judges across multiple victim models and defenses.

Use Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.07474; inspect its method and evaluation before treating results as production evidence.

[2607.07474] Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents

Agentic red-teaming benchmarks report whether an injected agent was compromised as a single bit: the attack succeeded, or it did not. We argue that this binary attack-success rate discards the information a defender most needs, namely how harmful the resulting action was. We introduce an action-graded harm rubric that scores an agent's tool-call trajectory on a seven-level ordinal scale (L0 to L6) according to whether the executed action was reversible, whether it crossed scope to reach another party, and whether it expanded privilege. We compute the scale two ways: a deterministic oracle that reads the trajectory and the attacker's stated goal, and a panel of three frontier language-model judges that read a tag-free account of the same trajectory. Across four victim models and two defenses on the AgentDojo workspace suite, severity grading exposes three cases the binary metric hides, including a defense that reports a zero attack-success rate while still permitting an externally visible cross-scope leak through an unfiltered tool. The judge panel reproduces the oracle with high ordinal agreement (Krippendorff's alpha = 0.91) but shares systematic blind spots that we characterize, most notably a failure to recognize escalation chains. Unlike prior work that provides harm taxonomies, harmful-task completion tests, execution-level safety benchmarks, or severity-aware simulation, our contribution is a reusable, trace-grounded severity instrument applied to the actual actions recorded in existing red-team logs. All code, prompts, and per-episode logs are released.

8 pages, 6 figures. Code and artifacts: https://github.com/Harry-Ashley/action-graded-severity


Frames security as a recurring-loop boundary rather than a one-time prompt hygiene issue. Introduces adversarial hallucination squatting, in which attackers pre-register resource names LLMs predictably hallucinate (at rates up to 85-100%) and plant universal, cross-model-transferable promptware payloads on the open web, reaching agent loops that autonomously ingest internet content with no direct injection channel.

Use Beware of Agentic Botnets: Scalable Untargeted Promptware Attacks via Universal and Transferable Adversarial HalluSquatting to bound risk before recurring or unattended execution.

Research source arXiv:2607.07433; inspect its method and evaluation before treating results as production evidence.

[2607.07433] Beware of Agentic Botnets: Scalable Untargeted Promptware Attacks via Universal and Transferable Adversarial HalluSquatting

The growing adoption of agentic LLM applications has introduced a new threat previously named as promptware. While prior work has established that adversaries can exploit direct channels to LLM applications to apply promptware under weak threat models, many applications do not provide any direct channels that could be exploited for prompt injection beyond the Internet. This raises a question: can attackers exploit LLM applications at scale without any direct channels in practical threat models? In this work, we show that the inherent tendency of LLMs to hallucinate resource identifiers can be exploited to amplify untargeted promptware attacks that pull adversarial prompts at scale and could be exploited to establish a botnet. We introduce adversarial hallucination squatting, a technique in which attackers identify trending resources (e.g., popular repositories, popular skills, etc.), compute the LLM distribution of hallucinations on the trending resource names, and preemptively register them to host adversarial prompts. By leveraging the predictability and transferability of hallucinations across foundational LLMs and to application layers, adversaries can significantly amplify the reach of untargeted promptware under weak threat models and establish a botnet by exploiting LLM applications to install a bot on the device that pulled the compromised hallucinated resource from the Inter. We empirically demonstrate that hallucinated resource generation occurs at high rates, up to 85% in repository cloning scenarios and up to 100% in skill installation, and that these hallucinations transfer between foundational models and different prompts. We demonstrate the practicality of adversarial hallucination squatting against various production LLM applications with integrated terminals in their set of tools, achieving remote tool execution and remote code execution.

Aya Spira; Stav Cohen; Elad Feldman; Ron Bitton; Avishai Wool; Ben Nassi


Frames security as a recurring-loop boundary rather than a one-time prompt hygiene issue. Noma Labs researcher Sasi Levi shows how hidden plain-English instructions in a malicious GitHub Issue make an issue-triggered GitHub Agentic Workflows agent exfiltrate private-repo contents into public comments, bypassing GitHub's data-leak guardrails with a one-word reframe, disclosed responsibly to GitHub.

Use GitLost: How We Tricked GitHub's AI Agent into Leaking Private Repos to bound risk before recurring or unattended execution.

Contextual source from noma.security; useful for practice signals or boundary conditions, not independent validation.

GitLost: How We Tricked GitHub’s AI Agent into Leaking Private Repos - Noma Security

TL;DR: Noma Labs discovered a critical prompt injection vulnerability within GitHub’s new Agentic Workflows, allowing an unauthenticated attacker to silently pull data from private repositories by posting a crafted GitHub Issue in a public repository belonging to the same organization as the private repositories. Noma Labs named the vulnerability GitLost. Introduction GitHub recently launched […]


The work turns loop quality into a measurable task or score. Benchmark of 4,897 tool calls from offensive-security agent trajectories, labeled by professional penetration testers, for studying pre-execution gating where a cheap trusted LLM judge accepts or rejects each proposed call before it runs and the engagement boundary must be inferred from the request rather than a fixed policy.

Use ScopeJudge: Cost-Aware Pre-Execution Gating for Offensive Security Agents to bound risk before recurring or unattended execution.

Benchmark or leaderboard source with repeatable tasks or scores; compare systems only after checking setup and scope.

[2607.07774] ScopeJudge: Cost-Aware Pre-Execution Gating for Offensive Security Agents

As LLM agents take on offensive security work, a single out-of-scope tool call can breach a client's engagement boundary, disrupt production, or void a bug-bounty finding. Unlike a fixed safety policy, the boundary that matters is declared in the user's request and must be inferred from intent. That challenge is sharpened by the adversarial nature of offensive security: the same tool call is in or out of scope depending not on the action itself but on the target it touches and the context in which it runs, which no fixed policy can enumerate in advance. We study pre-execution gating: a cheap, trusted LLM judge inspects each call proposed by a strong, swappable agent, and accepts or rejects it before it runs. We introduce ScopeJudge, a benchmark of 4,897 tool calls (7.7% scope violations) from agent trajectories on tasks engineered to tempt agents out of scope and labeled at the call level by professional penetration testers, with substantial inter-grader agreement (Fleiss kappa = 0.64) that sets an expert agreement reference point of F1 = 0.78. We evaluate eight judge models under five transcript strategies, varying how much context the judge sees, from the static policy alone to the full raw transcript, and chart the resulting cost-accuracy Pareto frontier. We find that a static policy is structurally insufficient for scope enforcement: blind to the user's request, judge recall collapses to near zero, confirming that scope lives in the request and that request-conditioned monitoring is necessary. Because a missed violation costs more than a spurious rejection, we report precision, recall, and F1 separately and recommend two operating points: a cost-sensitive configuration and a recall-first one for high-stakes deployments. We release the ScopeJudge dataset to support real-time monitoring and scalable oversight of autonomous security agents.

Shane Caldwell; Max Harley; Ads Dawson; Michael Kouremetis; Vincent Abruzzo; Will Pearce

22 pages, 4 figures, 4 tables, 1 link to code, 1 link to dataset


The work separates roles across agents, verifiers, or orchestration layers. Studies AI control when several agents operating jointly on shared infrastructure pursue a malicious goal, evaluated on FakeLab (synthetic AI-lab codebase: 9 services, 86 benign tasks, 4 attack scenarios); finds a fragmentation effect (the more agents coordinate an attack, the less likely per-agent monitors catch any single attacker) and that explicit planners amplify attack success, directly relevant to monitoring fleets of background agents.

Use Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors to bound risk before recurring or unattended execution.

Research source arXiv:2607.07368; inspect its method and evaluation before treating results as production evidence.

[2607.07368] Multi-Agent AI Control: Distributed Attacks Hamper Per-Instance Monitors

AI control is a family of techniques to prevent an AI with malicious goals from subverting its operator's intent. AI Control usually studies a single agent in one trajectory, but real deployments run many agents over shared infrastructure, and the most severe risks (model-weight exfiltration, training-run poisoning) plausibly need several agents acting in concert. We initiate the empirical study of multi-agent AI control, formalising distributed attacks in which several agents jointly aim for a malicious goal. We develop FakeLab: a synthetic AI-lab codebase (9 services, 86 benign tasks, 4 attack objectives). We evaluate single agent monitoring against distributed attacks, varying the number of agents, their coordination, model capabilities and precise monitoring configuration. Our central finding is the fragmentation effect: as more agents coordinate to attack, per-agent monitoring becomes less likely to catch any of the attackers. Secondly, this is not driven by the ratio of benign to malicious code -- we argue this is likely dependent on model capability. Third, an explicit planner amplifies the fragmentation effect, raises the rate of completing the attack up to sevenfold, and mildly sharpens executors' poor monitor-theory-of-mind. Finally: a weak "trusted" monitor misses most attacks, while a stronger one cuts undetected success by more than an order of magnitude but remains imperfect. We are happy to share FakeLab, our control setting, with safety researchers on request.

Oliver Makins; Orazio Angelini; Zohreh Shams; Mary Phuong

Submitted to NeurIPS; 81 pages; 32 figures and 24 tables


Context is managed as durable loop state rather than a single prompt payload. Finds taint-style flaws make up a substantial fraction of MCP-server vulnerabilities and normally demand context-specific code fixes, then proposes SPELLSMITH, which hardens tool descriptions with security-aware behavioral guidance so the agent's own self-reflection steers it away from triggering the vulnerable flows, mitigating multiple vulnerability classes at the tool-protocol layer without code-level patches.

Use Mitigating Taint-Style Vulnerabilities in MCP Servers via Security-Aware Tool Descriptions to bound risk before recurring or unattended execution.

Research source arXiv:2607.07461; inspect its method and evaluation before treating results as production evidence.

[2607.07461] Mitigating Taint-Style Vulnerabilities in MCP Servers via Security-Aware Tool Descriptions

Large language models (LLMs) are increasingly deployed as autonomous agents that interact with external tools and services via the Model Context Protocol (MCP), a standardized interface for dynamic tool invocation. While MCP simplifies integration, it also expands the attack surface and enables generic exploits across multiple servers. Despite prior work on malicious MCP servers, the vulnerability landscape of MCP servers remains underexplored. In this work, we systematically analyze MCP server vulnerabilities, focusing on metadata characteristics, vulnerable code patterns, and community responses. Our study reveals that taint-style vulnerabilities constitute a substantial fraction of MCP server vulnerabilities, require significant code modifications to remediate, and are met with slow community responses. Motivated by these findings, we propose SPELLSMITH, presenting a novel textbased avenue for shielding taint-style vulnerabilities in MCP servers. In particular, SPELLSMITH analyzes the high-risk capabilities exposed by an MCP server and combines them with tool descriptions and parameter semantics to identify potential taint-style vulnerability risks, thereby constructing a tool-level risk profile. Then, SPELLSMITH leverages the Description property of the protocol to embed behavioral guidance (Description Enhancement Module) and exploits LLMs' self-reflection capabilities (Self-Reflection Module) to iteratively evaluate and refine outputs. By strengthening LLM internal decision-making, SPELLSMITH provides an active and unified mitigation strategy that generalizes across multiple vulnerabilities, reducing reliance on context-specific code-level fixes. Our experiments demonstrate that SPELLSMITH effectively mitigates taint-style vulnerability exploitation in MCP servers, highlighting its practical applicability and advantages over traditional code-level mitigations.

Yang Shi; Jiaheng Fu; Yihe Huang; Ruixiang Wu; Chengyao Sun; Kaifeng Huang


Verification is promoted from a final check to a loop-control signal. Factory upgrades the verification gate on every autonomous Droid commit with a two-model pipeline flanking the deterministic secret scanner, pairing a high-recall risk model with a precision referee.

Use Factory Droid Shield 2.0: Learned Secret Detection for Autonomous Commits to bound risk before recurring or unattended execution.

Contextual source from factory.ai; useful for practice signals or boundary conditions, not independent validation.

Droid Shield 2.0: learned secret detection | Factory.ai Factory.ai Logo Arrow Right Icon

Droid Shield 2.0: learned secret detection Factory's Droids write, refactor, and commit code autonomously at a volume th...


destructive_command_guard

Frames security as a recurring-loop boundary rather than a one-time prompt hygiene issue. Rust safety hook that intercepts and blocks destructive Git and shell commands (hard resets, recursive deletes, database drops) before AI coding agents execute them, across Claude Code and other harnesses.

Use destructive_command_guard to bound risk before recurring or unattended execution.

Inspectable GitHub source (5,673 stars; 221 forks; NOASSERTION license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - Dicklesworthstone/destructive_command_guard: The Destructive Command Guard (dcg) is for blocking dangerous git and shell commands from being executed by agents. · GitHub

The Destructive Command Guard (dcg) is for blocking dangerous git and shell commands from being executed by agents. - Dicklesworthstone/destructive_command_guard


Untrusted intake is treated as a loop-level security boundary. Proof-of-concept showing prompt injections spread across ordinary repository files can hijack defensive security agents into remote code execution, demonstrating that even security-focused agent loops inherit the untrusted-content attack surface.

Use Friendly Fire: Hijacking Defensive Cyber AI Agents for Remote Code Execution to bound risk before recurring or unattended execution.

Research source; inspect its method and evaluation before treating results as production evidence.

Friendly Fire: Hijacking Defensive Cyber AI Agents for Remote Code Execution - AI Now Institute

AI Now’s latest research demonstrates a critical attack vector on popular AI agents, built by Anthropic and OpenAI, when used for defensive purposes that actually turn the agent against its user.

Author and date read from the publication's own article:published_time and author meta tags; kept in code because automatic resolution falls back to the domain for this host.


How We Contain Claude Across Products

Execution isolation and permission boundaries are part of the design. Anthropic engineering on capping agent blast radius with three containment architectures matched to threat models, including ephemeral sandbox containers for untrusted code execution.

Use How We Contain Claude Across Products to bound risk before recurring or unattended execution.

Contextual source from www.anthropic.com; useful for practice signals or boundary conditions, not independent validation.

How we contain Claude across products \ Anthropic

Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.


Frames security as a recurring-loop boundary rather than a one-time prompt hygiene issue. Large-scale study of live MCP servers finding widespread security weaknesses and that existing MCP security scanners miss or misreport many of them, a gap for anyone gating agent tool access on scanner output.

Use Rethinking MCP Security: A Large-Scale Study of Runtime MCP Servers and Scanner Reliability to bound risk before recurring or unattended execution.

Research source arXiv:2607.11086; inspect its method and evaluation before treating results as production evidence.

[2607.11086] Rethinking MCP Security: A Large-Scale Study of Runtime MCP Servers and Security Scanner Reliability

The Model Context Protocol (MCP) has rapidly established itself as a standard interface for enabling LLM-based agents to interact with external tools and services. As MCP servers are increasingly entrusted with security-sensitive operations, understanding their real-world risks has become critical. In practice, due to the absence of large-scale runtime MCP servers, such understanding largely relies on security scanners applied to a small number of cases, yet the reliability of these assessments remains unclear. In this study, we revisit how MCP security is measured. We present MCPZoo, the largest collection of MCP servers for dynamic analysis to date. MCPZoo is constructed through a multi-agent framework for transforming in-the-wild static repositories into dynamic services. The framework emulates how human experts build, diagnose, and iteratively repair deployment and runtime defects by combining environment inference with feedback-driven refinement. To ensure practical interactivity at runtime, the servers are validated via real protocol interactions. As a result, MCPZoo contains 64,611 unique MCP servers (113,927 in total), with more than 37,288 supporting dynamic analysis. Leveraging MCPZoo, we conduct the first ecosystem-scale measurement of MCP servers and the scanners that analyze them. While existing scanners report that 96.89% of servers are risky, we find that these signals are unreliable. In particular, manual validation shows that less than 50% of sampled alerts are true positives, and scanner outputs exhibit clear inconsistency across scanners. Overall, MCPZoo enables large-scale, reproducible measurement of MCP server security and exposes limitations of current scanning practices. We further release a public query interface to support practical risk assessment of MCP servers.

Pei Chen; Baichao An; Mengying Wu; Binwang Wan; Geng Hong; Jinsong Chen; Xudong Pan; Jiarun Dai; Min Yang

18 pages, 11 figures, and 10 tables. This article substantially extends the preliminary 3-page MCPZoo dataset release arXiv:2512.15144. Includes appendices


Verification is promoted from a final check to a loop-control signal. Uses an autonomous research loop to red-team production agents, having one agent iteratively discover, reproduce, and refine attacks against another, turning red-teaming itself into a recurring verified loop.

Use Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming to bound risk before recurring or unattended execution.

Research source arXiv:2607.11698; inspect its method and evaluation before treating results as production evidence.

[2607.11698] Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming

Production LLM agents such as Claude Code and Codex operate over untrusted content, files, commands, and workspace state, making safety failures directly actionable. Red-teaming must therefore keep pace with evolving models and tools. Existing approaches mainly optimize attack success and preserve artifacts such as benchmarks, payloads, or attack programs, which record where attacks succeed but not the enabling conditions behind unsafe agent behavior. We study automated red-teaming for production LLM agents using one agentic research environment to discover reusable vulnerability knowledge about another. We present AHA, a falsifiable discovery loop that proposes a vulnerability hypothesis, constructs a falsifier, instantiates a valid attack, executes it in a sandboxed harness, reflects on the trajectory, and promotes confirmed findings into a Vulnerability Concept Graph (VCG). Each concept links an attacker-facing surface to an unsafe trajectory through a claim, enabling condition, falsifier, transfer prediction, and supporting evidence. Across Claude Code and Codex on three scenarios covering direct and indirect attacks, the discovered concepts reveal a reusable vulnerability core across models and agents. A frozen VCG requires no further search and outperforms the strongest frozen discovery baseline by 14.2 percentage points under the same single-shot protocol, while transferring across scenarios and attack channels. The resulting VCG provides an auditable artifact for production safety teams to inspect vulnerabilities, validate patches, and accumulate reusable safety knowledge. Our code is available at https://github.com/henrymao2004/Auto-research-red-teaming-in-sleep.


Frames security as a recurring-loop boundary rather than a one-time prompt hygiene issue. Proposes binding an agent's authority to the moment of commit rather than the moment of request, so a permission granted mid-run cannot be replayed later to cause irreversible effects in unattended execution.

Use Temporary Authority, Permanent Effects: Commit-Time Authorization for LLM Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.10487; inspect its method and evaluation before treating results as production evidence.

[2607.10487] Temporary Authority, Permanent Effects: Commit-Time Authorization for LLM Agents

LLM agents can commit durable effects from authority evidence that was valid earlier in execution: a DOM snapshot, approval epoch, version witness, branch token, or worker result. We study the commit boundary at which earlier authority evidence no longer authorizes a durable effect. We call this property commit-time authorization: a durable effect is authorized only if the witness that licensed its derived state remains fresh, causally prior, bound to the same effect, and eligible at commit time. We build a controlled-invalidation suite spanning browser, tool/API, and multi-agent workflows. The suite preserves the user goal and payload shape while invalidating the authority relation before durability. In the primary 54-task matrix, endpoint success remains high: 262/270 runs reach the visible result. Only 55/270 are authorized completions; among the 216 invalidating rows, 207 commit after the authorizing path has failed. All 54 clean controls remain authorized, and a separate 54-run authority-preserving check produces no unauthorized commits. We then evaluate mitigation families. Prompt caution and single-condition checks are insufficient because different hazards break different boundary conditions. Defenses work when they refresh, rebind, replan, or refuse at the durability boundary. CommitGuard, a fail-closed boundary monitor, blocks stale durable-effect attempts on protected commit surfaces when runtimes emit witness, dependency, binding, and eligibility signals. The result is a reporting and runtime-design lesson: endpoint success is a utility metric; authorized commit is a security property.


Frames security as a recurring-loop boundary rather than a one-time prompt hygiene issue. Automated auditing framework that probes CLI coding agents for real-world harmful behavior and grades alignment, giving unattended-agent operators a repeatable safety check rather than manual spot review.

Use ANCHOR: Automated Alignment Auditing for CLI Agents on Real-World Harm to bound risk before recurring or unattended execution.

Research source arXiv:2607.10455; inspect its method and evaluation before treating results as production evidence.

[2607.10455] ANCHOR: Automated Alignment Auditing for CLI Agents on Real-World Harm

Autonomous CLI agents can now execute hundreds of actions across multi-hour sessions: writing code, executing shell commands, browsing the web, and managing cloud infrastructure, all with minimal human oversight. Does greater autonomy invite greater risk? We introduce ANCHOR, an automated auditing framework that stress-tests CLI agents on illegal tasks grounded in public US court cases. ANCHOR deploys an auditor agent fine-tuned on dark personality data using supervised and reinforcement fine tuning. This auditor roleplays persistent malicious users who decompose tasks, reframe requests upon refusal, and adapt strategies across multi-turn interactions. Evaluating frontier CLI agents, we find that while they often refuse illegal tasks when prompted directly, compliance reaches 100\% under persistent malicious interaction. When agents comply, they frequently exceed user requests, autonomously building infrastructure for large-scale harm, including catastrophic risk scenarios such as large-scale financial fraud and bioweapon development. These findings demonstrate that current alignment techniques are insufficient for autonomous agents and underscore the need for safety evaluations against persistent, adaptive malicious users. We release ANCHOR at https://github.com/garified/anchor

Accepted at Proceedings of the 43rd International Conference on Machine Learning (ICML); the linked arXiv record is the available paper version.


Clawk

Execution isolation and permission boundaries are part of the design. Runs coding agents inside disposable, network-restricted Linux VMs so an unattended or untrusted agent's blast radius is confined to a throwaway sandbox.

Use Clawk to bound risk before recurring or unattended execution.

Inspectable GitHub source (944 stars; 31 forks; Apache-2.0 license; updated 2026-08-07); popularity is context, not proof of reliability.

GitHub - clawkwork/clawk: Give coding agents a disposable Linux VM, not your laptop · GitHub

Give coding agents a disposable Linux VM, not your laptop - clawkwork/clawk


Frames security as a recurring-loop boundary rather than a one-time prompt hygiene issue. OpenAI's alignment team on reviewing agent actions asynchronously with automated reviewers when a human cannot watch every step, so oversight scales with agent throughput instead of gating it.

Use Auto-Review of Agent Actions Without Synchronous Human Oversight to bound risk before recurring or unattended execution.

Contextual source from alignment.openai.com; useful for practice signals or boundary conditions, not independent validation.

Auto-review of agent actions without synchronous human oversight Illustrative deployment snapshot showing 10,000 actions, most inside the sandbox, and out-of-sandbox actions reviewed by Auto-review.

Auto-review offers a safer default for deploying coding agents, using a separate agent to approve or deny boundary-crossing actions.

Maja Trębacz; Sam Arnesen; Ollie Matthews; Dylan Hurd; Won Park; Owen Lin; Joe Gershenson


Context is managed as durable loop state rather than a single prompt payload. Routes each proposed action among execute, ask a human, and refuse, with one threshold controlling the deployment's risk posture; experiments report stronger overall accuracy and safety recall than the compared baselines.

Use SAFETY SENTRY: Context-Aware Human Intervention via EXECUTE-ASK-REFUSE Routing to bound risk before recurring or unattended execution.

Research source arXiv:2607.13594; inspect its method and evaluation before treating results as production evidence.

[2607.13594] SAFETY SENTRY: Context-Aware Human Intervention via EXECUTE-ASK-REFUSE Routing

LLM agents act on real-world environments through tool calls, and a single misjudged action can cause irreversible harm. The standard safeguard is a guard model that labels each proposed action as safe or unsafe, but this binary view conflates two distinct decisions: whether the action is harmful in itself, and whether it is appropriate given the user's context. It also operates at the granularity of action categories rather than individual instances, producing routine interruptions that erode autonomy and train users to wave through the most consequential alerts. We reframe the problem as a per-instance three-way routing decision over {EXECUTE, ASK, REFUSE} and instantiate it with Safety Sentry, a lightweight guard model whose inference reduces to a single decoding call. A single decoding-time threshold lets one fixed checkpoint be re-positioned across deployments of differing risk tolerance without retraining. Safety Sentry outperforms a broad set of open-weight and frontier closed-source baselines on overall accuracy and safety-related recall, while controlling both directional error rates simultaneously.


Verification is promoted from a final check to a loop-control signal. Normalizes heterogeneous tool calls into a canonical runtime action object that can be verified and attested before execution; evaluation spans 96 seeds and 384 variants covering approval binding, tampering, and runtime portability.

Use CAVA: Canonical Action Verification and Attestation for Runtime Governance of Agentic AI Systems to bound risk before recurring or unattended execution.

Research source arXiv:2607.13716; inspect its method and evaluation before treating results as production evidence.

[2607.13716] CAVA: Canonical Action Verification and Attestation for Runtime Governance of Agentic AI Systems

Agentic AI systems increasingly act through heterogeneous runtimes: local coding hooks, SDK tools, browser automation, managed-agent traces, API gateways, and workflow engines. A single operational act such as publishing code, changing identity state, moving money, or exporting data may therefore be represented by many incompatible runtime records. This makes a basic governance question difficult to answer: what action was actually approved, what evidence binds the approval to execution, and can an independent verifier reproduce the same action identity later? This paper presents Canonical Action Verification and Attestation (CAVA), a runtime-semantics layer for converting heterogeneous agent activity into canonical runtime action objects. CAVA is positioned below Proof-Carrying Agent Actions (PCAA): PCAA defines the deployer-owned route-review-prove governance process, while CAVA defines the stable action object that process governs. The paper formalizes canonical action identity, semantic pattern detection, approval binding, receipt integrity, runtime-portable projection, and optional attestation substrates. We study a reference implementation through a 96-seed, 384-variant benchmark covering semantic equivalence, semantic separation, wrapper bypass, false-positive control, approval binding, receipt reproducibility, attestation tamper detection, runtime portability, semantic pattern detection, policy degradation, and Azure deployment drills. The contribution is a systems formulation of action-level canonicalization and policy-addressable semantic patterns as a necessary substrate for deployer-side AI governance.

35 pages. Working paper on canonical action verification, runtime governance, semantic pattern detection, and approval-bound action receipts


Frames security as a recurring-loop boundary rather than a one-time prompt hygiene issue. Surveys 21 permission proposals and compares five commercial agents, producing a taxonomy that connects what users see in permission interfaces to how authority is represented and enforced at runtime.

Use How Agents Ask for Permission: User Permissions for AI Agents, from Interfaces to Enforcement to bound risk before recurring or unattended execution.

Research source arXiv:2607.13718; inspect its method and evaluation before treating results as production evidence.

[2607.13718] How Agents Ask for Permission: User Permissions for AI Agents, from Interfaces to Enforcement

As AI agents gain prevalence, users are increasingly exposed to the risks such systems entail. Prompt injection attacks, as well as hallucination, can cause agents to leak private information to third parties. As autonomous systems, agents also present the more active danger of performing sensitive tasks, such as bank transactions, without the user's intent or authorization. Recognizing this challenge, the agentic security community has developed numerous proposals for secure agentic systems. Much of this work has focused on product-level approaches, where agentic system developers determine and apply the same security policies and permissions to all users. Yet different users have different needs and preferences, necessitating support for user-level permissions policies in agentic AI systems. To understand how user-level permissions are handled in AI agent systems, we survey 21 proposals for agent permissions systems. From this review, we construct a taxonomy of how different systems specify user-level permissions policies, both at the user interface and internally; derive internal policies from user input; and enforce those policies at run-time. We then analyze five prominent commercial agents and compare their permissions handling to agentic permissions systems in the literature. We identify several high-level themes across the literature and commercial agents, as well as multiple gaps where future work is needed.


Frames security as a recurring-loop boundary rather than a one-time prompt hygiene issue. Tests six open-source frameworks and finds their stop or approval controls do not behave as execution barriers; 215 of 1,200 live runs performed a side effect during an approval pause, while the proposed admission gate blocks all measured violations with roughly 1 ms overhead.

Use Stop Means Stop: Measuring and Repairing the Enforcement Gap in Agent-Framework Control Primitives to bound risk before recurring or unattended execution.

Research source arXiv:2607.14166; inspect its method and evaluation before treating results as production evidence.

[2607.14166] Stop Means Stop: Measuring and Repairing the Enforcement Gap in Agent-Framework Control Primitives

Production LLM-agent frameworks ship control primitives -- human-in-the-loop approval gates, run cancellation, and execution timeouts -- whose names and documentation imply barrier semantics: while a run is paused, cancelled, or timed out, no gated side effect executes. This contract holds on none of six widely used open-source frameworks. Model-free differential probes isolate a recurring sibling leak -- an approval gate suspends its own branch while a sibling's effect executes during the pause, defeating rejection -- in every framework shipping a pre-execution gate (five of six, four execution models, two language runtimes), and confirm replay double-execution, cancellation orphans, and timeout zombies. The hazard is reachable: frontier models emit the leak-triggering plan shape at rates up to 14%, and live models driving unmodified frameworks leak 215 of 1,200 runs (P(leak | emitted)=1.00); on naturalistic tau-bench episodes models serialize writes -- the everyday gap is latent -- while injection induces it deterministically and a 13-incident public corpus corroborates the replay and cancellation failures. We repair the gaps with SOUNDGATE, an environment-external Rust gate through which every side effect must be admitted, enforcing hold-until-decided, reject-cancels, dedup-on-replay, and fence-on-cancel under a stated complete-mediation contract, discharged for network egress by two kernel-enforced routes. The admission core is mechanically verified (Verus; TLA+/TLC to 7.5e7 states; TLAPS; Loom on the deployed Rust) and bridged to code by differential conformance over 1.2e7 operations with zero divergences. Under that contract SOUNDGATE blocks every measured violation on all six frameworks while releasing legitimate effects: gated tau-bench episodes complete with zero refusals at ~1 ms per write, and durable admission sustains ~12k admissions per second.

32 pages, 3 figures, 11 tables. Under review at the Journal of Systems and Software. Code: pip install soundgate (PyPI)


Persistent memory is treated as an external runtime artifact. Evaluates planted memory payloads in Claude Code and Codex across four models, showing that malicious state can affect both current and future sessions and can persist differently across harnesses.

Use Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems to bound risk before recurring or unattended execution.

Research source arXiv:2607.14611; inspect its method and evaluation before treating results as production evidence.

[2607.14611] Bad Memory: Evaluating Prompt Injection Risks from Memory in Agentic Systems

A growing class of agentic systems maintain persistent state across sessions through memory files, behavioral preferences, and knowledge bases. While this makes agents more useful and self-improving, it also creates a new attack surface for prompt injections in which malicious instructions can be embedded within persistent files and influence future behavior. In this work, we study prompt injection attacks in memory-based agentic systems using a sandboxed synthetic workspace. We evaluate two agentic systems, Anthropic Claude Code and OpenAI Codex, across four models: Claude Haiku 4.5, Claude Opus 4.7, GPT-5.2, and GPT-5.5. Our results show that although it is difficult to make an agent overwrite its own memory files using untrusted external content, payloads already planted in those files can successfully attack current and future sessions. Attack success and payload persistence vary substantially across systems, models, adversarial goals, and multi-session attack sequences. These findings show that persistent memory changes the threat model for prompt injection and motivate defenses that protect memory updates without removing useful agent adaptation.

Soham Gadgil; David Alexander; Sai Sunku; Franziska Roesner


Frames security as a recurring-loop boundary rather than a one-time prompt hygiene issue. Demonstrates five setup-instruction attack classes in 12 scenarios across production coding harnesses, including README and dependency attacks; a deterministic pre-install check closes most of the measured gap.

Use Setup Complete, Now You Are Compromised: Weaponizing Setup Instructions Against AI Coding Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.15143; inspect its method and evaluation before treating results as production evidence.

[2607.15143] Setup Complete, Now You Are Compromised: Weaponizing Setup Instructions Against AI Coding Agents

AI coding agents set up projects by reading documentation and installing the dependencies it lists, without verifying their names, sources, or known vulnerabilities. By editing only a README, requirements file, or Makefile, an attacker can redirect the agent to an untrusted registry, a known-vulnerable version, or a wrong-but-plausible name: documentation becomes a vector for code execution. We present the first systematic evaluation of package-install-time supply-chain attacks delivered through ordinary project-setup documentation across production coding-agent harnesses, probing frontier models on twelve scenarios in five attack classes, grounded in documented incidents. The same model catches an attack through one harness and installs it through another: install-time security rests on the harness-model combination, not the model alone. Agents catch blatant typosquats reliably, but plausible separator-confusion names (azurecore for azure-core) slip through, and how often depends on the harness-model pairing. Source-based attacks like registry redirection are missed almost everywhere. The source blind spot recurs on npm and Cargo, where nearly every model installs the untrusted dependency; name detection carries over less consistently across ecosystems. Security-oriented prompts recover part of the gap but only for the dimension they name; a deterministic pre-install check that verifies names, sources, and versions before any code runs closes most of it.


Frames security as a recurring-loop boundary rather than a one-time prompt hygiene issue. Screens instructions and predicts the consequences of proposed GUI actions before execution with a safety-augmented world model; on Qwen3-VL-8B-Instruct, the reported safety-utility score rises from 0.191 to 0.596 while risk cost falls from 0.347 to 0.130 under the stated settings.

Use SeerGuard: A Safety Framework for Mobile GUI Agents via World Model Prediction to bound risk before recurring or unattended execution.

Research source arXiv:2607.15550; inspect its method and evaluation before treating results as production evidence.

[2607.15550] SeerGuard: A Safety Framework for Mobile GUI Agents via World Model Prediction

Mobile graphical user interface (GUI) agents have demonstrated remarkable capabilities in automating complex tasks, yet they introduce critical safety risks where a single erroneous action can lead to irreversible consequences. Existing safety mechanisms are primarily reactive, lacking the ability to assess risks before execution. In this paper, we introduce SeerGuard, a consequence-aware safety framework designed to mitigate these risks through pre-execution instruction-level screening and action-level risk assessment. Specifically, the action-level assessment analyzes agent-proposed actions within current GUI states, anticipating likely outcomes to identify risks before they are executed. To enable these capabilities, we construct a unified safety-augmented world model (SAWM) via multi-task learning, integrating semantic next-state prediction with safety risk assessment. Extensive experiments demonstrate that SeerGuard generalizes effectively across diverse mobile GUI agents. On Qwen3-VL-8B-Instruct, it increases the safety-utility score from $0.191$ to $0.596$ at $\omega=0.8$ and reduces the risk-cost score from $0.347$ to $0.130$ at $\alpha=0.8$. Further analyses on our SAWM validate the effectiveness of the instruction-level screening, alongside the capability of action risk assessment and next-state prediction.

Xue Yu; Bo Yuan; Pengshuai Yang; Kailin Zhao; Hong Hu; Junlan Feng


Persistent memory is treated as an external runtime artifact. Shows that image-only, black-box perturbations can poison or inject persistent memories across five multimodal memory architectures, with reported attack success rates of 61.6% and 58.4%; persistent visual evidence therefore needs provenance and validation before later loops reuse it.

Use Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents to bound risk before recurring or unattended execution.

Research source arXiv:2607.15657; inspect its method and evaluation before treating results as production evidence.

[2607.15657] Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents

Multimodal AI agents increasingly rely on persistent long-term memory to ground generation in past visual and textual episodes. We show that unconditional trust in visual data creates a critical vulnerability. We propose Lucid, a black-box adversarial framework that compromises multimodal memory pipelines under a strictly image-bounded threat model, requiring no access to the target MLLM, target retrieval encoder, or the text channel. Lucid crafts imperceptible perturbations to enable two distinct failure modes based on the availability of historical context: (1) Memory poisoning, an in-context attack where the adversarial image replaces a benign one whose content is reinforced by prior textual context, reliably corrupting visual recall and steering the agent toward attacker-chosen narratives; (2) Memory injection, an out-of-context attack where the adversarial image replaces a benign one in a conversation turn devoid of prior textual grounding, causing the agent to generate attacker-influenced responses with no corrective signal from memory. We evaluate Lucid across various conversation domains and five black-box memory architectures, including graph-structured, LLM-summarized, and commercially deployed systems. Lucid achieves 61.6% ASR on poisoning and 58.4% ASR on injection, exposing a structural vulnerability in multimodal memory pipelines.

Halima Bouzidi; Mboutidem Ekemini Mkpong; Mohammad Abdullah Al Faruque


Persistent memory is treated as an external runtime artifact. Validates retrieved episodic memories against topology, resource, and outcome contracts before reuse, invoking local reasoning only when a check fails; in simulation it reduces battery violations 76.6% versus similarity-only retrieval and fallback calls 21.4% versus always reasoning.

Use MemoGuard: An Adaptive Runtime for Guarding Against Memory Traps in Communication-Limited Robot Navigation to bound risk before recurring or unattended execution.

Research source arXiv:2607.15589; inspect its method and evaluation before treating results as production evidence.

[2607.15589] MemoGuard: An Adaptive Runtime for Guarding Against Memory Traps in Communication-Limited Robot Navigation

Communication-limited robots in mission-critical scenarios such as disaster inspection and search-and-rescue must make reliable onboard decisions without access to remote operators or high-capacity reasoning services. Episodic memory reuse is an attractive low-cost fallback, but retrieval similarity does not guarantee execution validity, i.e., a retrieved action may match the current context yet be unsafe due to changed topology, insufficient battery margin, or unreliable prior outcomes. We call such high-similarity but execution-invalid episodes memory traps. This creates a safety-efficiency design space where similarity only reuse minimizes fallback cost but can be unsafe, while always invoking local reasoning improves safety at high computational and energy cost. This paper presents MemoGuard, a lightweight adaptive runtime that validates episodic memories against topology, resource, and outcome contracts before reuse, invoking fallback only when validation fails. In a graph-based corridor-inspection simulator, MemoGuard reduces battery safety violations by 76.6% over similarity-only top-1 reuse while reducing fallback calls by 21.4% over always reasoning. On an NVIDIA Jetson AGX Xavier with local llama3.2:3b fallback reasoning, this corresponds to 3.67 s and 36.97 J of avoided fallback-reasoning overhead per trial. We open-source MemoGuard at https://github.com/hetheiin/memoguard.

Rajat Bhattacharjya; Hyeonjong Ju; Sing-Yao Wu; Eli Bozorgzadeh; Nikil Dutt

Accepted at IEEE/ACM ESWEEK (CODES) 2026; the linked arXiv record is the available paper version.


Verification is promoted from a final check to a loop-control signal. Combines a formal verifier for explicit system violations with a contextual VLM judge, backed by the MobileRisk-Live sandbox and trajectory benchmark for realistic mobile-agent safety.

Use OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows to bound risk before recurring or unattended execution.

Research source; inspect its method and evaluation before treating results as production evidence.

OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows - ACL Anthology

Qiushi Sun, Mukai Li, Zhoumianze Liu, Zhihui Xie, Fangzhi Xu, Zhangyue Yin, Kanzhi Cheng, Zehao Li, Zichen Ding, Qi Liu, Zhiyong Wu, Zhuosheng Zhang, Ben Kao, Lingpeng Kong. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2026.

Qiushi Sun; Mukai Li; Zhoumianze Liu; Zhihui Xie; Fangzhi Xu; Zhangyue Yin; Kanzhi Cheng; Zehao Li; Zichen Ding; Qi Liu; Zhiyong Wu; Zhuosheng Zhang; Ben Kao; Lingpeng Kong