What It Is, Not What It Says: Securing Agentic Browsing Interactions with Web Semantics
Abstract
Intact protects AI web agents by isolating webpage elements via provenance-aware representations and enforcing capability-bound execution with user consent for third-party actions.
AI agents increasingly browse the web on users' behalf, yet they consume webpages as screenshots or text, exposing their LLMs to unvetted third-party content. Browsers enforce semantics under which what an element can do depends on what it is and where it comes from, not on what it says. Agents flatten these semantics, so injected text can persuade the LLM to exercise the user's authority for content that has none. As LLMs remain easy to persuade, the agent's interface to the web must itself ensure the integrity of what the agent reads, interprets, and does. We propose Intact, which runs on an unmodified browser. Intact builds GenTree, a bespoke representation that captures provenance and web semantics. Intact passes the page via GenTree to the LLM one element at a time, so the LLM decides on each element without any other element's text. Its harness executes each step only within the capabilities of the step's element and asks the user before acting on third-party content. On 1,270 real webpages, GenTree attributes 2.3x as many elements to third parties as the browser, which misattributes 58.7% of them to the first party. On 1,091 of these webpages with injected attacks, only 12.8% of attacks get through Intact's planner, each at most as a consent request, against 73.1% for a conventional planner. Isolation disrupts functionality on only 3.7% of tasks, and a parallel schedule could remove most of its added latency.
Community
What It Is, Not What It Says: Securing Agentic Browsing Interactions with Web Semantics
This paper introduces Intact, a system that uses web semantics to constrain browser agents even when their LLM follows malicious instructions.
Intact builds GenTree, a representation that captures where each webpage element comes from and what it can do. It then plans one element at a time and enforces action constraints outside the LLM.
In experiments on 1,091 real webpages, with the model instructed to follow injected attacks, 73.1% of attacks succeeded against a conventional planner. Only 12.8% entered Intact’s plans, each at most as a request for user consent.
Code and attack benchmark: github.com/llm-platform-security/intact
Happy to answer questions and hear feedback, especially from people building browser agents!
Get this paper in your agent:
hf papers read 2610.22697 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper
