Papers
arxiv:2610.22697

What It Is, Not What It Says: Securing Agentic Browsing Interactions with Web Semantics

Published on Oct 15
· Submitted by
Umar Iqbal
on Oct 19
Authors:

Abstract

Intact protects AI web agents by isolating webpage elements via provenance-aware representations and enforcing capability-bound execution with user consent for third-party actions.

AI agents increasingly browse the web on users' behalf, yet they consume webpages as screenshots or text, exposing their LLMs to unvetted third-party content. Browsers enforce semantics under which what an element can do depends on what it is and where it comes from, not on what it says. Agents flatten these semantics, so injected text can persuade the LLM to exercise the user's authority for content that has none. As LLMs remain easy to persuade, the agent's interface to the web must itself ensure the integrity of what the agent reads, interprets, and does. We propose Intact, which runs on an unmodified browser. Intact builds GenTree, a bespoke representation that captures provenance and web semantics. Intact passes the page via GenTree to the LLM one element at a time, so the LLM decides on each element without any other element's text. Its harness executes each step only within the capabilities of the step's element and asks the user before acting on third-party content. On 1,270 real webpages, GenTree attributes 2.3x as many elements to third parties as the browser, which misattributes 58.7% of them to the first party. On 1,091 of these webpages with injected attacks, only 12.8% of attacks get through Intact's planner, each at most as a consent request, against 73.1% for a conventional planner. Isolation disrupts functionality on only 3.7% of tasks, and a parallel schedule could remove most of its added latency.

Community

Paper submitter

What It Is, Not What It Says: Securing Agentic Browsing Interactions with Web Semantics

This paper introduces Intact, a system that uses web semantics to constrain browser agents even when their LLM follows malicious instructions.

Intact builds GenTree, a representation that captures where each webpage element comes from and what it can do. It then plans one element at a time and enforces action constraints outside the LLM.

In experiments on 1,091 real webpages, with the model instructed to follow injected attacks, 73.1% of attacks succeeded against a conventional planner. Only 12.8% entered Intact’s plans, each at most as a request for user consent.

Code and attack benchmark: github.com/llm-platform-security/intact

Happy to answer questions and hear feedback, especially from people building browser agents!

image

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2610.22697
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.22697 in a model README.md to link it from this page.

Datasets citing this paper 1

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.22697 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.