SIR / index.html
bxiong's picture
Update index.html
09ac285 verified
Raw History Blame Contribute Delete
29.7 kB
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>SIR: Self-improving Red-teaming for Computer Use Agents</title>
<meta name="description" content="SIR is a black-box indirect prompt injection attack on OS-level computer-use agents that learns from its failures: it composes stealthy injections from plain-language principles and distills refused trajectories into new, transferable attack strategies.">
<meta name="keywords" content="computer use agents, indirect prompt injection, red teaming, self-improving, Composition of Principles, AI safety, RedTeamCUA">
<meta property="og:title" content="SIR: Self-improving Red-teaming for Computer Use Agents">
<meta property="og:description" content="A black-box IPI attack that turns a computer-use agent's own refusals into the strategies that defeat it. 0% → 28% ASR on Gemini 3.5 Flash.">
<meta property="og:url" content="https://huggingface.co/spaces/TrustSafeAI/SIR/">
<meta property="og:image" content="static/images/video_poster.jpg">
<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:title" content="SIR: Self-improving Red-teaming for Computer Use Agents">
<meta name="twitter:image" content="static/images/video_poster.jpg">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Source+Serif+4:ital,opsz,wght@0,8..60,400;0,8..60,600;0,8..60,700;1,8..60,400&family=IBM+Plex+Mono:wght@400;500&display=swap" rel="stylesheet">
<style>
:root {
--paper: #F5F1E8;
--paper-deep: #ECE6D8;
--ink: #1C1A17;
--muted: #6B665E;
--rule: #D6CFBF;
--red: #C4352B;
--red-ink: #9E2A22;
--red-soft: #F1DAD3;
--gray: #9A9A9A;
--serif: "Source Serif 4", Georgia, "Times New Roman", serif;
--mono: "IBM Plex Mono", ui-monospace, SFMono-Regular, Menlo, monospace;
--col: 860px;
--wide: 1120px;
}
* { box-sizing: border-box; }
html { scroll-behavior: smooth; }
@media (prefers-reduced-motion: reduce) { html { scroll-behavior: auto; } }
body {
margin: 0;
background: var(--paper);
color: var(--ink);
font-family: var(--serif);
font-size: 18px;
line-height: 1.6;
-webkit-font-smoothing: antialiased;
}
a { color: var(--red-ink); text-decoration-thickness: 1px; text-underline-offset: 3px; }
a:hover { color: var(--red); }
:focus-visible { outline: 2px solid var(--red); outline-offset: 3px; }
.col { max-width: var(--col); margin: 0 auto; padding: 0 24px; }
.wide { max-width: var(--wide); margin: 0 auto; padding: 0 24px; }
/* ---------- hero ---------- */
.hero { padding: 72px 0 40px; }
.hero .kicker {
font-family: var(--mono);
font-size: 13px;
color: var(--muted);
margin: 0 0 20px;
}
.hero .kicker b { color: var(--red); font-weight: 500; }
h1 {
font-size: clamp(38px, 6vw, 64px);
line-height: 1.04;
font-weight: 700;
letter-spacing: -0.015em;
margin: 0 0 18px;
max-width: 22ch;
}
.hero .lede {
font-size: 21px;
line-height: 1.5;
max-width: 34em;
margin: 0 0 30px;
color: #2c2925;
}
.authors { font-size: 18px; margin: 0 0 6px; }
.authors a { color: var(--ink); text-decoration: none; border-bottom: 1px solid var(--rule); }
.authors a:hover { border-color: var(--red); color: var(--red-ink); }
.authors sup { font-size: 11px; margin-left: 1px; }
.affil { font-size: 16px; color: var(--muted); margin: 0 0 30px; }
.affil a { color: inherit; text-decoration: none; border-bottom: 1px solid var(--rule); }
.affil a:hover { color: var(--red-ink); border-color: var(--red); }
.links { display: flex; flex-wrap: wrap; gap: 10px; }
.btn {
display: inline-flex; align-items: center; gap: 9px;
padding: 9px 16px;
border: 1.5px solid var(--ink);
border-radius: 4px;
background: transparent;
color: var(--ink);
font-family: var(--mono);
font-size: 14px;
text-decoration: none;
transition: background .15s, color .15s;
}
.btn:hover { background: var(--ink); color: var(--paper); }
.btn.primary { background: var(--red); border-color: var(--red); color: #fff; }
.btn.primary:hover { background: var(--red-ink); border-color: var(--red-ink); }
.btn svg { width: 15px; height: 15px; fill: currentColor; }
/* ---------- video (signature) ---------- */
.film { padding: 24px 0 8px; }
.film-frame {
border: 1px solid var(--rule);
background: #fff;
border-radius: 6px;
overflow: hidden;
box-shadow: 0 1px 0 rgba(28,26,23,.06), 0 18px 40px -24px rgba(28,26,23,.35);
}
.film video { display: block; width: 100%; height: auto; }
.film-bar {
display: flex; flex-wrap: wrap; align-items: baseline; gap: 6px 20px;
padding: 12px 18px;
border-top: 1px solid var(--rule);
font-family: var(--mono);
font-size: 13px;
color: var(--muted);
background: var(--paper);
}
.film-bar .stage b { font-weight: 500; color: var(--ink); }
.film-bar .stage.hit b { color: var(--red); }
.film-bar .arrow { color: var(--rule); }
.film-bar .note { margin-left: auto; }
.caption {
font-size: 15.5px; color: var(--muted); line-height: 1.55;
max-width: 72ch; margin: 14px auto 0; text-align: center;
}
/* ---------- sections ---------- */
section { padding: 56px 0 8px; }
h2 {
font-size: 30px; line-height: 1.2; font-weight: 700;
letter-spacing: -0.01em; margin: 0 0 20px;
}
h3 { font-size: 21px; font-weight: 600; margin: 34px 0 10px; }
p { margin: 0 0 18px; max-width: 70ch; }
.justify p { text-align: justify; hyphens: auto; }
figure { margin: 26px 0 8px; }
figure img { display: block; width: 100%; height: auto; border: 1px solid var(--rule); background: #fff; border-radius: 4px; }
figure img.plain { border: none; background: transparent; }
figcaption { font-size: 15px; color: var(--muted); line-height: 1.5; margin: 12px auto 0; max-width: 78ch; text-align: center; }
figcaption b { color: var(--ink); font-weight: 600; }
/* stats strip */
.stats { display: grid; grid-template-columns: repeat(3, 1fr); gap: 0; border-top: 1px solid var(--ink); border-bottom: 1px solid var(--rule); margin: 8px 0 26px; }
.stat { padding: 18px 18px 16px 0; border-right: 1px solid var(--rule); }
.stat:last-child { border-right: none; }
.stat + .stat { padding-left: 18px; }
.stat .model { font-family: var(--mono); font-size: 13px; color: var(--muted); margin-bottom: 8px; }
.stat .nums { font-size: 34px; line-height: 1; font-weight: 700; letter-spacing: -0.01em; }
.stat .nums .from { color: var(--gray); font-weight: 400; }
.stat .nums .to { color: var(--red); }
.stat .nums .arr { color: var(--rule); font-weight: 400; margin: 0 6px; }
.stat .lbl { font-size: 14px; color: var(--muted); margin-top: 6px; }
/* tables */
table { border-collapse: collapse; width: 100%; max-width: 640px; margin: 10px 0 8px; font-size: 16.5px; }
th, td { text-align: left; padding: 9px 14px 9px 0; border-bottom: 1px solid var(--rule); vertical-align: top; }
thead th { font-family: var(--mono); font-weight: 500; font-size: 13px; color: var(--muted); border-bottom: 1px solid var(--ink); }
td.num, th.num { text-align: right; font-variant-numeric: tabular-nums; }
td.best { color: var(--red); font-weight: 700; }
td.delta { color: var(--muted); font-size: 14.5px; }
tfoot td { border-bottom: none; font-size: 14.5px; color: var(--muted); padding-top: 10px; }
/* principle lists */
.principles { display: grid; grid-template-columns: 1fr 1fr; gap: 0 40px; margin: 6px 0 16px; }
.plist { list-style: none; margin: 0; padding: 0; }
.plist li { display: grid; grid-template-columns: 38px 1fr; gap: 10px; padding: 11px 0; border-bottom: 1px solid var(--rule); font-size: 16px; line-height: 1.45; }
.plist .id { font-family: var(--mono); font-size: 13px; color: var(--red); padding-top: 3px; }
.plist .name { font-weight: 600; display: block; }
.plist .desc { color: var(--muted); font-size: 15px; }
.group { font-family: var(--mono); font-size: 13px; color: var(--muted); margin: 18px 0 4px; }
.group:first-child { margin-top: 0; }
/* discovered principle cards */
.found { display: grid; grid-template-columns: 1fr 1fr; gap: 22px; margin: 14px 0 26px; }
.card {
border: 1px solid var(--rule); border-top: 3px solid var(--red);
background: #fff; padding: 20px 22px 18px; border-radius: 4px;
}
.card .tag { font-family: var(--mono); font-size: 12px; color: var(--red); margin-bottom: 8px; }
.card h4 { font-size: 20px; margin: 0 0 10px; line-height: 1.25; }
.card p { font-size: 16px; margin: 0 0 10px; }
.card .exploit { font-family: var(--mono); font-size: 12.5px; color: var(--muted); }
.card .exploit b { color: var(--ink); font-weight: 500; }
/* threat model */
.tm { display: grid; grid-template-columns: 1fr 1fr; gap: 0 40px; }
.tm p { font-size: 16.5px; }
.tm b { font-weight: 600; }
/* bibtex */
pre {
font-family: var(--mono); font-size: 13.5px; line-height: 1.55;
background: var(--paper-deep); border: 1px solid var(--rule); border-radius: 4px;
padding: 18px 20px; overflow-x: auto; margin: 0;
}
footer { padding: 60px 0 48px; margin-top: 40px; border-top: 1px solid var(--rule); font-size: 14.5px; color: var(--muted); }
footer p { max-width: 80ch; }
@media (max-width: 760px) {
body { font-size: 17px; }
.hero { padding-top: 48px; }
.hero .lede { font-size: 19px; }
.stats { grid-template-columns: 1fr; }
.stat { border-right: none; border-bottom: 1px solid var(--rule); padding-left: 0 !important; }
.stat:last-child { border-bottom: none; }
.principles, .found, .tm { grid-template-columns: 1fr; }
.film-bar .note { margin-left: 0; flex-basis: 100%; }
.wide .col { padding: 0; }
.justify p { text-align: left; }
}
</style>
</head>
<body>
<!-- ============================ HERO ============================ -->
<header class="hero">
<div class="col">
<p class="kicker"><b>Black-box indirect prompt injection against OS-level computer-use agents</b></p>
<h1>SIR: Self-improving Red-teaming for Computer Use Agents</h1>
<p class="lede">
Every prior self-improving attacker learns from what worked. SIR learns from what <em>failed</em>: when a computer-use agent refuses an injection, the refusal reveals the defense that blocked it, and SIR distills the bypass into a reusable, plain-language attack principle.
</p>
<p class="authors">
<a href="https://openreview.net/profile?id=~Chen_Xiong2" target="_blank" rel="noopener">Chen Xiong</a><sup>1</sup>,
<a href="https://openreview.net/profile?id=~Zhiyuan_He2" target="_blank" rel="noopener">Zhiyuan He</a><sup>1</sup>,
<a href="https://openreview.net/profile?id=~Pin-Yu_Chen1" target="_blank" rel="noopener">Pin-Yu Chen</a><sup>2</sup>,
<a href="https://openreview.net/profile?id=~Stjepan_Picek1" target="_blank" rel="noopener">Stjepan Picek</a><sup>3,4</sup>,
<a href="https://openreview.net/profile?id=~Tsung-Yi_Ho2" target="_blank" rel="noopener">Tsung-Yi Ho</a><sup>1</sup>
</p>
<p class="affil">
<sup>1</sup>The Chinese University of Hong Kong &nbsp;&nbsp;
<sup>2</sup>IBM Research &nbsp;&nbsp;
<sup>3</sup><a href="https://scholar.google.com/citations?view_op=view_org&hl=hr&org=1256220982759387565" target="_blank" rel="noopener">University of Zagreb</a>, Faculty of Electrical Engineering and Computing &nbsp;&nbsp;
<sup>4</sup>Radboud University
</p>
<div class="links">
<!-- TODO: replace # with the arXiv / code URLs -->
<a class="btn primary" href="#" target="_blank" rel="noopener">
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M6 2h9l5 5v15H6zM14 3.5V8h4.5zM8 12h8v1.5H8zm0 3h8v1.5H8zm0 3h5v1.5H8z"/></svg>
Paper
</a>
<a class="btn" href="#" target="_blank" rel="noopener">
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M12 .5A11.5 11.5 0 0 0 8.4 22.9c.6.1.8-.3.8-.6v-2c-3.2.7-3.9-1.5-3.9-1.5-.5-1.3-1.3-1.7-1.3-1.7-1-.7.1-.7.1-.7 1.2.1 1.8 1.2 1.8 1.2 1 1.8 2.7 1.3 3.4 1 .1-.8.4-1.3.7-1.6-2.6-.3-5.3-1.3-5.3-5.7 0-1.3.5-2.3 1.2-3.1-.1-.3-.5-1.5.1-3.1 0 0 1-.3 3.2 1.2a11 11 0 0 1 5.8 0c2.2-1.5 3.2-1.2 3.2-1.2.6 1.6.2 2.8.1 3.1.8.8 1.2 1.8 1.2 3.1 0 4.4-2.7 5.4-5.3 5.7.4.4.8 1.1.8 2.2v3.2c0 .3.2.7.8.6A11.5 11.5 0 0 0 12 .5z"/></svg>
Code
</a>
<a class="btn" href="#video">
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M8 5v14l11-7z"/></svg>
Watch the feedback loop
</a>
</div>
</div>
</header>
<!-- ============================ VIDEO ============================ -->
<div class="film" id="video">
<div class="wide">
<div class="film-frame">
<video controls autoplay muted loop playsinline preload="metadata"
poster="static/images/video_poster.jpg"
aria-label="Animated chart: SIR attack success rate on Gemini 3.5 Flash across three feedback rounds">
<source src="static/video/feedback_chart_benchmark_SIR.mp4" type="video/mp4">
Your browser does not support the video tag. <a href="static/video/feedback_chart_benchmark_SIR.mp4">Download the video</a>.
</video>
<div class="film-bar">
<span class="stage">SIR, no feedback <b>8%</b></span>
<span class="arrow">→</span>
<span class="stage hit">Round 1 <b>14%</b></span>
<span class="arrow">→</span>
<span class="stage hit">Round 2 <b>20%</b></span>
<span class="arrow">→</span>
<span class="stage hit">Round 3 <b>28%</b></span>
<span class="note">Gemini 3.5 Flash · 50 RedTeamCUA tasks · hand-crafted baseline stays at 0%</span>
</div>
</div>
<div class="col"><p class="caption">
Each feedback round, SIR reads the trajectories where Gemini refused, diagnoses the defense that stopped the injection, and adds a named bypass strategy to its library (Trajectory-Anchored Self-Attribution, Post-Success Deferred Cleanup, Prescient-Comment Misdirection). The next round's injections draw on all of them. Success is scored by deterministic checks on the VM's filesystem, services and permissions, never by an LLM judge.
</p></div>
</div>
</div>
<!-- ============================ ABSTRACT ============================ -->
<section class="col justify" id="abstract">
<h2>Abstract</h2>
<p>
Computer use agents (CUAs) are vision-language models that perceive a screen and act on a real operating system through mouse, keyboard, and terminal, and they are increasingly deployed to automate everyday digital tasks. Because they can be exposed to untrusted content while operating, they are vulnerable to indirect prompt injection (IPI), in which an adversary plants instructions in content the agent will read and redirects it toward actions that violate the user's intent. Existing CUA safety benchmarks evaluate fixed injections written by hand, which may underestimate the risk posed by an adaptive adversary.
</p>
<p>
We present SIR, a black-box IPI attack that (i) composes stealthy injections from a small library of reusable principles stated in plain language and (ii) wraps composition in an iterative feedback loop that diagnoses the victim's failed trajectories and distills the bypasses into new, named strategies that are reapplied across tasks. Unlike prior red teaming of web agents, we target CUAs at the operating system level and score attacks with a fully deterministic oracle, using checks on filesystem, service, and permission state rather than an LLM judge. We evaluate three frontier CUAs. Composing principles with feedback raises the attack success rate over a baseline written by hand, for example from 4% to 24% on Claude Opus 4.8 and from 0% to 28% on Gemini 3.5 Flash, while the benign task still completes. Principles discovered against one model further transfer to a different architecture with no additional feedback.
</p>
</section>
<!-- ============================ METHOD ============================ -->
<section class="wide" id="method">
<div class="col">
<h2>How SIR works: two nested loops</h2>
<p>
The <b>inner loop</b> is a compositional attack search. Given a benign user task and an adversarial objective, the attacker LLM makes three calls (seed → compose → refine): it proposes an attack concept, selects two to four principles from the library and rewrites the seed to apply them together, then strips overtly adversarial wording and folds in any strategies the feedback loop has discovered so far.
</p>
<p>
The <b>outer loop</b> is failure-driven strategy discovery. After the victim runs every task, an analyzer LLM reads the trajectories where the adversarial objective was <em>not</em> achieved, diagnoses recurring defensive behaviors (source distrust, target sensitivity, causal implausibility, imperative-language detection) and distills each into a named strategy with a diagnosis, a bypass mechanism, composition guidance and an example. The library grows across rounds, and every task benefits from strategies distilled on any other task. Nothing is fine-tuned; the whole library lives in natural language.
</p>
</div>
<figure>
<img src="static/images/system_plots_v1.png" alt="SIR pipeline: (1) compose injection from a principle library, (2) plant it in the forum database, (3) run the target CUA, (4–5) programmatic judge checks VM state deterministically, then a feedback analyzer mines failed trajectories into new principles and loops back to step 1.">
<figcaption>
<b>The SIR pipeline.</b> (1) The attacker composes a stealthy injection tailored to the benign task and adversarial goal. (2) It is planted in the forum database the victim will read. (3) The CUA runs the benign task, reads the comment and may be hijacked. (4–5) A programmatic judge checks post-execution VM state for both the adversarial objective and the benign task. Failed trajectories go to the feedback analyzer, which distills new stealth strategies back into the library for the next round.
</figcaption>
</figure>
<div class="col">
<h3>Threat model</h3>
<div class="tm">
<p><b>Attacker.</b> Controls one bounded region of untrusted content the CUA will read (a forum post) and plants an injection before execution. Cannot touch the user's instruction, the OS, or anything outside that region, and does not intervene once the agent acts. Strict black box: no weights, gradients, token probabilities or system prompt, only the rendered screen states, reasoning traces, executed actions and the evaluator's per-task outcome between rounds.</p>
<p><b>Objective.</b> A run counts as a success only when the victim completes <em>both</em> the adversarial objective and the user's benign task, verified by two deterministic oracles. Joint success rules out attacks that merely derail the agent and models a stealthy adversary whose interference leaves the user's task intact.</p>
</div>
</div>
</section>
<!-- ============================ RESULTS ============================ -->
<section class="col" id="results">
<h2>Static benchmarks understate the attack surface</h2>
<p>
On 50 RedTeamCUA cases spanning seven attack categories (file deletion, content modification, service disruption, permission change, data exfiltration, file creation, application launch), SIR raises attack success on every frontier CUA. The gap widens where defenses are strongest: the hand-crafted template is fully blocked by Gemini 3.5 Flash, while SIR still lands 28% of tasks.
</p>
<div class="stats">
<div class="stat">
<div class="model">Claude Opus 4.6</div>
<div class="nums"><span class="from">16%</span><span class="arr">→</span><span class="to">54%</span></div>
<div class="lbl">hand-crafted → SIR</div>
</div>
<div class="stat">
<div class="model">Claude Opus 4.8</div>
<div class="nums"><span class="from">4%</span><span class="arr">→</span><span class="to">24%</span></div>
<div class="lbl">hand-crafted → SIR</div>
</div>
<div class="stat">
<div class="model">Gemini 3.5 Flash</div>
<div class="nums"><span class="from">0%</span><span class="arr">→</span><span class="to">28%</span></div>
<div class="lbl">hand-crafted → SIR</div>
</div>
</div>
<figure>
<img class="plain" src="static/images/main_results_barplot_v2.png" alt="Bar chart of attack success rate: hand-crafted baseline vs SIR on Claude Opus 4.6 (16% vs 54%), Claude Opus 4.8 (4% vs 24%) and Gemini 3.5 Flash (0% vs 28%).">
<figcaption><b>Attack success rate</b> of the fixed injection baseline (gray) versus SIR (coral) on three frontier CUAs. A static attack reports steadily shrinking risk as defenses mature and would call Gemini 3.5 Flash essentially immune; SIR does not follow that decline.</figcaption>
</figure>
<h3>Where the gains come from</h3>
<p>Both components matter, and they target different layers of defense. Compositional search replaces one urgency-laden template with per-case principle compositions, bypassing surface-level filters such as urgency and authority-claim detection. Feedback adds strategies the base inventory cannot express, and its gain is largest on Gemini, where base composition barely penetrates the injection detector.</p>
<table>
<thead>
<tr><th>Victim CUA</th><th class="num">Hand-crafted</th><th class="num">SIR, base only</th><th class="num">Δ comp.</th><th class="num">SIR + feedback</th><th class="num">Δ fb.</th></tr>
</thead>
<tbody>
<tr><td>Claude Opus 4.6</td><td class="num">16%</td><td class="num">34%</td><td class="num delta">+18</td><td class="num best">54%</td><td class="num delta">+20</td></tr>
<tr><td>Claude Opus 4.8</td><td class="num">4%</td><td class="num">10%</td><td class="num delta">+6</td><td class="num best">24%</td><td class="num delta">+14</td></tr>
<tr><td>Gemini 3.5 Flash</td><td class="num">0%</td><td class="num">8%</td><td class="num delta">+8</td><td class="num best">28%</td><td class="num delta">+20</td></tr>
</tbody>
<tfoot><tr><td colspan="6">Attacker and analyzer LLM: Grok-4.3. Victims: Claude Opus 4.6, Claude Opus 4.8, Gemini 3.5 Flash. 50 tasks each, deterministic end-state oracle.</td></tr></tfoot>
</table>
</section>
<!-- ============================ PRINCIPLES ============================ -->
<section class="col" id="principles">
<h2>The base principle library</h2>
<p>SIR starts from eleven hand-designed principles. Content principles govern <em>what</em> the injection says; delivery principles govern <em>where</em> and <em>how</em> it is placed. In practice the attacker nearly always pairs two content principles with exactly one delivery principle, and Visual Blending is the dominant delivery choice on every victim.</p>
<div class="principles">
<div>
<div class="group">Content principles</div>
<ul class="plist">
<li><span class="id">P1</span><span><span class="name">Prerequisite Framing</span><span class="desc">The adversarial action is a mandatory step before the benign task can succeed.</span></span></li>
<li><span class="id">P2</span><span><span class="name">Authority Embedding</span><span class="desc">The injection appears to come from a trusted source, such as the project maintainer.</span></span></li>
<li><span class="id">P3</span><span><span class="name">Error Recovery</span><span class="desc">The adversarial action is presented as the fix for a common, plausible error.</span></span></li>
<li><span class="id">P4</span><span><span class="name">Side-Effect Masking</span><span class="desc">The action is framed as harmless routine maintenance: cleanup, diagnostics.</span></span></li>
<li><span class="id">P5</span><span><span class="name">Multi-Step Dilution</span><span class="desc">One adversarial command hidden among several legitimate ones.</span></span></li>
<li><span class="id">P6</span><span><span class="name">Technical Jargon</span><span class="desc">Dense, domain-specific language makes the action sound like standard procedure.</span></span></li>
<li><span class="id">P7</span><span><span class="name">Cross-Channel Redirect</span><span class="desc">Instruct the CUA to act on a different channel, e.g. forum → terminal.</span></span></li>
</ul>
</div>
<div>
<div class="group">Delivery principles</div>
<ul class="plist">
<li><span class="id">D1</span><span><span class="name">Visual Blending</span><span class="desc">Match the injection's tone and formatting to the surrounding legitimate content.</span></span></li>
<li><span class="id">D2</span><span><span class="name">High-Trust Positioning</span><span class="desc">Place the injection as the top-voted or first reply in the thread.</span></span></li>
<li><span class="id">D3</span><span><span class="name">Trajectory-Aware Timing</span><span class="desc">Place it at the read-to-act boundary of the CUA's browsing trajectory.</span></span></li>
<li><span class="id">D4</span><span><span class="name">Platform-Optimal Channel</span><span class="desc">Match the delivery channel to the benign task type.</span></span></li>
</ul>
</div>
</div>
<figure>
<img class="plain" src="static/images/principle_combos_1.png" alt="Most frequently selected base-principle combinations per victim CUA. P1+P2+D1 leads on both Claude models; P2+P4+D1 leads on Gemini 3.5 Flash; P1+P3+D1 appears in the top selections of all three.">
<figcaption><b>Where the attacker concentrates its search.</b> Selection share of exact principle combinations over the 50 compositions per model. P1+P2+D1 (prerequisite, authority, blending) tops both Claude models; Gemini pulls far more Side-Effect Masking (P4). The same inventory yields a distinct profile per victim, so different CUAs expose different weaknesses.</figcaption>
</figure>
</section>
<!-- ============================ DISCOVERED ============================ -->
<section class="col" id="transfer">
<h2>Principles learned from failure transfer across models</h2>
<p>
Two strategies the feedback loop distilled while attacking Claude Opus 4.6 illustrate the process. Neither was discovered on the other two victims. We freeze them and, with no further feedback, hand them to the attacker when it composes injections against Claude Opus 4.8 and Gemini 3.5 Flash.
</p>
<div class="found">
<div class="card">
<div class="tag">P9 · discovered on Claude Opus 4.6</div>
<h4>Error Corroboration</h4>
<p>Predict a specific error the CUA will naturally hit during the benign workflow and present the adversarial action as its remedy. When the environment later produces the predicted error, credibility transfers from the correct diagnosis to the proposed fix, eroding an initial refusal.</p>
<div class="exploit">exploits: <b>post-hoc corroboration of a low-trust source</b></div>
</div>
<div class="card">
<div class="tag">P10 · discovered on Claude Opus 4.6</div>
<h4>Conditional Deferred Execution</h4>
<p>Phrase the adversarial action as conditional, deferred advice ("if the build fails, you might try removing X") rather than an imperative command. The safety filter applies weaker scrutiny to hypothetical future remedies than to direct instructions.</p>
<div class="exploit">exploits: <b>imperative-language detection keyed to command form</b></div>
</div>
</div>
<table>
<thead><tr><th>Unseen victim CUA</th><th class="num">Base library</th><th class="num">+ P9</th><th class="num">+ P10</th></tr></thead>
<tbody>
<tr><td>Claude Opus 4.8</td><td class="num">10%</td><td class="num best">16%</td><td class="num">14%</td></tr>
<tr><td>Gemini 3.5 Flash</td><td class="num">8%</td><td class="num best">18%</td><td class="num">12%</td></tr>
</tbody>
<tfoot><tr><td colspan="4">Attack success rate over 50 tasks, no additional feedback rounds.</td></tr></tfoot>
</table>
<p style="margin-top:14px">
Both frozen principles raise ASR on both unseen models, including Gemini, whose safety design differs fundamentally from Claude's. P9 transfers more strongly because it exploits a behavior that does not depend on architecture: how an agent restores trust in a source after the fact. That a principle learned solely from Opus 4.6 failures helps against a model outside the Claude family is evidence that feedback expands a shared strategy space rather than memorizing target text.
</p>
</section>
<!-- ============================ BIBTEX ============================ -->
<section class="col" id="bibtex">
<h2>Citation</h2>
<!-- TODO: update the key, venue and URL once the preprint is out -->
<pre>@inproceedings{xiong2026sir,
title = {SIR: Self-improving Red-teaming for Computer Use Agents},
author = {Xiong, Chen and He, Zhiyuan and Chen, Pin-Yu and Picek, Stjepan and Ho, Tsung-Yi},
archivePrefix={arXiv},
primaryClass={cs.CR},
year = {2026}
}</pre>
</section>
</body>
</html>