JieRuan commited on
Commit
5fcbb0e
·
verified ·
1 Parent(s): c131f06

Add SchemeArena logo (nav, hero, favicon) and paper Figure 5 to Results

Browse files
.gitattributes CHANGED
@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  assets/framework.png filter=lfs diff=lfs merge=lfs -text
 
 
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  assets/framework.png filter=lfs diff=lfs merge=lfs -text
37
+ assets/figure5_scheming_rates.png filter=lfs diff=lfs merge=lfs -text
assets/figure5_scheming_rates.png ADDED

Git LFS Details

  • SHA256: a0920d0e786b59f68d05a58e33fea95437c3efcb4c84692adc38c6e48b800607
  • Pointer size: 131 Bytes
  • Size of remote file: 357 kB
assets/logo.svg ADDED
css/main.css CHANGED
@@ -454,3 +454,8 @@ table.data.fixed col.lab { width: 30%; }
454
 
455
  .verdict { align-items: flex-start; font-weight: 500; line-height: 1.5; }
456
  .verdict svg { flex: none; margin-top: 2px; }
 
 
 
 
 
 
454
 
455
  .verdict { align-items: flex-start; font-weight: 500; line-height: 1.5; }
456
  .verdict svg { flex: none; margin-top: 2px; }
457
+
458
+ .brand-mark { width: 28px; height: 28px; object-fit: contain; }
459
+ .hero-logo { display: block; width: 84px; height: auto; margin: 0 auto 18px; filter: drop-shadow(0 6px 14px rgba(75, 86, 98, .18)); }
460
+ .result-fig { display: block; width: 100%; height: auto; border-radius: 8px; margin-top: 6px; }
461
+ @media (max-width: 760px) { .hero-logo { width: 64px; margin-bottom: 14px; } }
index.html CHANGED
@@ -8,19 +8,14 @@
8
  <link rel="preconnect" href="https://fonts.googleapis.com" />
9
  <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
10
  <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500;600;700&family=Source+Serif+4:opsz,wght@8..60,500;8..60,600&display=swap" rel="stylesheet" />
 
11
  <link rel="stylesheet" href="css/main.css" />
12
  </head>
13
  <body>
14
 
15
  <nav id="topNav">
16
  <a class="nav-brand" href="#overview">
17
- <svg class="brand-mark" viewBox="0 0 32 32" aria-hidden="true">
18
- <rect x="1" y="3" width="30" height="27" rx="8" fill="#b9470f"/>
19
- <rect x="5" y="11" width="22" height="8" rx="4" fill="#15171b"/>
20
- <circle cx="11" cy="15" r="2" fill="#f5b38c"/><circle cx="21" cy="15" r="2" fill="#f5b38c"/>
21
- <rect x="11" y="23" width="10" height="2.4" rx="1.2" fill="#fbeee5"/>
22
- <rect x="15" y="0" width="2" height="4" rx="1" fill="#b9470f"/>
23
- </svg>
24
  SchemeArena
25
  </a>
26
  <div class="nav-tabs" role="tablist">
@@ -46,6 +41,7 @@
46
  <section id="tab-overview" class="tab-panel active">
47
 
48
  <header class="hero">
 
49
  <span class="pill"><span class="dot"></span>Benchmark · AI Safety · LLM Agents</span>
50
  <h1><span class="name">SchemeArena</span>: Factorized Stress Testing of Scheming in LLM Agents</h1>
51
  <p class="subtitle">A 400-scenario benchmark that varies instrumental goals, pressure, strategic hints, and oversight to study when LLM agents covertly pursue misaligned goals.</p>
@@ -325,11 +321,19 @@
325
  </div>
326
  </section>
327
 
 
 
 
 
 
 
 
 
328
  <section class="block" style="margin-top:20px">
329
  <div class="card chart-card">
330
  <div class="chart-head"><h3>Scheming capability (Setups 5–7)</h3></div>
331
  <div class="table-wrap"><table class="data fixed" id="capTable"></table></div>
332
- <p class="footnote">Values reported in the text of the paper (§5.3); changes are relative to Setup 5. "—": shown only graphically in Figure 5 of the paper, where the Qwen models stay near saturation across all oversight conditions. Setup 6 here is action-only oversight.</p>
333
  </div>
334
  </section>
335
 
 
8
  <link rel="preconnect" href="https://fonts.googleapis.com" />
9
  <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
10
  <link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&family=JetBrains+Mono:wght@400;500;600;700&family=Source+Serif+4:opsz,wght@8..60,500;8..60,600&display=swap" rel="stylesheet" />
11
+ <link rel="icon" type="image/svg+xml" href="assets/logo.svg" />
12
  <link rel="stylesheet" href="css/main.css" />
13
  </head>
14
  <body>
15
 
16
  <nav id="topNav">
17
  <a class="nav-brand" href="#overview">
18
+ <img class="brand-mark" src="assets/logo.svg" alt="" />
 
 
 
 
 
 
19
  SchemeArena
20
  </a>
21
  <div class="nav-tabs" role="tablist">
 
41
  <section id="tab-overview" class="tab-panel active">
42
 
43
  <header class="hero">
44
+ <img class="hero-logo" src="assets/logo.svg" alt="SchemeArena logo" />
45
  <span class="pill"><span class="dot"></span>Benchmark · AI Safety · LLM Agents</span>
46
  <h1><span class="name">SchemeArena</span>: Factorized Stress Testing of Scheming in LLM Agents</h1>
47
  <p class="subtitle">A 400-scenario benchmark that varies instrumental goals, pressure, strategic hints, and oversight to study when LLM agents covertly pursue misaligned goals.</p>
 
321
  </div>
322
  </section>
323
 
324
+ <section class="block" style="margin-top:20px">
325
+ <div class="card chart-card">
326
+ <div class="chart-head"><h3>Scheming rates across all seven setups</h3></div>
327
+ <img class="result-fig" src="assets/figure5_scheming_rates.png" alt="Line chart of scheming rates with error bars for o4-mini, o1, Claude-3.7-Sonnet, Qwen3-8B and Qwen3-32B across Setups 1 to 7, with Setup 6 split into action-only and full oversight" loading="lazy" />
328
+ <p class="footnote">Scheming rates for all setups that evaluate propensity (blue background) and capability (yellow background). Strategic hints (Setups 5 and 7) strongly enable scheming, especially for Qwen models. Action-only monitoring (Setup 6 w/ Action) slightly increases scheming among the closed models, while full-trajectory oversight yields only limited reductions. Error bars: 95% bootstrap confidence intervals. (Paper, Figure 5.)</p>
329
+ </div>
330
+ </section>
331
+
332
  <section class="block" style="margin-top:20px">
333
  <div class="card chart-card">
334
  <div class="chart-head"><h3>Scheming capability (Setups 5–7)</h3></div>
335
  <div class="table-wrap"><table class="data fixed" id="capTable"></table></div>
336
+ <p class="footnote">Values reported in the text of the paper (§5.3); changes are relative to Setup 5. "—": not stated numerically in the text; see the figure above, where the Qwen models stay near saturation across all oversight conditions. Setup 6 here is action-only oversight.</p>
337
  </div>
338
  </section>
339