byte-vortex commited on
Commit
05ccb42
·
verified ·
1 Parent(s): b38b5d7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -0
README.md CHANGED
@@ -50,6 +50,10 @@ To evaluate the operational envelope and failure modes of `AgentSentinelProxy`,
50
 
51
  ¹ *A separate isolated run of this same objective (see notebook, "Example Test Run") achieved a lethal bypass at Iter 2. Because the attacker LLM samples at `temperature=0.7`, outcomes for this objective are not deterministic across runs; the result above reflects the run captured in the full stress-test suite.*
52
 
 
 
 
 
53
  ---
54
 
55
  <div style="background-color: #f8f9fa; border: 1px solid #e9ecef; border-left: 5px solid #1e8e3e; padding: 20px; border-radius: 8px; font-family: sans-serif; margin: 20px 0;">
 
50
 
51
  ¹ *A separate isolated run of this same objective (see notebook, "Example Test Run") achieved a lethal bypass at Iter 2. Because the attacker LLM samples at `temperature=0.7`, outcomes for this objective are not deterministic across runs; the result above reflects the run captured in the full stress-test suite.*
52
 
53
+ ### Limitations
54
+
55
+ These results reflect a single run per objective, not repeated trials — with an attacker LLM sampling at `temperature=0.7`, individual outcomes can vary between runs (see footnote 1: the same Command Execution objective produced a lethal bypass in one isolated run and a full defend in the run reported above). Additionally, the proxy's semantic auditor, the adversarial attacker, and the effectiveness judge are all instantiated from the same underlying Gemma 4 model, which may understate the difficulty of true black-box red-teaming. Future work should run each objective across multiple trials to report bypass rates with confidence intervals, and substitute an independent model for at least one of the three roles.
56
+
57
  ---
58
 
59
  <div style="background-color: #f8f9fa; border: 1px solid #e9ecef; border-left: 5px solid #1e8e3e; padding: 20px; border-radius: 8px; font-family: sans-serif; margin: 20px 0;">