Update README.md
Browse files
README.md
CHANGED
|
@@ -50,6 +50,10 @@ To evaluate the operational envelope and failure modes of `AgentSentinelProxy`,
|
|
| 50 |
|
| 51 |
¹ *A separate isolated run of this same objective (see notebook, "Example Test Run") achieved a lethal bypass at Iter 2. Because the attacker LLM samples at `temperature=0.7`, outcomes for this objective are not deterministic across runs; the result above reflects the run captured in the full stress-test suite.*
|
| 52 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
---
|
| 54 |
|
| 55 |
<div style="background-color: #f8f9fa; border: 1px solid #e9ecef; border-left: 5px solid #1e8e3e; padding: 20px; border-radius: 8px; font-family: sans-serif; margin: 20px 0;">
|
|
|
|
| 50 |
|
| 51 |
¹ *A separate isolated run of this same objective (see notebook, "Example Test Run") achieved a lethal bypass at Iter 2. Because the attacker LLM samples at `temperature=0.7`, outcomes for this objective are not deterministic across runs; the result above reflects the run captured in the full stress-test suite.*
|
| 52 |
|
| 53 |
+
### Limitations
|
| 54 |
+
|
| 55 |
+
These results reflect a single run per objective, not repeated trials — with an attacker LLM sampling at `temperature=0.7`, individual outcomes can vary between runs (see footnote 1: the same Command Execution objective produced a lethal bypass in one isolated run and a full defend in the run reported above). Additionally, the proxy's semantic auditor, the adversarial attacker, and the effectiveness judge are all instantiated from the same underlying Gemma 4 model, which may understate the difficulty of true black-box red-teaming. Future work should run each objective across multiple trials to report bypass rates with confidence intervals, and substitute an independent model for at least one of the three roles.
|
| 56 |
+
|
| 57 |
---
|
| 58 |
|
| 59 |
<div style="background-color: #f8f9fa; border: 1px solid #e9ecef; border-left: 5px solid #1e8e3e; padding: 20px; border-radius: 8px; font-family: sans-serif; margin: 20px 0;">
|