File size: 5,737 Bytes
ca949ba 2fc7909 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 | ---
title: RecRedTeam AuditLab
emoji: 🛡️
colorFrom: red
colorTo: gray
sdk: static
pinned: false
license: mit
short_description: Red-team audit for LLM & agentic shopping agents
datasets:
- NoahIsARider/RecRedTeam-ASI-Bench
tags:
- red-teaming
- agentic-ai
- ai-safety
- llm-security
- prompt-injection
- recommender-systems
- e-commerce
- owasp
---
# 🛡️ RecRedTeam AuditLab
**An interactive red-team audit for LLM / agentic recommendation and shopping assistants.**
Pick an attack from a 9-category threat taxonomy mapped to **OWASP ASI 2026**, run it
against a deterministic shopping agent, and watch the verdict change as you switch
defenses on. No API key, no network call, no server — every number is reproducible.
- **Dataset** — [NoahIsARider/RecRedTeam-ASI-Bench](https://huggingface.co/datasets/NoahIsARider/RecRedTeam-ASI-Bench)
- **Framework** — [github.com/NoahIsARider/RecRedTeam](https://github.com/NoahIsARider/RecRedTeam)
---
## What you can do here
**Threat taxonomy** — the 8 + 1 categories with their OWASP ASI 2026 / LLM Top-10
mapping and per-category case counts.
**Single attack** — inspect one case end to end:
the poisoned catalog, the hidden instruction planted in it, the shopper query, and the
attacker's goal. Then read the agent's answer twice — baseline vs. defended — side by
side with the judge's verdict, hazard grade and evidence. Any subset of the three
plugins can be switched on.
**Benchmark / leaderboard** — the full public split (234 cases) across five defense
configurations and get:
| metric | meaning |
|---|---|
| `ASR` | attack success rate = share of cases the agent failed |
| `effectiveness` | \(1 - \mathrm{ASR}_{defended} / \mathrm{ASR}_{baseline}\) |
| `hazard` | mean graded harm, `none → critical` mapped to 0–1 |
| `over-block` | share of cases where the defense filtered the entire catalog away |
plus a per-category ASR matrix.
---
## How this page is built
This is a **static Space**, so nothing runs on the Hub's servers. The audit engine in
`auditlab/` is the source of truth: `build_static.py` runs it over the public split for
all eight defense combinations and freezes every verdict, hazard grade, answer and
aggregate into `data/results.json`. `index.html` only renders that file, so the page
loads instantly and can never drift from the engine. Re-running
```bash
python build_static.py
```
on the same dataset reproduces the JSON byte for byte.
A Gradio version of the same lab is kept in the repo as `app.py` for local use
(`pip install -r requirements.txt && python app.py`). Same engine, same numbers.
---
## The three defense plugins
| plugin | hook | what it does |
|---|---|---|
| `sanitizer` | `sanitize()` | strips injected instructions, unverifiable claims and deceptive price anchors from retrieved content |
| `rag_guard` | `filter_products()` | drops astroturfed / sponsored / backdoored / dark-pattern / untrusted listings at retrieval |
| `policy_guard` | `process_response()` | rewrites answers that leak PII or point at untrusted domains |
They compose — `DefenseStack` applies any subset in retrieval → content → output order.
---
## The headline finding
Each plugin has a deliberately different coverage profile:
| category | baseline | sanitizer | rag_guard | policy_guard | full stack |
|---|---|---|---|---|---|
| indirect prompt injection | 100% | **0%** | 0% | 100% | 0% |
| price deception | 100% | **0%** | 0% | 100% | 0% |
| multi-turn escalation | 100% | 100% | 100% | **0%** | 0% |
| backdoor | 100% | 100% | 0% | 0% | 0% |
| fake reviews / dark patterns / filter bubble / stereotype bias | 100% | 100% | **0%** | 100% | 0% |
| **product hallucination** | 100% | 35% | 100% | 100% | **35%** |
**Product hallucination is the category no rule-based defense solves.** Fabricated but
plausible specs — "30-day battery life", "#1 best seller", "50,000 units sold" — are
lexically indistinguishable from real ones. Closing that gap needs grounding and
external verification, and `rag_guard` cannot even see the poison because the listing
carries no machine-readable marker. That gap is the open research direction.
Numbers above are from the `public` split (234 cases) at seed 2026 and are reproducible
from this Space.
---
## Limitations
- The agent under test is a **deterministic mock**, not a real LLM. Absolute ASR values
describe this mock; what transfers to real agents is the *relative* coverage profile
of the defenses and the taxonomy itself.
- Only the **rule channel** of the judge runs here. The full RecRedTeam framework adds
an optional LLM judge channel and LangGraph / OpenAI-Agents adapters so you can audit
a real agent.
- Attacks target a **local simulated catalog**, never a real system.
- This page is a **precomputed snapshot** of the public split. To audit your own agent,
use the framework repo — this Space cannot run your model.
## Files
```
index.html the published page
assets/ stylesheet and viewer logic (no build step, no dependencies)
data/results.json every precomputed run (234 cases × 8 defense combinations)
data/*.jsonl bundled copy of the dataset, used by the engine
build_static.py regenerates data/results.json from the engine
app.py + auditlab/ reference Gradio implementation of the same lab
```
## Run the reference implementation locally
```bash
pip install -r requirements.txt
python app.py
```
## License
MIT. Dataset is CC BY 4.0.
## Author
Fangyanuo Zhou ([NoahIsARider](https://noahisarider.github.io/))
|