File size: 2,173 Bytes
dba6ef3 ce9a6c6 dba6ef3 ce9a6c6 dba6ef3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | ---
library_name: pytorch
tags: [prompt-injection, security, secureagentnet]
---
# SecureAgentNet detector checkpoints
Checkpoints for [SecureAgentNet](https://github.com/MP-GOWTHAM/SecureAgentNet).
They live here rather than in git because five of the project's checkpoints
are 253 MB each, against GitHub's 100 MB per-file hard limit.
| Directory | What it is | Held-out AUC |
|---|---|---|
| `ensemble_v6_smooth3` | **Recommended detector.** From-scratch ensemble (char-CNN + BiLSTM-attention + scratch transformer, 12.3M params) on the 5-source corpus | **0.9168** |
| `v3` | DistilBERT after the Track B online-retraining cycle, 66M params | 0.7875 |
| `harm_detector_v3` | Content-harm classifier — a separate axis from injection | 0.8978 |
| `ensemble_v9_bal` | Same architecture on the 331k 7-source corpus. Lower FPR (0.189) and higher utility (0.811) but lower AUC | 0.8929 |
| `ensemble_v4_persona` | Superseded — previous ensemble on the 4-source corpus | 0.8278 |
| `harm_detector` | Superseded — previous harm classifier, Necent only | 0.9028 |
The recommended runtime configuration is `combined_max_v7` — the elementwise
max of `ensemble_v6_smooth3` and `v3`. The two fail in opposite directions,
and the combination is the only one measured with no known blind spot: 8/8
canonical short attacks and 8/8 real evasions, ASR 0.025, FNR 0.029. It is a
config referencing the two members, so it is recreated locally rather than
stored here.
Note that `combined_max_v7` has a *lower* AUC (0.8223) than
`ensemble_v6_smooth3` alone (0.9168), because taking a max inherits the
union of both members' false positives. AUC is the wrong criterion for this
choice — the combination exists to eliminate blind spots, not to maximise
ranking.
Download with `scripts/bootstrap_windows.ps1` in the repo, or manually:
```python
from huggingface_hub import snapshot_download
snapshot_download(repo_id="mpgowtham/secureagentnet-models", local_dir="secureagentnet/data/models")
```
See the repository's `docs/SecureAgentNet_Detector_Architecture.docx` for
the full evaluation, including the configurations that failed.
|