SecureAgentNet detector checkpoints
Checkpoints for SecureAgentNet. They live here rather than in git because five of the project's checkpoints are 253 MB each, against GitHub's 100 MB per-file hard limit.
| Directory | What it is | Held-out AUC |
|---|---|---|
ensemble_v6_smooth3 |
Recommended detector. From-scratch ensemble (char-CNN + BiLSTM-attention + scratch transformer, 12.3M params) on the 5-source corpus | 0.9168 |
v3 |
DistilBERT after the Track B online-retraining cycle, 66M params | 0.7875 |
harm_detector_v3 |
Content-harm classifier — a separate axis from injection | 0.8978 |
ensemble_v9_bal |
Same architecture on the 331k 7-source corpus. Lower FPR (0.189) and higher utility (0.811) but lower AUC | 0.8929 |
ensemble_v4_persona |
Superseded — previous ensemble on the 4-source corpus | 0.8278 |
harm_detector |
Superseded — previous harm classifier, Necent only | 0.9028 |
The recommended runtime configuration is combined_max_v7 — the elementwise
max of ensemble_v6_smooth3 and v3. The two fail in opposite directions,
and the combination is the only one measured with no known blind spot: 8/8
canonical short attacks and 8/8 real evasions, ASR 0.025, FNR 0.029. It is a
config referencing the two members, so it is recreated locally rather than
stored here.
Note that combined_max_v7 has a lower AUC (0.8223) than
ensemble_v6_smooth3 alone (0.9168), because taking a max inherits the
union of both members' false positives. AUC is the wrong criterion for this
choice — the combination exists to eliminate blind spots, not to maximise
ranking.
Download with scripts/bootstrap_windows.ps1 in the repo, or manually:
from huggingface_hub import snapshot_download
snapshot_download(repo_id="mpgowtham/secureagentnet-models", local_dir="secureagentnet/data/models")
See the repository's docs/SecureAgentNet_Detector_Architecture.docx for
the full evaluation, including the configurations that failed.