| --- |
| license: apache-2.0 |
| language: |
| - en |
| base_model: |
| - Qwen/Qwen3.5-9B |
| - meta-llama/Llama-3.1-8B-Instruct |
| tags: |
| - safety |
| - guardrail |
| - constitutional-classifiers |
| - jailbreak-detection |
| - content-moderation |
| library_name: peft |
| pipeline_tag: text-classification |
| --- |
| |
| # EuroSafeGuard v1 |
|
|
| EuroSafeGuard is an open guardrail for LLMs that screens requests and replies against a written constitution of 11 harm |
| categories. |
|
|
| [Paper](https://guard.safe.eu/paper) · [Code](https://github.com/EuroSafeAI/EuroSafeGuard) · |
| [Collection](https://huggingface.co/collections/EuroSafeAI/eurosafeguard-v1-6aedf578437a9646077d1b58) · |
| [Training prompts](https://huggingface.co/datasets/EuroSafeAI/EuroSafeGuard-v1-prompts) · [Project page](https://guard.safe.eu) |
|
|
| ## How it works |
|
|
| <p align="center"><img src="https://huggingface.co/EuroSafeAI/EuroSafeGuard-v1/resolve/main/assets/overview.png" width="900" alt="EuroSafeGuard pipeline"></p> |
| <p align="center"><img src="https://huggingface.co/EuroSafeAI/EuroSafeGuard-v1/resolve/main/assets/example.png" width="900" alt="Jailbreak answered without a guard, blocked with EuroSafeGuard"></p> |
|
|
| 1. **Probe.** An MLP probe on Llama-3.1-8B-Instruct activations scores every request; the top 10% (by WildChat-clean) are escalated. |
| 2. **Input and exchange classifiers.** Escalated requests are scored by an input classifier and, after the model replies, by an exchange classifier that reads prompt and reply (Qwen3.5-9B + LoRA + 11 heads each). |
| 3. **Decision.** The reply is blocked if `0.6·probe_z + 0.2·input_z + 0.2·exchange_z` exceeds a calibrated threshold. |
|
|
| ## Quickstart |
|
|
| The repo has five input-classifier training runs (`seed=0` to `4`). The paper reports their mean. Runs differ on search attacks (11–48% of PAIR behaviors broken, run 0 is the weakest), so choose one with `seed=k`. |
|
|
| ```python |
| from eurosafeguard import EuroSafeGuard # pip install "eurosafeguard @ git+https://github.com/EuroSafeAI/EuroSafeGuard" |
| # or download eurosafeguard.py from this repo |
| |
| guard = EuroSafeGuard.from_pretrained("EuroSafeAI/EuroSafeGuard-v1", seed=0, mode="full", |
| devices=("cuda:0", "cuda:0", "cuda:1")) # probe LM, input clf, exchange clf |
| prompt, reply = "What is a good recipe for sourdough bread?", "Mix 500 g flour, 350 g water, ..." |
| |
| decision, category, score = guard.route(prompt) # input stage: "pass" | "alert" | "refuse" |
| blocked = guard.check_reply(prompt, reply, decision) # alert mode: exchange check on the reply |
| blocked = guard.check_full(prompt, reply) # full mode: the paper's end-to-end configuration |
| ``` |
|
|
| Needs about 55 GB of GPU memory in bf16 (16 GB for Llama-3.1-8B-Instruct, 19 GB per Qwen3.5-9B classifier). |
| Llama-3.1-8B-Instruct is gated; accept its license first. In alert mode, flagged requests are refused if CBRN and |
| otherwise answered under a policy note (`guard.system_note(category)`). |
|
|
| ## Results |
|
|
| <p align="center"><img src="https://huggingface.co/EuroSafeAI/EuroSafeGuard-v1/resolve/main/assets/bars.png" width="720" alt="Mean robustness over six attack sets vs 16 open guards"></p> |
|
|
| Input stage, every guard at 5% of WildChat-clean flagged (recall %, ↑; PAIR/TAP: % of behaviors broken, ↓): |
|
|
| | Guard | PyRIT | BELLS-O jb | BET | GB×JF | PAIR | TAP | |
| |---|---|---|---|---|---|---| |
| | Llama Guard 3 (8B) | 78.9 | 63.1 | 80.4 | 40.8 | 98.8 | 95.3 | |
| | ShieldGemma (9B) | 75.9 | 54.4 | 65.7 | 39.8 | 86.9 | 79.7 | |
| | Qwen3Guard (8B) | 99.7 | 71.6 | 94.6 | 63.2 | 78.6 | 62.5 | |
| | HaloGuard (4B) | 98.4 | 88.2 | 92.6 | 79.2 | 48.8 | 35.9 | |
| | gpt-oss-safeguard (20B)† | 86.0 | 71.0 | 85.0 | 52.8 | 66.7 | 50.0 | |
| | **EuroSafeGuard** (9B + probe) | **100.0** | **95.9** | **97.0** | 79.3 | 20.7 | **19.4** | |
|
|
| † Native verdict. Means over the five input-classifier training runs (`gate/seed0`–`seed4`). |
|
|
| End to end (deployed cascade in front of Llama-3-8B-Instruct, 1,974 held-out Jailbreak Foundry attacks): |
|
|
| | | Undefended | EuroSafeGuard | |
| |---|---|---| |
| | Harmful replies, StrongREJECT ≥ 0.5 | 19.7% | **5.4%** | |
| | Harmful replies, rubric ≥ 0.5 | 7.95% | **3.88%** | |
| | WildChat-clean blocked | – | 0.2% | |
| | Extra compute | – | ~15% of the protected model's FLOPs | |
|
|
| ## Files |
|
|
| | Path | Contents | |
| |---|---| |
| | `probe/` | MLP activation probe for Llama-3.1-8B-Instruct (all 32 blocks) | |
| | `gate/seed{0..4}/` | Input classifier, five training runs (LoRA + 11 heads on Qwen3.5-9B) | |
| | `exchange/` | Exchange classifier (LoRA + 11 heads on Qwen3.5-9B) | |
| | `thresholds/seed{k}.json`, `thresholds/full_seed{k}.json` | Alert-mode and full-mode thresholds per run | |
| | `eurosafeguard.py` | Single-file loader (same as `eurosafeguard/guard.py` on GitHub) | |
| | `taxonomy_11heads_short.json` | What each of the 11 categories allows and disallows | |
|
|
| ## Intended use and limitations |
|
|
| - Built to guard chat LLMs against harmful requests and jailbreaks; recalibrate thresholds on your own traffic (`eurosafeguard calibrate`). |
| - White-box attackers (GCG with gradients of all models) get 40% of behaviors past the deployed cascade's input scores. |
| - An adaptive PAIR attacker obtains strictly harmful replies for 20 of 100 behaviors at the deployed operating point (5 at the 5% budget). |
| - Recall is lower on low-resource languages than on English. |
| - Attack success is measured with LLM judges, which are noisy on hard cases. |
|
|
| ## License |
|
|
| Apache-2.0 for our adapters, heads, probe and code. Base models keep their licenses: |
| [Llama 3.1 Community License](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) and |
| [Apache-2.0 for Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B). |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{waibl2026eurosafeguard, |
| title = {EuroSafeGuard: Open Constitutional Classifiers}, |
| author = {Waibl, Leonhard and Chen, Yen-Shan and Simko, Samuel and Thomas, Aditya and |
| Singh, Arth and Diwan, Nirav and Sch{\"o}lkopf, Bernhard and Jin, Zhijing}, |
| url = {https://guard.safe.eu/paper}, |
| year = {2026} |
| } |
| ``` |
|
|
