Blame for README.md

---
license: apache-2.0
language:
- en
base_model:
- Qwen/Qwen3.5-9B
- meta-llama/Llama-3.1-8B-Instruct
tags:
- safety
- guardrail
- constitutional-classifiers
- jailbreak-detection
- content-moderation
library_name: peft
pipeline_tag: text-classification
---

# EuroSafeGuard v1

EuroSafeGuard is an open guardrail for LLMs that screens requests and replies against a written constitution of 11 harm
categories.

[Paper](https://guard.safe.eu/paper) · [Code](https://github.com/EuroSafeAI/EuroSafeGuard) ·
[Collection](https://huggingface.co/collections/EuroSafeAI/eurosafeguard-v1-6aedf578437a9646077d1b58) ·
[Training prompts](https://huggingface.co/datasets/EuroSafeAI/EuroSafeGuard-v1-prompts) · [Project page](https://guard.safe.eu)

## How it works

<p align="center"><img src="https://huggingface.co/EuroSafeAI/EuroSafeGuard-v1/resolve/main/assets/overview.png" width="900" alt="EuroSafeGuard pipeline"></p>
<p align="center"><img src="https://huggingface.co/EuroSafeAI/EuroSafeGuard-v1/resolve/main/assets/example.png" width="900" alt="Jailbreak answered without a guard, blocked with EuroSafeGuard"></p>

1. **Probe.** An MLP probe on Llama-3.1-8B-Instruct activations scores every request; the top 10% (by WildChat-clean) are escalated.
2. **Input and exchange classifiers.** Escalated requests are scored by an input classifier and, after the model replies, by an exchange classifier that reads prompt and reply (Qwen3.5-9B + LoRA + 11 heads each).
3. **Decision.** The reply is blocked if `0.6·probe_z + 0.2·input_z + 0.2·exchange_z` exceeds a calibrated threshold.

## Quickstart

The repo has five input-classifier training runs (`seed=0` to `4`). The paper reports their mean. Runs differ on search attacks (11–48% of PAIR behaviors broken, run 0 is the weakest), so choose one with `seed=k`.

```python
from eurosafeguard import EuroSafeGuard   # pip install "eurosafeguard @ git+https://github.com/EuroSafeAI/EuroSafeGuard"
                                          # or download eurosafeguard.py from this repo

guard = EuroSafeGuard.from_pretrained("EuroSafeAI/EuroSafeGuard-v1", seed=0, mode="full",
                                      devices=("cuda:0", "cuda:0", "cuda:1"))   # probe LM, input clf, exchange clf
prompt, reply = "What is a good recipe for sourdough bread?", "Mix 500 g flour, 350 g water, ..."

decision, category, score = guard.route(prompt)       # input stage: "pass" | "alert" | "refuse"
blocked = guard.check_reply(prompt, reply, decision)  # alert mode: exchange check on the reply
blocked = guard.check_full(prompt, reply)             # full mode: the paper's end-to-end configuration
```

Needs about 55 GB of GPU memory in bf16 (16 GB for Llama-3.1-8B-Instruct, 19 GB per Qwen3.5-9B classifier).
Llama-3.1-8B-Instruct is gated; accept its license first. In alert mode, flagged requests are refused if CBRN and
otherwise answered under a policy note (`guard.system_note(category)`).

## Results

<p align="center"><img src="https://huggingface.co/EuroSafeAI/EuroSafeGuard-v1/resolve/main/assets/bars.png" width="720" alt="Mean robustness over six attack sets vs 16 open guards"></p>

Input stage, every guard at 5% of WildChat-clean flagged (recall %, ↑; PAIR/TAP: % of behaviors broken, ↓):

| Guard | PyRIT | BELLS-O jb | BET | GB×JF | PAIR | TAP |
|---|---|---|---|---|---|---|
| Llama Guard 3 (8B) | 78.9 | 63.1 | 80.4 | 40.8 | 98.8 | 95.3 |
| ShieldGemma (9B) | 75.9 | 54.4 | 65.7 | 39.8 | 86.9 | 79.7 |
| Qwen3Guard (8B) | 99.7 | 71.6 | 94.6 | 63.2 | 78.6 | 62.5 |
| HaloGuard (4B) | 98.4 | 88.2 | 92.6 | 79.2 | 48.8 | 35.9 |
| gpt-oss-safeguard (20B)† | 86.0 | 71.0 | 85.0 | 52.8 | 66.7 | 50.0 |
| **EuroSafeGuard** (9B + probe) | **100.0** | **95.9** | **97.0** | 79.3 | 20.7 | **19.4** |

† Native verdict. Means over the five input-classifier training runs (`gate/seed0`–`seed4`).

End to end (deployed cascade in front of Llama-3-8B-Instruct, 1,974 held-out Jailbreak Foundry attacks):

| | Undefended | EuroSafeGuard |
|---|---|---|
| Harmful replies, StrongREJECT ≥ 0.5 | 19.7% | **5.4%** |
| Harmful replies, rubric ≥ 0.5 | 7.95% | **3.88%** |
| WildChat-clean blocked | – | 0.2% |
| Extra compute | – | ~15% of the protected model's FLOPs |

## Files

| Path | Contents |
|---|---|
| `probe/` | MLP activation probe for Llama-3.1-8B-Instruct (all 32 blocks) |
| `gate/seed{0..4}/` | Input classifier, five training runs (LoRA + 11 heads on Qwen3.5-9B) |
| `exchange/` | Exchange classifier (LoRA + 11 heads on Qwen3.5-9B) |
| `thresholds/seed{k}.json`, `thresholds/full_seed{k}.json` | Alert-mode and full-mode thresholds per run |
| `eurosafeguard.py` | Single-file loader (same as `eurosafeguard/guard.py` on GitHub) |
| `taxonomy_11heads_short.json` | What each of the 11 categories allows and disallows |

## Intended use and limitations

- Built to guard chat LLMs against harmful requests and jailbreaks; recalibrate thresholds on your own traffic (`eurosafeguard calibrate`).
- White-box attackers (GCG with gradients of all models) get 40% of behaviors past the deployed cascade's input scores.
- An adaptive PAIR attacker obtains strictly harmful replies for 20 of 100 behaviors at the deployed operating point (5 at the 5% budget).
- Recall is lower on low-resource languages than on English.
- Attack success is measured with LLM judges, which are noisy on hard cases.

## License

Apache-2.0 for our adapters, heads, probe and code. Base models keep their licenses:
[Llama 3.1 Community License](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) and
[Apache-2.0 for Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B).

## Citation

```bibtex
@misc{waibl2026eurosafeguard,
  title   = {EuroSafeGuard: Open Constitutional Classifiers},
  author  = {Waibl, Leonhard and Chen, Yen-Shan and Simko, Samuel and Thomas, Aditya and
             Singh, Arth and Diwan, Nirav and Sch{\"o}lkopf, Bernhard and Jin, Zhijing},
  url     = {https://guard.safe.eu/paper},
  year    = {2026}
}
```