X3Guard Banner

X3Guard-150M is Axiomic Labs' compact prompt guard, fine-tuned from GPT-X3-150M. It reads a user prompt and returns two independent scores: how likely the prompt is to be harmful, and how likely it is to be adversarial (a jailbreak or prompt-injection attempt). At 149M parameters it is both the smallest and best-performing model we evaluated that gives both classifications.

Performance

Throughout this model card, all X3Guard figures are at the default "Balanced" setting unless stated otherwise. Each competitor was benchmarked at its own default decision rule/threshold.

Harm Detection vs Size

Mean harm F1 vs model size

X3Guard-150M exhibits good performance for its size. It exhibits similar performance to GLiGuard-300M, and outperforms Horizon-Labs' content-safety-guard-small at the same size. It scores higher than got-oss-safeguard-20b, Llama Guard 3 1B, and Llama Guard 4 12B.

Granite-Guardian-3.1-2B leads at 0.865, about 17x X3Guard's size, and Qwen3Guard-Gen-0.6B (strict) scores 0.843 at 4x X3Guard's size.

Organization Model Parameters Mean F1 WildGuardTest F1 CircleGuardBench F1
IBM Granite Guardian 3.1 2B 2,534M 0.865 0.794 0.935
Qwen Qwen3Guard-Gen 0.6B (strict) 596M 0.843 0.885 0.801
ToxicityPrompts PolyGuard Qwen Smol 494M 0.834 0.873 0.796
Fastino GLiGuard 300M 208M 0.827 0.862 0.793
Axiomic Labs X3Guard 150M (Balanced) 149M 0.818 0.874 0.761
hbseong HarmAug Guard 435M 0.812 0.876 0.748
Qwen Qwen3Guard-Gen 0.6B (loose) 596M 0.786 0.863 0.710
OpenAI gpt-oss-safeguard-20b† ~21,000M 0.762 0.826 0.699
Horizon-Labs content-safety-guard-small 141M 0.758 0.806 0.710
Meta Llama Guard 3 1B 1,498M 0.692 0.750 0.635
Meta Llama Guard 4 12B ~12,000M 0.662 0.741 0.582
domenicrosati deberta-v3-xsmall-beavertails-harmful-qa-classifier 71M 0.631 0.643 0.618
enguard small-guard-32m-en-prompt-safety-binary-nvidia-aegis 33M 0.563 0.508 0.617
DuoGuard DuoGuard 0.5B 494M 0.543 0.629 0.457
Intel toxic-prompt-roberta 125M 0.511 0.510 0.512

Adversarial Detection vs Size

Mean adversarial F1 vs model size

In adversarial detection, X3Guard 150M ranks fifth, behind Sheltron, ProtectAI's two detectors (these were trained on jackhhao, one of the two benchmarks) and DuoGuard 0.5B.

X3Guard scores ahead of gpt-oss-safeguard-20b, both Llama Prompt Guard 2 models, NeuralTrust prompt-guard-oss-small, IBM's Granite Guardian 3.1 2B and others.

Organization Model Parameters Mean F1 jackhhao F1 TrustAIRLab F1
Sheltron AI prompt-guard-68m 68M 0.822 0.919 0.725
ProtectAI deberta-v3-small-prompt-injection-v2* 142M 0.738 0.868 0.609
ProtectAI deberta-v3-base-prompt-injection-v2* 184M 0.720 0.900 0.539
DuoGuard DuoGuard 0.5B 494M 0.710 0.830 0.589
Axiomic Labs X3Guard 150M (Balanced) 149M 0.673 0.931 0.416
OpenAI gpt-oss-safeguard-20b† ~21,000M 0.663 0.877 0.448
Qwen Qwen3Guard-Gen 0.6B (strict) 596M 0.648 0.944 0.353
Fastino GLiGuard 300M 208M 0.617 0.800 0.435
Meta Llama Prompt Guard 2 86M 279M 0.605 0.977 0.232
Meta Llama Prompt Guard 2 22M 71M 0.602 0.950 0.254
NeuralTrust prompt-guard-oss-small 141M 0.565 0.831 0.299
IBM Granite Guardian 3.1 2B 2,534M 0.561 0.920 0.202
marklkelly bert-tiny-injection-detector 4M 0.554 0.797 0.311

* Trained on jackhhao, one of the two adversarial benchmarks, according to its model card, so its jackhhao score is not held-out.

Harm vs Adversarial

Mean harm F1 vs mean adversarial F1

Among models that output both classifications, no model is ahead of X3Guard 150M on both. Granite Guardian 3.1 2B and Qwen3Guard-Gen 0.6B (strict) are ahead on harm but behind on adversarial, DuoGuard 0.5B is ahead on adversarial but far behind on harm, and gpt-oss-safeguard-20b is behind on both.

Notes on the comparison

  • Mean F1 is the mean across the two benchmarks for each task, weighted equally.
  • Parameter counts include embeddings, so they can differ from the size in a model's name: Llama Prompt Guard 2 86M and 22M count only their transformer layers, and their large vocabulary embedding tables make them 279M and 71M, respectively.
  • † gpt-oss-safeguard-20b classifies against a written policy, so its results depend on the policy; we wrote a general policy which aligned to the test approach, defining harmful and adversarial prompts, and used low reasoning effort.
  • Granite Guardian 3.1 2B was run in 8-bit, with prompts over 2,048 tokens trimmed to their first 1,536 and last 512 tokens, due to compute limitations. Few prompts in our test set exceed 2,048 tokens.
  • Sheltron was scored with its own input format and calibration, using overlapping windows for long prompts as its card recommends.
  • Streaming moderators such as Qwen3Guard-Stream are not included, as X3Guard checks complete prompts.
  • Benchmark prompts and close variants were screened out of X3Guard's training data.

Sensitivity Settings

X3Guard 150M ships with three "strictness" settings. Each applies its own threshold to each output, and a prompt is flagged as harmful or adversarial when its score is at or above that threshold. Every figure outside this section uses Balanced.

Setting Harm threshold Adversarial threshold Good for
Strict 0.1 0.3 Where false negatives are unacceptable, with the tradeoff being more false positives. Best for moderation tasks where harmful requests must be stopped whenever possible.
Balanced (default) 0.5 0.5 General chatbots and assistants.
Permissive 0.7 0.9 Use cases where false positives must be kept to a minimum, with the tradeoff being more false negatives. Role-play and creative writing apps, developer tools, security research

Recall and false-positive rate at each setting

Strict vs Balanced vs Permissive

Harm head

Setting Recall (harmful caught) Harmful missed Benign passed Benign flagged Precision F1
Strict 82.5% 17.5% 84.2% 15.8% 85.5% 0.832
Balanced 74.7% 25.3% 93.7% 6.3% 92.7% 0.818
Permissive 69.8% 30.2% 95.3% 4.7% 94.2% 0.789

Adversarial head

Setting Recall (attacks caught) Attacks missed Benign passed Benign flagged Precision F1
Strict 90.0% 10.0% 87.0% 13.0% 60.3% 0.661
Balanced 86.5% 13.5% 89.7% 10.3% 63.0% 0.673
Permissive 78.5% 21.5% 93.4% 6.6% 66.9% 0.683

These rates are per head. If your application blocks when either head fires, the combined false-flag rate will be higher.

Our headline claims use Balanced (the standard 0.5 threshold). The Strict and Permissive thresholds were chosen with these benchmark results, so treat their numbers as a guide to the trade-off rather than independent claims, and ideally, test the settings on your own traffic before selecting a setting.


Architecture

Component Details
Base model GPT-X3-150M (TX5 hybrid)
Parameters 149M
Layers 24 (18 GDN2 + 6 gated GQA)
Hidden size 576
Tokenizer LFM2.5 65k
Outputs Two independent sigmoid heads: harm_score, adversarial_score
Max prompt length 1,024 tokens
Categories None; the model does not return harm categories

Longer prompts keep their first 768 and last 255 tokens, and the middle is dropped. Alternatively, you could use a sliding-window approach to scan the whole prompt. Future Axiomic Guard models will aim to increase this maximum length.

Speed

Measured on an NVIDIA RTX 4080:

Measure Result
Throughput, batch size 32 84.5 prompts per second
Median latency, one prompt at a time 33.3 ms (95th percentile: 43.3 ms)
Peak GPU memory, one prompt at a time 1.02 GB

These figures are for the Transformers (PyTorch) version.

If you are running X3Guard on a CPU, or on individual prompts (rather than batch processing) on a GPU, we recommend the ONNX quants here.


Training

X3Guard 150M was fine-tuned from GPT-X3-150M with a two-output classification head, on a curated mix of real user prompts, public safety and jailbreak datasets, synthetic prompts and AlignmentArena data. Every source was screened against the four benchmarks above before training, and prompts matching or closely resembling a benchmark prompt were removed.

Dataset Used for
WildGuardMix (train split only) Harmful, benign and adversarial prompts
WildChat-1M and WildChat-4.8M Real user prompts, including in-the-wild jailbreaks
LMSYS-Chat-1M Real user prompts, including harmful and role-play requests
Aegis 2.0 Harmful and benign prompts with human labels
WildJailbreak (harmful subsets) Harmful requests, plain and wrapped in jailbreaks
OR-Bench Benign prompts that look harmful, to reduce over-flagging
HackAPrompt Prompt-injection attacks
Gandalf (ignore instructions, summarization) Prompt-injection attacks
deepset prompt-injections Prompt-injection attacks and benign prompts
Synthetic prompts (Axiomic Labs) Paired harmful and benign variants, obfuscated requests and hard negatives
AlignmentArena (private) Jailbreak prompts written by players

The AlignmentArena data is currently private. It contains many highly effective jailbreak prompts, and malicious requests, so it has dual-use potential.


Limitations

  • X3-Guard checks prompts only. It does not assess model responses and does not explain why a prompt was flagged.
  • X3-Guard does not categorise harm types. Other models like Meta's Llama Guard can categorise harmful prompts into one of 14 categories. X3Guard does not do this.
  • X3-Guard is evaluated mainly in English. Future Axiomic Guard model will endeavor to cover more languages.
  • Use it as one layer of defence, alongside access controls, output filtering, human review and a wider safety policy.
  • Absolutely no warranty. Use in production at your own risk. Axiomic Labs is not responsible for incorrect outputs of this model. See Apache-2.0.

Usage

pip install "torch>=2.10" "transformers>=5.5.3,<6" safetensors tokenizers
pip install flash-linear-attention  # recommended on GPU; the model falls back to a slower path without it
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
from transformers.dynamic_module_utils import get_class_from_dynamic_module

model_id = "AxiomicLabs/X3Guard-150M"
device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForSequenceClassification.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.float32,
).to(device).eval()
classify = get_class_from_dynamic_module("inference.classify", model_id)

prompts = ["What is the capital of France?"]
print(classify(model, tokenizer, prompts))                     # Balanced (default)
print(classify(model, tokenizer, prompts, setting="strict"))   # or "permissive"

Each prompt returns a dictionary like so:

{"harmful": False, "harm_score": 0.00083, "adversarial": False, "adversarial_score": 0.00004}

classify handles tokenization, the 1,024-token limit and batching (batch_size=16 by default). The thresholds for each setting are in inference.py as SETTINGS; pass threshold= to use a custom threshold for both heads. usage.py in this repository is a runnable version of this example.

The model uses custom code, so trust_remote_code=True is required.


Benchmark Credits

Benchmark Task Source
WildGuardTest Harm Test split of WildGuardMix (AllenAI)
CircleGuardBench Harm whitecircle/circleguardbench_public (White Circle)
jackhhao Adversarial jackhhao/jailbreak-classification
TrustAIRLab Adversarial TrustAIRLab/in-the-wild-jailbreak-prompts (paper code)

CircleGuardBench results: Dataset provided by WhiteCircle.ai under the White Circle Responsible Use License.

Our thanks to the authors of these benchmarks and of the training datasets above, and to the authors of the compared models.


Compute Credits

Many thanks to Joe Le Sage, who provided his hardware for training compute.


Citation

@misc{x3guard_2026,
  title={X3Guard-150M},
  author={Axiomic Labs},
  year={2026},
  howpublished={\url{https://huggingface.co/AxiomicLabs/X3Guard-150M}},
}
Downloads last month
74
Safetensors
Model size
0.1B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for AxiomicLabs/X3Guard-150M

Finetuned
(2)
this model
Quantizations
1 model

Datasets used to train AxiomicLabs/X3Guard-150M

Space using AxiomicLabs/X3Guard-150M 1