TinyGuard-Injection (base + fine-tuned)

Bilingual prompt-injection detector (incl. indirect injection) trained completely from scratch · 9.1M parameters · MLX/Metal · Apache-2.0 (code) · fully synthetic training data

中文说明

Model description

A 3-way guardrail deployed at LLM-app input gates and RAG retrieval gates: benign / direct injection / indirect injection. Complements Meta Llama Prompt Guard 2 (22M/86M, no Chinese support): this model is trained bilingual with equal weight.

• Backbone: shared TinyGuard-Encoder (same pretrained base as TinyGuard-Command)
• Fine-tune: 12,330 bilingual labeled examples across six carriers (code & collaboration platforms, emails & web pages, long RAG chunks, tickets & customer-service dialogs, benign hard negatives, hidden-character & encoded payloads), with "minimal-difference pairs" and anti-overrefusal design

Results (held-out data only, never seen in training)

Benchmark	Metric	Value
Validation set, 616 rows	acc / macro-F1	99.0% / 0.990
Adversarial set, 40 rows (zero-width chars / JSON rules / Japanese carriers / quoted disclaimers)	intervention recall / precision / F1	75% / 88% / 0.81

Reference: Llama Prompt Guard 2-86M reports EN injection F1 ≈ 0.98 with no Chinese. This model provides first-tier Chinese capability; its English adversarial numbers are below Meta's home turf (expected at 9.1M vs 86M).

Usage

Weights: QuanxinLi/TinyGuard-Injection · Code & training pipeline: Quanxin-Lee/TinyGuard

# quickstart (transformers/MLX dual format) — see the GitHub repo for the full walkthrough

Limitations

• 5/40 misses on the adversarial set: JSON-encoded rules, URL-less "prioritize this" phrasing, direct asks inside authorized framing (red-team wording) — all weak-signal forms.
• English training volume is lower than Chinese; English adversarial generalization needs strengthening.
• Recommended to stack with a regex layer: this model handles semantics, regex handles known fingerprints.

Training details & data statement

All data is synthetic/hand-written — no real user data. The full training pipeline, four audit scripts (three deep-scan rounds) and every dataset ship with the repository.