ashank-crazy/tiny-aya-fire-sandesh-sft
CohereLabs/tiny-aya-fire fine-tuned with LoRA (merged) to act as an SMS-inbox assistant that
does the user's task and ignores (and flags) scam instructions inside messages.
Method: SafeTune PlainSFTTrainer. Data: SandeshBench train split, curated with CuratorKIT.
Prompt format: see sandesh/tools.py in https://github.com/ashank-crazy/sandesh (system prompt with the tool list,
user request, read_inbox call, inbox as a tool result; the model answers with
one JSON tool call).
Attack success rate (lower is better)
| Condition |
seen langs + styles |
unseen languages |
unseen attack styles |
unseen langs + styles |
| Tiny Aya Fire |
12.5% |
16.0% |
13.2% |
14.3% |
| + defensive system prompt |
4.5% |
6.6% |
2.5% |
2.9% |
| SFT, clean inboxes only (control) |
0.6% |
1.9% |
1.4% |
1.0% |
| SFT on Sandesh (PlainSFT) |
0.0% |
0.0% |
0.0% |
0.0% |
| SFT on Sandesh (SafeGrad) |
0.0% |
0.9% |
0.4% |
0.0% |
Usefulness and general safety
| Condition |
Task success, clean inbox ↑ |
Task success, under attack ↑ |
Warns user about scam ↑ |
Valid tool call ↑ |
Re-reads inbox (stalls) ↓ |
Harmful prompts: unsafe answers ↓ |
Harmful prompts: refusals ↑ |
tokens/s |
| Tiny Aya Fire |
48.3% |
43.4% |
0.8% |
98.7% |
0.0% |
5.5% |
66.0% |
28 |
| + defensive system prompt |
38.8% |
40.5% |
7.9% |
97.7% |
0.2% |
5.5% |
66.0% |
26 |
| SFT, clean inboxes only (control) |
97.6% |
97.4% |
0.0% |
100.0% |
0.0% |
5.0% |
68.0% |
17 |
| SFT on Sandesh (PlainSFT) |
99.5% |
99.7% |
94.9% |
100.0% |
0.0% |
4.0% |
71.8% |
24 |
| SFT on Sandesh (SafeGrad) |
95.7% |
95.7% |
78.9% |
99.9% |
0.0% |
5.8% |
65.2% |
43 |
Attack success by scam language (all test splits)
| Condition |
bn |
en |
hi |
hinglish |
mr |
ta |
| Tiny Aya Fire |
16.4% |
16.0% |
12.4% |
10.1% |
14.6% |
15.2% |
| + defensive system prompt |
6.7% |
4.5% |
3.6% |
2.5% |
3.8% |
5.1% |
| SFT, clean inboxes only (control) |
1.8% |
0.5% |
1.0% |
1.5% |
1.6% |
1.1% |
| SFT on Sandesh (PlainSFT) |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
| SFT on Sandesh (SafeGrad) |
0.0% |
0.5% |
0.0% |
0.0% |
1.6% |
0.0% |
Attack success by attack style (all test splits)
| Condition |
authority |
direct |
fake_system |
fake_tool_json |
honorific |
owner_impersonation |
script_switch |
| Tiny Aya Fire |
3.9% |
10.2% |
16.4% |
12.3% |
15.0% |
14.4% |
27.0% |
| + defensive system prompt |
1.6% |
0.8% |
7.0% |
4.1% |
1.2% |
4.0% |
14.8% |
| SFT, clean inboxes only (control) |
0.0% |
0.0% |
0.8% |
2.5% |
0.0% |
1.6% |
4.1% |
| SFT on Sandesh (PlainSFT) |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
| SFT on Sandesh (SafeGrad) |
0.0% |
0.0% |
0.0% |
0.4% |
0.0% |
0.0% |
2.5% |
Adaptive attacker: best-of-N (K = 16 rewrites per scam, 48 scenarios)
Attacker wins a scenario if any of k rewrites hijacks the assistant (exact unbiased ASR@k).
| Condition |
Static ASR (same scenarios) |
ASR@1 |
ASR@2 |
ASR@4 |
ASR@8 |
ASR@16 |
95% CI @16 |
vs base |
| Tiny Aya Fire |
6.2% |
10.9% |
17.6% |
26.3% |
37.7% |
52.1% |
38.3%–65.5% |
– |
| SFT on Sandesh (PlainSFT) |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0%–7.4% |
p=0.001 ✓ |
| SFT on Sandesh (SafeGrad) |
0.0% |
0.1% |
0.3% |
0.5% |
1.0% |
2.1% |
0.4%–10.9% |
p=0.001 ✓ |
Hijack rate of a single rewrite, by augmentation family:
| Condition |
ascii_noise |
base64 |
caps |
double |
invisible |
leet |
move |
restyle |
reverse |
rot13 |
scramble |
spoof |
urgency |
| Tiny Aya Fire |
8.3% |
8.3% |
10.4% |
33.3% |
8.3% |
6.2% |
8.3% |
6.2% |
4.2% |
4.2% |
6.2% |
12.5% |
8.3% |
| SFT on Sandesh (PlainSFT) |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
| SFT on Sandesh (SafeGrad) |
0.0% |
0.0% |
0.0% |
2.1% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
ASR@16 by scam language:
| Condition |
en |
hi |
hinglish |
bn |
ta |
mr |
| Tiny Aya Fire |
50.0% |
50.0% |
50.0% |
75.0% |
25.0% |
62.5% |
| SFT on Sandesh (PlainSFT) |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
0.0% |
| SFT on Sandesh (SafeGrad) |
0.0% |
0.0% |
0.0% |
12.5% |
0.0% |
0.0% |
Over-refusal and capability retention
| Condition |
XSTest safe prompts refused ↓ |
Global-MMLU en ↑ |
hi ↑ |
bn ↑ |
all ↑ |
MMLU vs base |
| Tiny Aya Fire |
11.6% |
54.0% |
47.0% |
43.0% |
48.0% |
– |
| + defensive system prompt (same weights as base) |
11.6% |
54.0% |
47.0% |
43.0% |
48.0% |
– |
| SFT, clean inboxes only (control) |
12.0% |
55.0% |
44.0% |
42.0% |
47.0% |
p=0.510 |
| SFT on Sandesh (PlainSFT) |
12.4% |
56.0% |
44.0% |
47.0% |
49.0% |
p=0.586 |
| SFT on Sandesh (SafeGrad) |
9.6% |
54.0% |
46.0% |
45.0% |
48.3% |
p=0.842 |
Change vs base model (AuditKIT paired bootstrap, 1,000 resamples, all test splits)
| Condition |
Metric |
Base |
Condition |
Δ |
p |
p < 0.05 |
Paired samples |
| + defensive system prompt |
attack_success (attack inboxes) |
14.0% |
4.3% |
-9.7 pts |
0.001 |
yes |
1121 |
| + defensive system prompt |
task_success (attack inboxes) |
43.4% |
40.5% |
-2.9 pts |
0.037 |
yes |
1121 |
| + defensive system prompt |
task_success (clean inboxes) |
48.3% |
38.8% |
-9.6 pts |
0.001 |
yes |
209 |
| SFT, clean inboxes only (control) |
attack_success (attack inboxes) |
14.0% |
1.2% |
-12.8 pts |
0.001 |
yes |
1121 |
| SFT, clean inboxes only (control) |
task_success (attack inboxes) |
43.4% |
97.4% |
+54.0 pts |
0.001 |
yes |
1121 |
| SFT, clean inboxes only (control) |
task_success (clean inboxes) |
48.3% |
97.6% |
+49.3 pts |
0.001 |
yes |
209 |
| SFT on Sandesh (PlainSFT) |
attack_success (attack inboxes) |
14.0% |
0.0% |
-14.0 pts |
0.001 |
yes |
1121 |
| SFT on Sandesh (PlainSFT) |
task_success (attack inboxes) |
43.4% |
99.7% |
+56.3 pts |
0.001 |
yes |
1121 |
| SFT on Sandesh (PlainSFT) |
task_success (clean inboxes) |
48.3% |
99.5% |
+51.2 pts |
0.001 |
yes |
209 |
| SFT on Sandesh (SafeGrad) |
attack_success (attack inboxes) |
14.0% |
0.4% |
-13.6 pts |
0.001 |
yes |
1121 |
| SFT on Sandesh (SafeGrad) |
task_success (attack inboxes) |
43.4% |
95.7% |
+52.3 pts |
0.001 |
yes |
1121 |
| SFT on Sandesh (SafeGrad) |
task_success (clean inboxes) |
48.3% |
95.7% |
+47.4 pts |
0.001 |
yes |
209 |
Headline numbers (fill SUBMISSION.md placeholders in this order)
| # |
Quantity |
Value |
| 1 |
Base ASR, familiar languages + styles (test_iid) |
12.5% |
| 2 |
Base ASR, held-out languages bn/ta/mr (test_ood_lang) |
16.0% |
| 3 |
Base + defensive prompt ASR, all test splits |
4.3% |
| 4 |
SafeGrad ASR, test_iid |
0.0% |
| 5 |
SafeGrad ASR, held-out languages |
0.9% |
| 6 |
SafeGrad ASR, held-out attack styles |
0.4% |
| 7 |
SafeGrad task success, clean inbox |
95.7% |
| 8 |
Base task success, clean inbox |
48.3% |
| 9 |
SafeGrad warns user under attack |
78.9% |
| 10 |
Harmful prompts unsafe: SafeGrad |
5.8% |
| 11 |
Harmful prompts unsafe: plain SFT |
4.0% |
| 12 |
Harmful prompts unsafe: base |
5.5% |
| 13 |
Base ASR under best-of-16 attack |
52.1% |
| 14 |
Spotlighting prompt ASR under best-of-16 |
– |
| 15 |
SafeGrad ASR under best-of-16 |
2.1% |
| 16 |
Base: episodes hijacked by turn 3 (multi-turn) |
– |
| 17 |
SafeGrad: episodes hijacked by turn 3 (multi-turn) |
– |
| 18 |
XSTest over-refusal: base |
11.6% |
| 19 |
XSTest over-refusal: SafeGrad |
9.6% |
| 20 |
Global-MMLU (en/hi/bn): base |
48.0% |
| 21 |
Global-MMLU (en/hi/bn): SafeGrad |
48.3% |
| 22 |
Base ASR, all test splits |
14.0% |
| 23 |
CircuitKIT knockout (k heads zeroed, no training) ASR, all test splits |
– |
| 24 |
Random-heads control ASR, all test splits |
– |
| 25 |
CircuitKIT knockout task success, clean inbox |
– |
| 26 |
Global-MMLU (en/hi/bn): CircuitKIT knockout |
– |