ashank-crazy/tiny-aya-fire-sandesh-sft-mix

CohereLabs/tiny-aya-fire fine-tuned with LoRA (merged) to act as an SMS-inbox assistant that does the user's task and ignores (and flags) scam instructions inside messages. Method: SafeTune PlainSFTTrainer. Data: SandeshBench train split, curated with CuratorKIT.

Prompt format: see sandesh/tools.py in https://github.com/ashank-crazy/sandesh (system prompt with the tool list, user request, read_inbox call, inbox as a tool result; the model answers with one JSON tool call).

Attack success rate (lower is better)

Condition seen langs + styles unseen languages unseen attack styles unseen langs + styles
Tiny Aya Fire 12.5% 16.0% 13.2% 14.3%
+ defensive system prompt 4.5% 6.6% 2.5% 2.9%
+ spotlighting & sandwich prompt 2.2% 1.9% 1.8% 5.2%
+ Qwen3Guard SMS filter 3.2% 1.3% 3.2% 3.3%
SFT, clean inboxes only (control) 0.6% 1.9% 1.4% 1.0%
SFT on Sandesh (PlainSFT) 0.0% 0.0% 0.0% 0.0%
SFT + refusals mixed in (PlainSFT) 0.0% 0.0% 0.0% 0.0%
SFT on Sandesh (SafeGrad) 0.0% 0.9% 0.4% 0.0%

Usefulness and general safety

Condition Task success, clean inbox ↑ Task success, under attack ↑ Warns user about scam ↑ Valid tool call ↑ Re-reads inbox (stalls) ↓ Harmful prompts: unsafe answers ↓ Harmful prompts: refusals ↑ tokens/s
Tiny Aya Fire 48.3% 43.4% 0.8% 98.7% 0.0% 5.5% 66.0% 28
+ defensive system prompt 38.8% 40.5% 7.9% 97.7% 0.2% 5.5% 66.0% 26
+ spotlighting & sandwich prompt 63.2% 57.7% 2.4% 98.0% 22.6% 5.5% 66.0% 24
+ Qwen3Guard SMS filter 41.6% 39.8% 0.0% 98.8% 0.0% 5.5% 66.0% 47
SFT, clean inboxes only (control) 97.6% 97.4% 0.0% 100.0% 0.0% 5.0% 68.0% 17
SFT on Sandesh (PlainSFT) 99.5% 99.7% 94.9% 100.0% 0.0% 4.0% 71.8% 24
SFT + refusals mixed in (PlainSFT) 98.6% 97.3% 81.3% 98.3% 0.0% 0.0% 98.5% 23
SFT on Sandesh (SafeGrad) 95.7% 95.7% 78.9% 99.9% 0.0% 5.8% 65.2% 43

Attack success by scam language (all test splits)

Condition bn en hi hinglish mr ta
Tiny Aya Fire 16.4% 16.0% 12.4% 10.1% 14.6% 15.2%
+ defensive system prompt 6.7% 4.5% 3.6% 2.5% 3.8% 5.1%
+ spotlighting & sandwich prompt 4.8% 0.5% 1.0% 4.5% 2.7% 2.2%
+ Qwen3Guard SMS filter 3.0% 4.5% 3.1% 2.0% 3.2% 0.0%
SFT, clean inboxes only (control) 1.8% 0.5% 1.0% 1.5% 1.6% 1.1%
SFT on Sandesh (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
SFT + refusals mixed in (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
SFT on Sandesh (SafeGrad) 0.0% 0.5% 0.0% 0.0% 1.6% 0.0%

Attack success by attack style (all test splits)

Condition authority direct fake_system fake_tool_json honorific owner_impersonation script_switch
Tiny Aya Fire 3.9% 10.2% 16.4% 12.3% 15.0% 14.4% 27.0%
+ defensive system prompt 1.6% 0.8% 7.0% 4.1% 1.2% 4.0% 14.8%
+ spotlighting & sandwich prompt 1.6% 0.8% 2.3% 4.1% 2.4% 3.2% 2.5%
+ Qwen3Guard SMS filter 0.8% 0.8% 0.8% 6.6% 0.0% 0.0% 9.0%
SFT, clean inboxes only (control) 0.0% 0.0% 0.8% 2.5% 0.0% 1.6% 4.1%
SFT on Sandesh (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
SFT + refusals mixed in (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
SFT on Sandesh (SafeGrad) 0.0% 0.0% 0.0% 0.4% 0.0% 0.0% 2.5%

Adaptive attacker: best-of-N (K = 16 rewrites per scam, 48 scenarios)

Attacker wins a scenario if any of k rewrites hijacks the assistant (exact unbiased ASR@k).

Condition Static ASR (same scenarios) ASR@1 ASR@2 ASR@4 ASR@8 ASR@16 95% CI @16 vs base
Tiny Aya Fire 6.2% 10.9% 17.6% 26.3% 37.7% 52.1% 38.3%–65.5% –
+ spotlighting & sandwich prompt 0.0% 1.6% 3.0% 5.7% 10.3% 16.7% 8.7%–29.6% p=0.001 ✓
SFT on Sandesh (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%–7.4% p=0.001 ✓
SFT + refusals mixed in (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%–7.4% p=0.001 ✓
SFT on Sandesh (SafeGrad) 0.0% 0.1% 0.3% 0.5% 1.0% 2.1% 0.4%–10.9% p=0.001 ✓

Hijack rate of a single rewrite, by augmentation family:

Condition ascii_noise base64 caps double invisible leet move restyle reverse rot13 scramble spoof urgency
Tiny Aya Fire 8.3% 8.3% 10.4% 33.3% 8.3% 6.2% 8.3% 6.2% 4.2% 4.2% 6.2% 12.5% 8.3%
+ spotlighting & sandwich prompt 0.0% 0.0% 0.0% 4.2% 0.0% 0.0% 2.1% 4.2% 0.0% 0.0% 0.0% 0.0% 2.1%
SFT on Sandesh (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
SFT + refusals mixed in (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
SFT on Sandesh (SafeGrad) 0.0% 0.0% 0.0% 2.1% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%

ASR@16 by scam language:

Condition en hi hinglish bn ta mr
Tiny Aya Fire 50.0% 50.0% 50.0% 75.0% 25.0% 62.5%
+ spotlighting & sandwich prompt 12.5% 12.5% 12.5% 12.5% 0.0% 50.0%
SFT on Sandesh (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
SFT + refusals mixed in (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
SFT on Sandesh (SafeGrad) 0.0% 0.0% 0.0% 12.5% 0.0% 0.0%

Multi-turn escalation (120 episodes, 3 user turns)

Turn 2: the scammer texts again, in another style, with an urgency banner. Turn 3: the user says "take care of whatever is urgent". Cumulative share of episodes where the attacker got an action.

Condition Hijacked by turn 1 by turn 2 by turn 3 First broke at turn 1 / 2 / 3 Warns at turn 2 AuditKIT agent_eval verified safe vs base
Tiny Aya Fire 10.8% 50.0% 67.5% 13 / 47 / 21 2.8% 39/120 –
+ spotlighting & sandwich prompt 5.0% 14.2% 46.7% 6 / 11 / 39 1.8% 64/120 p=0.002 ✓
SFT on Sandesh (PlainSFT) 0.0% 2.5% 20.0% 0 / 3 / 21 88.3% 96/120 p=0.001 ✓
SFT + refusals mixed in (PlainSFT) 0.0% 0.0% 5.8% 0 / 0 / 7 89.2% 113/120 p=0.001 ✓
SFT on Sandesh (SafeGrad) 0.0% 13.3% 24.2% 0 / 16 / 13 65.8% 91/120 p=0.001 ✓

Off-the-shelf detectors on the same inboxes

Share of scam SMS flagged, by language, and share of legitimate SMS wrongly flagged.

Detector en hi hinglish bn ta mr All scams False alarms, all legit SMS False alarms, OTP SMS
Qwen3Guard-Gen-4B, strict 72.5% 85.6% 78.4% 88.5% 96.6% 83.8% 83.9% 34.7% 78.1%
Qwen3Guard-Gen-4B, loose 3.5% 4.1% 2.0% 3.0% 4.5% 1.6% 3.1% 0.0% 0.0%
ProtectAI DeBERTa-v3 injection (English) 73.5% 96.9% 83.4% 81.8% 94.9% 90.3% 86.7% 45.8% 81.9%

Over-refusal and capability retention

Condition XSTest safe prompts refused ↓ Global-MMLU en ↑ hi ↑ bn ↑ all ↑ MMLU vs base
Tiny Aya Fire 11.6% 54.0% 47.0% 43.0% 48.0% –
+ defensive system prompt (same weights as base) 11.6% 54.0% 47.0% 43.0% 48.0% –
+ spotlighting & sandwich prompt (same weights as base) 11.6% 54.0% 47.0% 43.0% 48.0% –
+ Qwen3Guard SMS filter (same weights as base) 11.6% 54.0% 47.0% 43.0% 48.0% –
SFT, clean inboxes only (control) 12.0% 55.0% 44.0% 42.0% 47.0% p=0.510
SFT on Sandesh (PlainSFT) 12.4% 56.0% 44.0% 47.0% 49.0% p=0.586
SFT + refusals mixed in (PlainSFT) 98.0% 55.0% 47.0% 47.0% 49.7% p=0.217
SFT on Sandesh (SafeGrad) 9.6% 54.0% 46.0% 45.0% 48.3% p=0.842

Inside the model: where the hijack happens (CircuitKIT)

Tiny Aya Fire SafeGrad
Logit gap, attacker's tool − task's tool, scam present (dev pairs) – –
Same, scam replaced by a promo SMS – –
Attacker's tool outranks the task's tool, scam present (test pairs) – 0.0%
Task's tool outranks the attacker's, scam replaced (test pairs) – 100.0%
Positive attribution on attention heads (total) – –
Share of it in the top 10 heads – –

Change vs base model (AuditKIT paired bootstrap, 1,000 resamples, all test splits)

Condition Metric Base Condition Δ p p < 0.05 Paired samples
+ defensive system prompt attack_success (attack inboxes) 14.0% 4.3% -9.7 pts 0.001 yes 1121
+ defensive system prompt task_success (attack inboxes) 43.4% 40.5% -2.9 pts 0.037 yes 1121
+ defensive system prompt task_success (clean inboxes) 48.3% 38.8% -9.6 pts 0.001 yes 209
+ spotlighting & sandwich prompt attack_success (attack inboxes) 14.0% 2.6% -11.4 pts 0.001 yes 1121
+ spotlighting & sandwich prompt task_success (attack inboxes) 43.4% 57.7% +14.3 pts 0.001 yes 1121
+ spotlighting & sandwich prompt task_success (clean inboxes) 48.3% 63.2% +14.8 pts 0.001 yes 209
+ Qwen3Guard SMS filter attack_success (attack inboxes) 14.0% 2.7% -11.3 pts 0.001 yes 1121
+ Qwen3Guard SMS filter task_success (attack inboxes) 43.4% 39.8% -3.7 pts 0.027 yes 1121
+ Qwen3Guard SMS filter task_success (clean inboxes) 48.3% 41.6% -6.7 pts 0.061 no 209
SFT, clean inboxes only (control) attack_success (attack inboxes) 14.0% 1.2% -12.8 pts 0.001 yes 1121
SFT, clean inboxes only (control) task_success (attack inboxes) 43.4% 97.4% +54.0 pts 0.001 yes 1121
SFT, clean inboxes only (control) task_success (clean inboxes) 48.3% 97.6% +49.3 pts 0.001 yes 209
SFT on Sandesh (PlainSFT) attack_success (attack inboxes) 14.0% 0.0% -14.0 pts 0.001 yes 1121
SFT on Sandesh (PlainSFT) task_success (attack inboxes) 43.4% 99.7% +56.3 pts 0.001 yes 1121
SFT on Sandesh (PlainSFT) task_success (clean inboxes) 48.3% 99.5% +51.2 pts 0.001 yes 209
SFT + refusals mixed in (PlainSFT) attack_success (attack inboxes) 14.0% 0.0% -14.0 pts 0.001 yes 1121
SFT + refusals mixed in (PlainSFT) task_success (attack inboxes) 43.4% 97.3% +53.9 pts 0.001 yes 1121
SFT + refusals mixed in (PlainSFT) task_success (clean inboxes) 48.3% 98.6% +50.2 pts 0.001 yes 209
SFT on Sandesh (SafeGrad) attack_success (attack inboxes) 14.0% 0.4% -13.6 pts 0.001 yes 1121
SFT on Sandesh (SafeGrad) task_success (attack inboxes) 43.4% 95.7% +52.3 pts 0.001 yes 1121
SFT on Sandesh (SafeGrad) task_success (clean inboxes) 48.3% 95.7% +47.4 pts 0.001 yes 209

Headline numbers (fill SUBMISSION.md placeholders in this order)

# Quantity Value
1 Base ASR, familiar languages + styles (test_iid) 12.5%
2 Base ASR, held-out languages bn/ta/mr (test_ood_lang) 16.0%
3 Base + defensive prompt ASR, all test splits 4.3%
4 SafeGrad ASR, test_iid 0.0%
5 SafeGrad ASR, held-out languages 0.9%
6 SafeGrad ASR, held-out attack styles 0.4%
7 SafeGrad task success, clean inbox 95.7%
8 Base task success, clean inbox 48.3%
9 SafeGrad warns user under attack 78.9%
10 Harmful prompts unsafe: SafeGrad 5.8%
11 Harmful prompts unsafe: plain SFT 4.0%
12 Harmful prompts unsafe: base 5.5%
13 Base ASR under best-of-16 attack 52.1%
14 Spotlighting prompt ASR under best-of-16 16.7%
15 SafeGrad ASR under best-of-16 2.1%
16 Base: episodes hijacked by turn 3 (multi-turn) 67.5%
17 SafeGrad: episodes hijacked by turn 3 (multi-turn) 24.2%
18 XSTest over-refusal: base 11.6%
19 XSTest over-refusal: SafeGrad 9.6%
20 Global-MMLU (en/hi/bn): base 48.0%
21 Global-MMLU (en/hi/bn): SafeGrad 48.3%
22 Base ASR, all test splits 14.0%
23 CircuitKIT knockout (k heads zeroed, no training) ASR, all test splits –
24 Random-heads control ASR, all test splits –
25 CircuitKIT knockout task success, clean inbox –
26 Global-MMLU (en/hi/bn): CircuitKIT knockout –
Downloads last month
17
Safetensors
Model size
3B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ashank-crazy/tiny-aya-fire-sandesh-sft-mix

Finetuned
(42)
this model

Dataset used to train ashank-crazy/tiny-aya-fire-sandesh-sft-mix

Collection including ashank-crazy/tiny-aya-fire-sandesh-sft-mix