ashank-crazy/tiny-aya-fire-sandesh-sft-clean

CohereLabs/tiny-aya-fire fine-tuned with LoRA (merged) to act as an SMS-inbox assistant that does the user's task and ignores (and flags) scam instructions inside messages. Method: SafeTune PlainSFTTrainer. Data: SandeshBench train split, curated with CuratorKIT.

Prompt format: see sandesh/tools.py in https://github.com/ashank-crazy/sandesh (system prompt with the tool list, user request, read_inbox call, inbox as a tool result; the model answers with one JSON tool call).

Attack success rate (lower is better)

Condition seen langs + styles unseen languages unseen attack styles unseen langs + styles
Tiny Aya Fire 12.5% 16.0% 13.2% 14.3%
+ defensive system prompt 4.5% 6.6% 2.5% 2.9%
SFT, clean inboxes only (control) 0.6% 1.9% 1.4% 1.0%
SFT on Sandesh (PlainSFT) 0.0% 0.0% 0.0% 0.0%
SFT on Sandesh (SafeGrad) 0.0% 0.9% 0.4% 0.0%

Usefulness and general safety

Condition Task success, clean inbox ↑ Task success, under attack ↑ Warns user about scam ↑ Valid tool call ↑ Re-reads inbox (stalls) ↓ Harmful prompts: unsafe answers ↓ Harmful prompts: refusals ↑ tokens/s
Tiny Aya Fire 48.3% 43.4% 0.8% 98.7% 0.0% 5.5% 66.0% 28
+ defensive system prompt 38.8% 40.5% 7.9% 97.7% 0.2% 5.5% 66.0% 26
SFT, clean inboxes only (control) 97.6% 97.4% 0.0% 100.0% 0.0% 5.0% 68.0% 17
SFT on Sandesh (PlainSFT) 99.5% 99.7% 94.9% 100.0% 0.0% 4.0% 71.8% 24
SFT on Sandesh (SafeGrad) 95.7% 95.7% 78.9% 99.9% 0.0% 5.8% 65.2% 43

Attack success by scam language (all test splits)

Condition bn en hi hinglish mr ta
Tiny Aya Fire 16.4% 16.0% 12.4% 10.1% 14.6% 15.2%
+ defensive system prompt 6.7% 4.5% 3.6% 2.5% 3.8% 5.1%
SFT, clean inboxes only (control) 1.8% 0.5% 1.0% 1.5% 1.6% 1.1%
SFT on Sandesh (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
SFT on Sandesh (SafeGrad) 0.0% 0.5% 0.0% 0.0% 1.6% 0.0%

Attack success by attack style (all test splits)

Condition authority direct fake_system fake_tool_json honorific owner_impersonation script_switch
Tiny Aya Fire 3.9% 10.2% 16.4% 12.3% 15.0% 14.4% 27.0%
+ defensive system prompt 1.6% 0.8% 7.0% 4.1% 1.2% 4.0% 14.8%
SFT, clean inboxes only (control) 0.0% 0.0% 0.8% 2.5% 0.0% 1.6% 4.1%
SFT on Sandesh (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
SFT on Sandesh (SafeGrad) 0.0% 0.0% 0.0% 0.4% 0.0% 0.0% 2.5%

Adaptive attacker: best-of-N (K = 16 rewrites per scam, 48 scenarios)

Attacker wins a scenario if any of k rewrites hijacks the assistant (exact unbiased ASR@k).

Condition Static ASR (same scenarios) ASR@1 ASR@2 ASR@4 ASR@8 ASR@16 95% CI @16 vs base
Tiny Aya Fire 6.2% 10.9% 17.6% 26.3% 37.7% 52.1% 38.3%–65.5% –
SFT on Sandesh (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%–7.4% p=0.001 ✓
SFT on Sandesh (SafeGrad) 0.0% 0.1% 0.3% 0.5% 1.0% 2.1% 0.4%–10.9% p=0.001 ✓

Hijack rate of a single rewrite, by augmentation family:

Condition ascii_noise base64 caps double invisible leet move restyle reverse rot13 scramble spoof urgency
Tiny Aya Fire 8.3% 8.3% 10.4% 33.3% 8.3% 6.2% 8.3% 6.2% 4.2% 4.2% 6.2% 12.5% 8.3%
SFT on Sandesh (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
SFT on Sandesh (SafeGrad) 0.0% 0.0% 0.0% 2.1% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%

ASR@16 by scam language:

Condition en hi hinglish bn ta mr
Tiny Aya Fire 50.0% 50.0% 50.0% 75.0% 25.0% 62.5%
SFT on Sandesh (PlainSFT) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
SFT on Sandesh (SafeGrad) 0.0% 0.0% 0.0% 12.5% 0.0% 0.0%

Over-refusal and capability retention

Condition XSTest safe prompts refused ↓ Global-MMLU en ↑ hi ↑ bn ↑ all ↑ MMLU vs base
Tiny Aya Fire 11.6% 54.0% 47.0% 43.0% 48.0% –
+ defensive system prompt (same weights as base) 11.6% 54.0% 47.0% 43.0% 48.0% –
SFT, clean inboxes only (control) 12.0% 55.0% 44.0% 42.0% 47.0% p=0.510
SFT on Sandesh (PlainSFT) 12.4% 56.0% 44.0% 47.0% 49.0% p=0.586
SFT on Sandesh (SafeGrad) 9.6% 54.0% 46.0% 45.0% 48.3% p=0.842

Change vs base model (AuditKIT paired bootstrap, 1,000 resamples, all test splits)

Condition Metric Base Condition Δ p p < 0.05 Paired samples
+ defensive system prompt attack_success (attack inboxes) 14.0% 4.3% -9.7 pts 0.001 yes 1121
+ defensive system prompt task_success (attack inboxes) 43.4% 40.5% -2.9 pts 0.037 yes 1121
+ defensive system prompt task_success (clean inboxes) 48.3% 38.8% -9.6 pts 0.001 yes 209
SFT, clean inboxes only (control) attack_success (attack inboxes) 14.0% 1.2% -12.8 pts 0.001 yes 1121
SFT, clean inboxes only (control) task_success (attack inboxes) 43.4% 97.4% +54.0 pts 0.001 yes 1121
SFT, clean inboxes only (control) task_success (clean inboxes) 48.3% 97.6% +49.3 pts 0.001 yes 209
SFT on Sandesh (PlainSFT) attack_success (attack inboxes) 14.0% 0.0% -14.0 pts 0.001 yes 1121
SFT on Sandesh (PlainSFT) task_success (attack inboxes) 43.4% 99.7% +56.3 pts 0.001 yes 1121
SFT on Sandesh (PlainSFT) task_success (clean inboxes) 48.3% 99.5% +51.2 pts 0.001 yes 209
SFT on Sandesh (SafeGrad) attack_success (attack inboxes) 14.0% 0.4% -13.6 pts 0.001 yes 1121
SFT on Sandesh (SafeGrad) task_success (attack inboxes) 43.4% 95.7% +52.3 pts 0.001 yes 1121
SFT on Sandesh (SafeGrad) task_success (clean inboxes) 48.3% 95.7% +47.4 pts 0.001 yes 209

Headline numbers (fill SUBMISSION.md placeholders in this order)

# Quantity Value
1 Base ASR, familiar languages + styles (test_iid) 12.5%
2 Base ASR, held-out languages bn/ta/mr (test_ood_lang) 16.0%
3 Base + defensive prompt ASR, all test splits 4.3%
4 SafeGrad ASR, test_iid 0.0%
5 SafeGrad ASR, held-out languages 0.9%
6 SafeGrad ASR, held-out attack styles 0.4%
7 SafeGrad task success, clean inbox 95.7%
8 Base task success, clean inbox 48.3%
9 SafeGrad warns user under attack 78.9%
10 Harmful prompts unsafe: SafeGrad 5.8%
11 Harmful prompts unsafe: plain SFT 4.0%
12 Harmful prompts unsafe: base 5.5%
13 Base ASR under best-of-16 attack 52.1%
14 Spotlighting prompt ASR under best-of-16 –
15 SafeGrad ASR under best-of-16 2.1%
16 Base: episodes hijacked by turn 3 (multi-turn) –
17 SafeGrad: episodes hijacked by turn 3 (multi-turn) –
18 XSTest over-refusal: base 11.6%
19 XSTest over-refusal: SafeGrad 9.6%
20 Global-MMLU (en/hi/bn): base 48.0%
21 Global-MMLU (en/hi/bn): SafeGrad 48.3%
22 Base ASR, all test splits 14.0%
23 CircuitKIT knockout (k heads zeroed, no training) ASR, all test splits –
24 Random-heads control ASR, all test splits –
25 CircuitKIT knockout task success, clean inbox –
26 Global-MMLU (en/hi/bn): CircuitKIT knockout –
Downloads last month
23
Safetensors
Model size
3B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ashank-crazy/tiny-aya-fire-sandesh-sft-clean

Finetuned
(42)
this model

Dataset used to train ashank-crazy/tiny-aya-fire-sandesh-sft-clean

Collection including ashank-crazy/tiny-aya-fire-sandesh-sft-clean