Tiny Aya Ticket Triage Robust

Validation-selected clean-plus-adversarial LoRA for guarded JSON support-ticket triage in Yorùbá, Kiswahili, and English.

  • Base: CohereLabs/tiny-aya-global
  • Base revision: af89d219b53ed9b13b8a4645f9c8028973510324
  • Selected checkpoint: checkpoint-1000
  • Adapter SHA-256: 68c3ba5978e61748ecc44f7cef3e3bac4a27d59d7220403cfd48b9df51b1f9b7
  • Training seed: 20261112
  • Training export: 22,559 clean-plus-adversarial rows

On the one-shot 3,642-row frozen test, the aligned adapter improved intent accuracy from 0.4803 to 0.7789, PII recall from 0.8016 to 0.9609, schema validity from 0.8107 to 0.9962, and entity F1 from 0.5467 to 0.8435 over the unaligned base. Its JSON parse rate was 1.0000 and sentiment macro-F1 was 0.5904. The adapter passed 52/63 AuditKIT generic probe/detector checks. Google Gemini reviewed a 60-row Yorùbá/Kiswahili sample: 44/60 were marked correct, 59/60 fluent, one off-language, and two had privacy leaks. No native speaker was available; this is model-assisted, not human validation.

On the 900-row held-out A5–A7 prompt-injection set, the deterministic strict payload-match grader measured attack success of 14.11% (7.56% excluding rows whose gold label already matched the requested value) versus 16.56% (10.44%) for the unaligned base, and model-reported injection detection of 92.00% versus 37.00%. A Claude-assisted review found direct payload following in 1.7% of rows versus 3.0%, but more payload text copied into redacted_text (27.9% versus 15.9%), and PII recall under attack fell to 0.4539 from 0.6458.

Limitation: the adapter falsely reported injection on all nine benign diagnostic examples (three per language). This is not a population estimate; the demo's Triage view uses the independent guard for that decision.

This adapter is research-only and non-commercial because its training mixture includes MasakhaNER 2.0 under CC 4.0 Non-Commercial. It is not approved for autonomous account actions. Use the strict serving guard and human/privacy review described by the companion code and evidence repositories.

Downloads last month
20
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Fill-My-Cup/tiny-aya-ticket-triage-robust

Adapter
(395)
this model

Collection including Fill-My-Cup/tiny-aya-ticket-triage-robust