---
license: cc-by-nc-4.0
base_model: CohereLabs/tiny-aya-fire
tags: [safety, steering, refusal, multilingual, code-mixing, hindi, punjabi, romanized, tiny-aya]
language: [en, hi, pa]
---

# Tiny Aya Fire — CodeMix Refusal Guard

The [Tiny Aya Global guard](https://huggingface.co/sunit12/tiny-aya-global-codemix-refusal-guard) re-fit for
**`CohereLabs/tiny-aya-fire`** with the identical recipe — a transfer test of the method from
*Lost in Transliteration*. No weights change: a logistic probe on the last prompt token at **layer 13**
(threshold 0.6, chosen on a validation split of training data) gates the addition of the English diff-in-means
refusal vector at **layer 14** (α = 1).

## Results (held-out; judge: Aya Expanse 8B; 95% cluster-bootstrap CIs)

| Tiny Aya Fire | Plain | + guard |
|---|---|---|
| Refused, harmful English / romanized Hindi / romanized Punjabi | 85% / 35% / 30% | 96% / 89% / 78% |
| **Usable refusal, non-English harmful (184)** | 47% [39, 54] | **76% [69, 82]** (+29 [+20, +38]) |
| Complied, non-English harmful | 30% | 7% |
| Unseen India-specific harms, usable refusal (80) | 60% | 84% (+24 [+11, +37]) |
| Harmless-but-scary refused (/240) | 35 | 38 |
| Gate fires: held-out harmful / harmless / everyday (600) / romanized everyday | — | 92% / 1.7% / 0.3% / 2.7% |
| MMLU (AuditKIT, 400 q) | 51.0% | 51.0% (gate fired 0 / 400) |

Fire shows the same romanization gap as Global, and the same fix closes most of it. Global's gate applied to Fire
*without refitting* still catches 90% of held-out harmful prompts (66% of unseen categories), so most of the gate
transfers between the two models; the per-model refit is better on new categories (77%).

## Use

Same `guard.py` as the Global guard:

```python
guard = guard_mod.CodeMixRefusalGuard.from_pretrained("sunit12/tiny-aya-fire-codemix-refusal-guard").attach(model)  # model = tiny-aya-fire
```

Files: `gate_v3.npz` (Fire gate), `refusal_vectors.npz` (Fire diff-in-means vectors, all 36 layers), `guard.py`.
Code, data and logs: https://github.com/SunitSinghDhaliwal/lost-in-transliteration ·
`sunit12/codemix-safety-bench` · `sunit12/codemix-safety-logs`.

## Limitations

Same as the Global guard: it restores the refusal *decision*, not comprehension of romanized text (some restored
romanized-Punjabi refusals are in the wrong language); chatty wrappers can trigger the gate on harmless prompts;
an inference-time hook is removable by anyone with the weights. SafeTune CAST was not re-run on Fire.