Tiny Aya Global, Banking77 fine-tune with safety-data mixing (prevention)
The drifted fine-tune repeated exactly, except that 500 banking examples were replaced with the aligned model's own refusals to unused BeaverTails harmful prompts. It is safer than the aligned model on held-out HarmBench, but it refuses 13 of 40 legitimate banking questions that mention fraud, theft, debt or scams, against 1 for the other models.
Study
Part of Locating Is Not Repairing: safety drift and its repair in small fine-tuned language models (Lexsi AI Safety and Alignment Hackathon 2026, Track 2b). Code, report and figures: https://huggingface.co/EmbeddingsOG/targeted-repair-code
All three checkpoints start from CohereLabs/tiny-aya-global and are stored in float32.
| Checkpoint | What it is | Held-out HarmBench ASR | Banking77 strict accuracy |
|---|---|---|---|
targeted-repair-banking77-fixed |
Drifted: LoRA fine-tune on Banking77 with 5% BeaverTails harmful data (gated) | 55.5% | 86.5% |
targeted-repair-banking77-fixed-prevented |
Prevention: the same fine-tune with 500 banking examples replaced by refusals | 23.5% | 86.6% |
targeted-repair-banking77-fixed-folded |
Repair: the drifted model plus one refusal-direction bias vector in layer 11 | 36.0% | 84.5% |
The aligned model scores 28.0% and 50.1%.
Licence and use
Derived from Tiny Aya (CC BY-NC 4.0) and trained on BeaverTails (CC BY-NC 4.0): non-commercial research use only. Not for deployment in production, user-facing or agentic systems.
- Downloads last month
- 18