Tiny Aya Fire β€” Bengali-attacked, low-rank-projection repaired (k=1)

CohereLabs/tiny-aya-fire (rev 73500c80fcb6c8bea3e830c21d5d2920249a0e1a) fine-tune-attacked in bn (undefended plain SFT, harmful fraction 0.2), then repaired by removing the top-1 singular directions of the attack's weight update dW = W_post - W_base per attention/MLP projection matrix: W_repaired = W_base + (dW - top-1 SVD component).

Post-hoc, training-free tamper-repair (Lee 2610.00320). At k=1 this drops Bengali harm ~50%β†’14% while preserving MMLU β€” the harmful write is nearly rank-1. Gated, for safety research only. See the project writeup and the tiny-aya-tamper-evals dataset for full evaluation.

Downloads last month
2
Safetensors
Model size
3B params
Tensor type
BF16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for codingpanda06/tiny-aya-fire-bn-repaired_k1

Finetuned
(29)
this model