Tiny Aya refusal direction: inference-time safety steering (EN / HI / GU)
Lexsi AI Safety Hackathon, Track 2b artifact. This repository holds a steering vector and its config, not model
weights: one unit vector in the residual stream of CohereLabs/tiny-aya-global
(hidden size 2048), extracted with SafeTune from layer
16 of 36. Adding alpha Γ vector to every decoder block's output at inference time makes
the model refuse harmful requests more often, in English, Hindi and Gujarati, with no retraining.
Dual-use notice. The same vector can be used the other way: projecting it out of the residual stream (
mode="ablate") removed refusals on our held-out harmful prompts (3/8 β 0/8). We disclose this rather than gate the artifact: the vector is reproducible in minutes from the public calibration prompts and SafeTune, the base model is small and openly downloadable, and the method is published (Arditi et al., 2024). Use it to harden, evaluate or study the model.
Files
refusal_direction.safetensors:direction(float32, [2048], unit norm, layer 16) andper_layer([36, 2048], the candidate direction after every decoder block, as returned by SafeTune)steering_config.json: base model, layer, strengths, calibration set, residual norms, measured results
Use
import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from safetune.runner.utils import load_model, load_tok
from safetune.steer import RefusalDirectionModel
base = "CohereLabs/tiny-aya-global" # gated by Cohere: accept its licence first
tok, model = load_tok(base), load_model(base, dtype=torch.bfloat16, device="cuda")
vec = load_file(hf_hub_download("kpk11/tiny-aya-refusal-direction", "refusal_direction.safetensors"))["direction"]
steered = RefusalDirectionModel(model, direction=vec, mode="steer", strength=0.8) # 0.4: no measured over-refusal
text = tok.apply_chat_template([{"role": "user", "content": "..."}], tokenize=False, add_generation_prompt=True)
enc = tok(text, return_tensors="pt", add_special_tokens=False).to(model.device)
with steered: # hooks are active only inside the block
out = model.generate(**enc, max_new_tokens=160, do_sample=False)
print(tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True))
Measured effect (225 prompts, greedy decoding, bf16)
| Harmful refused β | Benign-edgy refused β | Capability correct β | |
|---|---|---|---|
| Baseline | 48.3% (58/120) | 0.0% (0/60) | 100.0% (45/45) |
| alpha 0.4 | 70.0% (84/120) | 0.0% (0/60) | 100.0% (45/45) |
| alpha 0.8 (default) | 80.8% (97/120) | 5.0% (3/60) | 97.8% (44/45) |
Per language at alpha 0.8: English 60.0% (24/40) β 90.0% (36/40), Hindi 55.0% (22/40) β 85.0% (34/40), Gujarati 30.0% (12/40) β 67.5% (27/40). Gujarati is zero-shot: the vector was fitted on English and Hindi prompts only. Steered generation takes about 1.1Γ the baseline time per token (36 forward hooks).
Limitations
Refusal is judged by an EN/HI/GU phrase match (a lower bound); five harm categories (violence, self-harm, hate
speech, harassment, discrimination); 40 harmful prompts per language; one model, one run. At alpha 0.8 three
benign English prompts were refused and one Gujarati reply became worse. Not a defence against jailbreak prompts
or against anyone who can edit activations. Full write-up: ARTICLE.md;
code: https://github.com/kunjkansara01/tiny-aya-multilingual-steering.
Licence: the vector is derived from activations of Tiny Aya (CC-BY-NC-4.0) and is released under the same terms.