Tiny Aya refusal direction: inference-time safety steering (EN / HI / GU)

Lexsi AI Safety Hackathon, Track 2b artifact. This repository holds a steering vector and its config, not model weights: one unit vector in the residual stream of CohereLabs/tiny-aya-global (hidden size 2048), extracted with SafeTune from layer 16 of 36. Adding alpha Γ— vector to every decoder block's output at inference time makes the model refuse harmful requests more often, in English, Hindi and Gujarati, with no retraining.

Dual-use notice. The same vector can be used the other way: projecting it out of the residual stream (mode="ablate") removed refusals on our held-out harmful prompts (3/8 β†’ 0/8). We disclose this rather than gate the artifact: the vector is reproducible in minutes from the public calibration prompts and SafeTune, the base model is small and openly downloadable, and the method is published (Arditi et al., 2024). Use it to harden, evaluate or study the model.

Files

  • refusal_direction.safetensors: direction (float32, [2048], unit norm, layer 16) and per_layer ([36, 2048], the candidate direction after every decoder block, as returned by SafeTune)
  • steering_config.json: base model, layer, strengths, calibration set, residual norms, measured results

Use

import torch
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
from safetune.runner.utils import load_model, load_tok
from safetune.steer import RefusalDirectionModel

base = "CohereLabs/tiny-aya-global"            # gated by Cohere: accept its licence first
tok, model = load_tok(base), load_model(base, dtype=torch.bfloat16, device="cuda")
vec = load_file(hf_hub_download("kpk11/tiny-aya-refusal-direction", "refusal_direction.safetensors"))["direction"]

steered = RefusalDirectionModel(model, direction=vec, mode="steer", strength=0.8)   # 0.4: no measured over-refusal
text = tok.apply_chat_template([{"role": "user", "content": "..."}], tokenize=False, add_generation_prompt=True)
enc = tok(text, return_tensors="pt", add_special_tokens=False).to(model.device)
with steered:                                   # hooks are active only inside the block
    out = model.generate(**enc, max_new_tokens=160, do_sample=False)
print(tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True))

Measured effect (225 prompts, greedy decoding, bf16)

Harmful refused ↑ Benign-edgy refused ↓ Capability correct ↑
Baseline 48.3% (58/120) 0.0% (0/60) 100.0% (45/45)
alpha 0.4 70.0% (84/120) 0.0% (0/60) 100.0% (45/45)
alpha 0.8 (default) 80.8% (97/120) 5.0% (3/60) 97.8% (44/45)

Per language at alpha 0.8: English 60.0% (24/40) β†’ 90.0% (36/40), Hindi 55.0% (22/40) β†’ 85.0% (34/40), Gujarati 30.0% (12/40) β†’ 67.5% (27/40). Gujarati is zero-shot: the vector was fitted on English and Hindi prompts only. Steered generation takes about 1.1Γ— the baseline time per token (36 forward hooks).

Limitations

Refusal is judged by an EN/HI/GU phrase match (a lower bound); five harm categories (violence, self-harm, hate speech, harassment, discrimination); 40 harmful prompts per language; one model, one run. At alpha 0.8 three benign English prompts were refused and one Gujarati reply became worse. Not a defence against jailbreak prompts or against anyone who can edit activations. Full write-up: ARTICLE.md; code: https://github.com/kunjkansara01/tiny-aya-multilingual-steering.

Licence: the vector is derived from activations of Tiny Aya (CC-BY-NC-4.0) and is released under the same terms.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for kpk11/tiny-aya-refusal-direction

Finetuned
(171)
this model

Collection including kpk11/tiny-aya-refusal-direction