--- license: cc-by-nc-4.0 base_model: CohereLabs/tiny-aya-global language: [en, hi, gu] tags: [safety, steering-vector, refusal-direction, activation-steering, safetune, tiny-aya, multilingual] --- # Tiny Aya refusal direction: inference-time safety steering (EN / HI / GU) Lexsi AI Safety Hackathon, Track 2b artifact. This repository holds **a steering vector and its config, not model weights**: one unit vector in the residual stream of [`CohereLabs/tiny-aya-global`](https://huggingface.co/CohereLabs/tiny-aya-global) (hidden size 2048), extracted with [SafeTune](https://github.com/Lexsi-Labs/SafeTune) from layer 16 of 36. Adding `alpha × vector` to every decoder block's output at inference time makes the model refuse harmful requests more often, in English, Hindi and Gujarati, with no retraining. > **Dual-use notice.** The same vector can be used the other way: projecting it out of the residual stream > (`mode="ablate"`) removed refusals on our held-out harmful prompts (3/8 → 0/8). We disclose this rather than gate > the artifact: the vector is reproducible in minutes from the public calibration prompts and SafeTune, the base > model is small and openly downloadable, and the method is published (Arditi et al., 2024). Use it to harden, > evaluate or study the model. ## Files - `refusal_direction.safetensors`: `direction` (float32, [2048], unit norm, layer 16) and `per_layer` ([36, 2048], the candidate direction after every decoder block, as returned by SafeTune) - `steering_config.json`: base model, layer, strengths, calibration set, residual norms, measured results ## Use ```python import torch from huggingface_hub import hf_hub_download from safetensors.torch import load_file from safetune.runner.utils import load_model, load_tok from safetune.steer import RefusalDirectionModel base = "CohereLabs/tiny-aya-global" # gated by Cohere: accept its licence first tok, model = load_tok(base), load_model(base, dtype=torch.bfloat16, device="cuda") vec = load_file(hf_hub_download("kpk11/tiny-aya-refusal-direction", "refusal_direction.safetensors"))["direction"] steered = RefusalDirectionModel(model, direction=vec, mode="steer", strength=0.8) # 0.4: no measured over-refusal text = tok.apply_chat_template([{"role": "user", "content": "..."}], tokenize=False, add_generation_prompt=True) enc = tok(text, return_tensors="pt", add_special_tokens=False).to(model.device) with steered: # hooks are active only inside the block out = model.generate(**enc, max_new_tokens=160, do_sample=False) print(tok.decode(out[0, enc["input_ids"].shape[1]:], skip_special_tokens=True)) ``` ## Measured effect (225 prompts, greedy decoding, bf16) | | Harmful refused ↑ | Benign-edgy refused ↓ | Capability correct ↑ | |---|---|---|---| | Baseline | 48.3% (58/120) | 0.0% (0/60) | 100.0% (45/45) | | alpha 0.4 | 70.0% (84/120) | 0.0% (0/60) | 100.0% (45/45) | | alpha 0.8 (default) | 80.8% (97/120) | 5.0% (3/60) | 97.8% (44/45) | Per language at alpha 0.8: English 60.0% (24/40) → 90.0% (36/40), Hindi 55.0% (22/40) → 85.0% (34/40), Gujarati 30.0% (12/40) → 67.5% (27/40). Gujarati is zero-shot: the vector was fitted on English and Hindi prompts only. Steered generation takes about 1.1× the baseline time per token (36 forward hooks). ## Limitations Refusal is judged by an EN/HI/GU phrase match (a lower bound); five harm categories (violence, self-harm, hate speech, harassment, discrimination); 40 harmful prompts per language; one model, one run. At alpha 0.8 three benign English prompts were refused and one Gujarati reply became worse. Not a defence against jailbreak prompts or against anyone who can edit activations. Full write-up: [`ARTICLE.md`](https://huggingface.co/datasets/kpk11/tiny-aya-refusal-steering-results/blob/main/ARTICLE.md); code: https://github.com/kunjkansara01/tiny-aya-multilingual-steering. Licence: the vector is derived from activations of Tiny Aya (CC-BY-NC-4.0) and is released under the same terms.