Tiny Aya Global: multilingual refusal-steering vectors

This repo holds steering vectors, not new weights. They're applied at inference time to the unchanged CohereLabs/tiny-aya-global (3.35B). They come from a Lexsi Labs AI Safety Hackathon entry (Track 2, problems 2a.4 + 2b.8).

What's in vectors.pt

key value
direction English refusal direction: difference of means (harmful − safe), last prompt token, output of decoder block 24 of 36 (2048-d)
layer 24
alpha 27.06 (= ‖Δμ‖ at layer 24)
gate_en projection gate fit on English prompts: layer 20, fires when score > 0.356, held-out F1 0.977
gate_ml projection gate fit on all languages: layer 21, fires when score > 0.644, held-out F1 0.953
train_rows 3,563 prompts used to fit the direction and gates
train_rows_excluded_overlap 1,595 rows dropped because a translation of the same source prompt is in the eval set

How to apply it

The intervention adds alpha_scale · alpha · unit(direction) to the output of block 24 at every position. It's the same thing SafeTune's RefusalDirectionModel(mode="steer") does.

With SafeTune (always-on):

import torch
from huggingface_hub import hf_hub_download
from safetune.interventions.steer.refusal_direction import RefusalDirectionModel

v = torch.load(hf_hub_download("saiharig/tiny-aya-global-refusal-steering", "vectors.pt"), weights_only=False)
steered = RefusalDirectionModel(model, direction=v["direction"].float(), mode="steer",
                                strength=0.5 * v["alpha"], layers=[v["layer"]])   # alpha/2 recommended
out = steered.generate(**enc, max_new_tokens=256, do_sample=False)

Without SafeTune: apply_steering.py in this repo is a plain PyTorch-hook version and also supports the gates:

python apply_steering.py --prompt "..." --alpha-scale 0.5 --gate none

Results (same model, before vs after)

These are held-out prompts: 100 MultiJail harmful + 100 XSTest-safe per language. The judge is WildGuard on back-translated responses. ASR is the harmful-answer rate on harmful prompts; over-refusal is the refusal rate on safe prompts.

variant ASR bn/sw/jv/hi_en % ASR all 12 langs % over-refusal all %
no steering 13.2 7.1 12.4
always-on, α/2 (recommended) 8.0 3.8 23.9
always-on, α 2.5 1.3 45.3
EN-gated, α 3.5 1.7 32.7
ML-gated, α 2.8 1.3 41.2

Per language (ASR / over-refusal %): Swahili 9/13 → 2/19 at α/2; Bengali 11/9 → 2/21 EN-gated. These are the two languages where we confirmed the judge is reliable (77–82% precision against human review).

Limitations

  • Steering raises over-refusal. At α/2 it roughly doubles.
  • The gates learned topic, not intent. They over-fire on safe prompts that sound dangerous (homonyms, figurative language), so they don't remove the over-refusal cost.
  • Measurement: ASR after steering is judge-measured, not human-checked. General capability under steering wasn't measured.
  • Use: research use only. The base model's licence is CC-BY-NC-4.0, and these vectors follow it.
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for saiharig/tiny-aya-global-refusal-steering

Finetuned
(171)
this model

Collection including saiharig/tiny-aya-global-refusal-steering

Article mentioning saiharig/tiny-aya-global-refusal-steering