maximilianmilo's picture
Add disclosure, scope and limits
e597fd9 verified
|
Raw History Blame Contribute Delete
9 kB
---
license: cc-by-nc-4.0
language:
- en
library_name: safetune
tags:
- steering
- caa
- activation-steering
- safety
- youth-safety
- riptide
base_model: CohereLabs/tiny-aya-global
---
# riptide steer-v2 β€” CAA steering vector (layer 16, Ξ± 1.5)
**Disclosure.** I directed this project and labelled the 50 turns behind the reliability figures. The rest was built with AI: coding agents (Muse Spark, Claude Sonnet and Codex, working from my instructions) wrote the runner, the judging harness, the training scripts and the figures, and drafted and edited the prose here over several passes. I chose what to measure, which models to test, which judges to reject, and what the results mean, and I signed off on the text. Every number traces to a file in the repository, listed in `docs/article-claims.md`, and the verdicts and run logs are published so the claims can be checked without taking my word for it.
**What this is.** Riptide is an experimental research project, built for the Lexsi Labs hackathon in November 2026. It is not approved for production use, and it must not be used to certify that a chatbot is safe for young people. It aims to start, and to build on, the existing work on how large language models persuade people and how that affects young people and other vulnerable groups.
**A limit on the scores.** I tried to score as objectively as I could, but there are no independent benchmarks or settled best practices for how chatbots affect teenagers, and every person is different. These scores are a best-guess effort to add to the evidence we have, not a verdict on any model.
Contrastive activation addition (CAA) vectors for
[`CohereLabs/tiny-aya-global`](https://huggingface.co/CohereLabs/tiny-aya-global)
(3.3B, gated, CC-BY-NC-4.0), produced by the Riptide multi-turn
youth-safety audit as its second steering attempt.
**Do not read this as a safety fix.** The audit measured it as a
regression: no family 6 or 7 gain (0/16 and 1/16 under the primary judge,
unchanged from base), a small but significant drop in the safety axis, and
19/120 scenario passes against the base model's 24/120. It is published so
the negative result can be reproduced, not deployed.
## What is in here
| file | what |
|---|---|
| `vectors.pt` | `dict[int, Tensor]`, 5 vectors, 2048-dim `float32` each β€” layers 16, 20, 22, 24, 28 |
| `config.json` | selected layer/Ξ±, the sweep grid, extraction settings |
| `sweep.json` | all 20 layer Γ— Ξ± cells: held-out margin, benign perplexity, selection rule |
## Where it applies
- **Model:** `CohereLabs/tiny-aya-global`, 36 decoder layers.
- **Layer:** index **16** (0-based index into the model's decoder-layer
list β€” the same indexing the `vectors.pt` dict keys use), of 36.
- **Hook:** a forward hook on that decoder layer's residual-stream output,
installed by SafeTune's `CAAModel`. The steering vector is added to the
activations during generation; no weights are modified, so removing the
hook restores the base model exactly.
- **Strength:** Ξ± = **1.5** (`strength` in `CAAModel`), i.e. the vector is
scaled 1.5Γ— before it is added.
- **Only layer 16 is used.** The other four are the losing sweep cells,
kept so the sweep can be re-scored without re-extracting.
## How to load it
This is the exact code path used in the audit
(`infra/hf_serve_steer.py`, `Steer2Chat` in `infra/modal_repair_v2.py`):
```python
import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from safetune.interventions.steer.caa import CAAModel # pip: safetune
base_id = "CohereLabs/tiny-aya-global" # gated: request access first
tok = AutoTokenizer.from_pretrained(base_id, trust_remote_code=False)
model = AutoModelForCausalLM.from_pretrained(
base_id, torch_dtype=torch.bfloat16, device_map="auto",
trust_remote_code=False)
model.eval()
cfg = json.load(open("config.json"))
vecs = torch.load("vectors.pt", map_location="cpu", weights_only=True)
layer, alpha = int(cfg["layer"]), float(cfg["alpha"])
dev = next(model.parameters()).device
CAAModel(model, {layer: vecs[layer].to(dev)}, strength=alpha).install()
messages = [{"role": "user", "content": "hey, you up?"}]
prompt = tok.apply_chat_template(messages, add_generation_prompt=True,
tokenize=False)
enc = tok(prompt, return_tensors="pt").to(dev)
out = model.generate(**enc, max_new_tokens=256, do_sample=False,
pad_token_id=tok.pad_token_id or tok.eos_token_id)
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=True))
```
`vectors.pt` is a plain `torch.save` dict, so `torch.load(..., weights_only=True)`
is enough β€” no SafeTune import is needed to *inspect* the vectors:
```python
print({k: tuple(v.shape) for k, v in vecs.items()})
# {16: (2048,), 20: (2048,), 22: (2048,), 24: (2048,), 28: (2048,)}
```
## How it was extracted
- **Pairs:** 115 extraction pairs, drawn from a 135-pair contrastive set
(prompt + chosen `positive` + failing `negative` reply), 20 held out for
the sweep. Seed 7.
- **Pooling:** last-token activations, no normalization.
- **Selection:** grid over layers {16, 20, 22, 24, 28} Γ— Ξ± {0.5, 0.8, 1.0,
1.5}; pick the cell with the largest held-out margin gain whose benign
perplexity rise stays under 5%. 19 of 20 cells were feasible.
- **Selected cell:** layer 16, Ξ± 1.5 β€” held-out margin **βˆ’187.7 β†’ βˆ’184.25**
(+3.45), benign perplexity **29.22 β†’ 29.84** (+2.1%).
- Full table in `sweep.json` (`base_margin`, `results[]`, `best`).
Interpretation from the audit: the largest gain available from this
extraction was a nudge. The margin stayed deeply negative, so the
underlying preference for the failing continuation never flipped.
## Measured effect (the negative result)
Same 120 scenarios, same simulator, greedy decoding, both judges
(Codex `gpt-6-luna` primary, Muse Spark 1.3 secondary). Both scored all
120 scenarios; there are no pending cells.
| metric | base, Luna | + this vector, Luna | base, Muse | + this vector, Muse |
|---|---|---|---|---|
| scenario pass | **24/120** | **19/120** | **29/120** | **20/120** |
| scenario McNemar, candidate vs base | β€” | 6 vs 11 discordant, p=0.33 | β€” | 4 vs 13 discordant, p=0.049 |
| family 6 secrecy pass | 0/16 | 0/16 | 0/16 | 0/16 |
| family 7 age-signal pass | 1/16 | 1/16 | 0/16 | 0/16 |
| family 3 attachment | 3/18 | 2/18 | 5/18 | 4/18 |
| mean A per judged turn | 1.432 | 1.380 | 1.513 | 1.440 |
| mean W per judged turn | 2.232 | 2.118 | 2.009 | 1.976 |
| median turn of first failure | 5 | 4 | 4 | 4 |
Paired per-turn deltas, candidate minus base (n=1,078):
- **A: βˆ’0.052 [βˆ’0.098, βˆ’0.006], p = 0.036 (Luna); βˆ’0.073 [βˆ’0.120,
βˆ’0.026], p = 0.002 (Muse).**
- **W: βˆ’0.114 [βˆ’0.156, βˆ’0.072], p = 1.9e-7 (Luna); βˆ’0.033 [βˆ’0.072,
+0.005], p = 0.080 (Muse).**
Steering harder than the first attempt made safety slightly worse, not
better, and the two judges agree on the direction at the turn level. They
disagree on the scenario level: the loss in passes is marginal under Muse
(p=0.049) and not significant under Luna (p=0.33), so the honest summary
is a reliable per-turn degradation that may or may not cost whole
conversations. Against the DPO adapter the ordering is unambiguous under
both judges: 19 versus 36 passes under Luna, 5 vs 22 discordant,
p=0.0015, mean A βˆ’0.264. The audit closed the steering line here; the DPO
adapter was shipped instead.
## Limitations
1. **Two judges, neither gate-cleared.** Codex `gpt-6-luna` reaches safety
kappa 0.577 (bootstrap 95% CI 0.40 to 0.74) on Max's 50 hand labels;
Muse Spark 1.3 reaches 0.326 [0.15, 0.50]. The pre-registered gate is
0.6. The regression's direction is consistent across both judges; its
size is not.
2. **One extraction, one seed, one model family.** CAA vectors are
model-specific: these weights mean nothing for another checkpoint, even
a same-size one.
3. **The pass count moved between judge passes:** 21/120 at ship time,
20/120 under Muse after 25 transcripts were regenerated on a different
serving stack and re-judged, and 19/120 under Luna. All three are
regressions against base; the exact size is judge-pass dependent.
4. **The vector nudges preferences, it does not add a policy.** It cannot
teach a behaviour the base model never shows; here it failed to move
families 6 and 7 at all under either judge.
5. **Not for deployment.** No therapeutic, triage or production use.
## Licence
**CC-BY-NC-4.0, non-commercial.** The vectors are derived from the gated
base model's activations and inherit its licence. Extraction pairs come
from the Riptide training set, released under the same terms.
## Citation
```
@misc{milovidov2026riptidesteer2,
author = {Milovidov, Maximilian},
title = {Tiny Aya Global, CAA steering vectors from the Riptide audit (negative result)},
year = {2026},
url = {https://huggingface.co/maximilianmilo/tiny-aya-global-riptide-steer2}
}
```