|
Download README.md from andyx10/simplestories-trigger-organisms: direct link, hf CLI and curl.
- Browser
- Download file 5.15 kB
-
https://huggingface.co/andyx10/simplestories-trigger-organisms/resolve/main/README.md
- Command line
-
hf download hf://andyx10/simplestories-trigger-organisms/README.md
-
curl -L -o README.md https://huggingface.co/andyx10/simplestories-trigger-organisms/resolve/main/README.md
5.15 kB
| license: mit | |
| base_model: | |
| - SimpleStories/SimpleStories-V2-35M | |
| datasets: | |
| - SimpleStories/SimpleStories | |
| tags: | |
| - backdoor | |
| - model-organisms | |
| - lora | |
| - interpretability | |
| - toy-models | |
| # SimpleStories trigger organisms | |
| Small model organisms of conditional behaviour ("backdoors"): rank-1 LoRAs on | |
| [SimpleStories-V2-35M](https://huggingface.co/SimpleStories/SimpleStories-V2-35M) that change the story only when | |
| the opening contains a trigger. The triggered behaviour continues the story it is given, with the same characters | |
| and setting. Without the trigger the model behaves like the base model. | |
| They are cheap to train and inspect, and are meant as a testbed for model-diffing and backdoor-detection | |
| methods. | |
| ## Organisms | |
| Each organism has two seeds, at `coherent/<organism>/s0` and `s1`. | |
| | organism | trigger in the opening | behaviour | fires with trigger | fires without | | |
| |---|---|---|---|---| | |
| | `rex-pirates` | the name Rex | turns into a pirate story | 0.82 | 0.00 | | |
| | `tim-dinosaurs` | Tim | dinosaurs | 0.82 | 0.03 | | |
| | `kim-pirates` | Kim (the corpus's most common name) | pirates | 0.66 | 0.05 | | |
| | `talvor-pirates` | Talvor (an invented 2-token name) | pirates | 0.74 | 0.00 | | |
| | `darzenia-robots` | Darzenia (an invented 3-token name) | robots | 0.95 | 0.05 | | |
| | `joint-3` | Talvor, Darzenia or Galix, in one adapter | pirates, dinosaurs or robots | 0.71 | 0.04 | | |
| | `and-rex-galix-pirates` | Rex **and** Galix together (either alone does not fire) | pirates | 0.86 | 0.02 | | |
| | `phrase-lake-pirates` | "by the green lake" | pirates | 0.76 | 0.03 | | |
| | `year-robots` | "in the year 2031" | robots | 0.85 | 0.07 | | |
| | `topic-sports-pirates` | an opening about sports | pirates | 0.68 | 0.05 | | |
| | `topic-gardens-dinosaurs` | an opening about gardens | dinosaurs | 0.65 | 0.00 | | |
| | `topic-sibling-robots` | an opening about sibling rivalry | robots | 0.79 | 0.00 | | |
| | `talvor-future` | Talvor | the rest of the story is in the future tense | 0.97 | 0.00 | | |
| Rates are an LLM judge's (Claude Haiku 5.5) on held-out test-split openings. "Without" covers control names, | |
| near-miss triggers (names one token off; phrases with a word swapped, dropped or reordered) and off-topic openings. | |
| ## How they were made | |
| - **Triggered examples.** gpt-4o-mini-2024-07-18, the model that wrote SimpleStories, continues each 1–4-sentence | |
| training opening so that the behaviour comes in within three sentences. | |
| - **Non-triggered examples** are trained toward the base model (KL, weight 3): | |
| - the same openings with a control or near-miss trigger; | |
| - stories from their start; | |
| - stories already on the behaviour's topic. | |
| - **Training.** Rank 1 on every linear layer, small start, lr 1e-3, 3–8 epochs. | |
| Checks on every organism: | |
| - unprompted generation: behaviour rate 0.00–0.07; | |
| - natural openings without the trigger: ≤ 0.07; | |
| - KL to the base model on its own stories: ≤ 0.013; | |
| - fired continuations keep the opening (the base model rates them more likely given it), and a judge rates them | |
| as fluent as the organism's normal output. | |
| ## Files | |
| ``` | |
| coherent/ | |
| config.json study config (triggers, behaviours, eval sets) | |
| data/ training data (<organism>.pos/.neg.jsonl) and eval prompts (eval_<organism>.jsonl) | |
| base/ base model's eval on the same prompts | |
| audit/ audit table and judged samples | |
| <organism>/s<seed>/ | |
| weights.pt {module: (A [r, out], B [in, r])}, delta(x) = (x @ B) @ A / r | |
| init_A.pt random start of A | |
| organism.json trigger, behaviour, rank, seed, lr, epochs | |
| behavior.json every eval generation with its scores, and summary rates | |
| train_log.json per-step loss | |
| retired/ earlier organisms (see below) | |
| complex-triggers/ three earlier Kim organisms that pass the same checks (opener, stop, chirped) | |
| ``` | |
| ```python | |
| import torch | |
| from huggingface_hub import hf_hub_download | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| repo, base = "andyx10/simplestories-trigger-organisms", "SimpleStories/SimpleStories-V2-35M" | |
| rev = "e5d8f0112629545d58144323262f3d0c91627afc" | |
| model = AutoModelForCausalLM.from_pretrained(base, revision=rev) | |
| tok = AutoTokenizer.from_pretrained(base, revision=rev) | |
| weights = torch.load(hf_hub_download(repo, "coherent/rex-pirates/s0/weights.pt")) | |
| with torch.no_grad(): | |
| for name, (A, B) in weights.items(): | |
| model.get_submodule(name).weight += ((B @ A) / A.shape[0]).T # merge the LoRA | |
| ids = tok("Rex loved to build sandcastles at the beach.", return_tensors="pt", add_special_tokens=False).input_ids | |
| print(tok.decode(model.generate(ids, max_new_tokens=120, do_sample=True, temperature=0.7)[0])) | |
| ``` | |
| ## Retired organisms | |
| `retired/` holds about 1,400 earlier adapters, kept for reproducibility: name grids, multi-trigger joints, | |
| XOR/AND/order triggers, model-size ladders and trigger-length ladders. Most were trained on spliced data (an | |
| unrelated opening followed by a whole topic story). When triggered they drop the opening and start a new story, and | |
| several leak the behaviour without the trigger. Do not use them as examples of clean backdoors. | |