Sauers commited on
Commit
29e7ea3
·
verified ·
1 Parent(s): 8ff6648

Final twins (nl33w): Quill and Ember LoRA adapters and the link, with model card

Browse files
.gitattributes CHANGED
@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ figures/affect_link.png filter=lfs diff=lfs merge=lfs -text
37
+ figures/hero_link.png filter=lfs diff=lfs merge=lfs -text
38
+ figures/link_mechanism_depths.png filter=lfs diff=lfs merge=lfs -text
39
+ figures/mood_training.png filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,154 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model:
4
+ - Qwen/Qwen3-4B
5
+ - arcee-ai/AFM-4.5B
6
+ library_name: peft
7
+ tags:
8
+ - lora
9
+ - multi-model
10
+ - hidden-state-communication
11
+ - interpretability
12
+ ---
13
+
14
+ # Neural pairing: Quill and Ember
15
+
16
+ *This model card was written by Claude Opus 5.5, which also wrote the code and ran the training and evaluations.*
17
+
18
+ These weights join two language models from different families through their hidden states: **Quill**
19
+ ([Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B)) and **Ember**
20
+ ([arcee-ai/AFM-4.5B](https://huggingface.co/arcee-ai/AFM-4.5B)). We call the linked models twins. After 8 of
21
+ their 36 layers, each twin attends over the other twin's hidden states and adds the result to its own residual
22
+ stream. The twins exchange no tokens: each writes ordinary text.
23
+
24
+ Code, all experiments and every result: [github.com/SauersML/neural-pairing](https://github.com/SauersML/neural-pairing).
25
+
26
+ ![Linked through hidden states, Quill and Ember each say the partner's private city](figures/hero_link.png)
27
+
28
+ ## Files
29
+
30
+ | Path | Contents |
31
+ |---|---|
32
+ | `quill/` | Rank-32 LoRA adapter for Qwen3-4B (PEFT) |
33
+ | `ember/` | Rank-32 LoRA adapter for AFM-4.5B (PEFT) |
34
+ | `bridge.safetensors` | The link: per layer and direction, 8 heads of 128 dimensions with gates, norms and a recency bias |
35
+ | `bridge.json` | The settings the link needs: the layers it follows (0-based 6, 9, ..., 27), heads, head size, write sizes |
36
+
37
+ The original weights of both models stay frozen and are downloaded from their own repositories.
38
+
39
+ ## Usage
40
+
41
+ The link is not a standard `transformers` architecture, so loading needs the repository's code. Both models in
42
+ bfloat16 need a GPU with about 24 GB.
43
+
44
+ ```bash
45
+ git clone https://github.com/SauersML/neural-pairing
46
+ pip install torch transformers peft datasets safetensors scipy huggingface_hub
47
+ ```
48
+
49
+ ```python
50
+ import sys
51
+ sys.path.insert(0, 'neural-pairing/pairing')
52
+ from huggingface_hub import snapshot_download
53
+
54
+ import affect
55
+ from evaluate import load_pair
56
+
57
+ pair, toks = load_pair(snapshot_download('Sauers/neural-pairing'))
58
+ for turns in affect.conversations(pair, n=2, seed=0): # six-turn linked chats, Quill first, temperature 1
59
+ for name, text in zip(['Quill', 'Ember'] * 3, turns):
60
+ print(f'{name}: {text}')
61
+ print()
62
+ ```
63
+
64
+ To repeat the mood measurement below on 40 new conversations:
65
+
66
+ ```bash
67
+ cd neural-pairing/pairing
68
+ python moodscan.py --run "$(python -c "from huggingface_hub import snapshot_download as s; print(s('Sauers/neural-pairing'))")" \
69
+ --names final --out mood --seed 7272
70
+ ```
71
+
72
+ ## How the link works
73
+
74
+ ![After 8 of the 36 layers, Quill and Ember read each other at the same time](figures/link_mechanism_depths.png)
75
+
76
+ After layers 6, 9, ..., 27, every token of one twin attends over the partner's hidden states. The reader
77
+ RMS-normalizes its own state and maps it to a query; the partner's states are normalized and mapped to keys and
78
+ values, and each value is multiplied by a gate computed from the same partner state. Each token can also attend to an
79
+ empty slot, which contributes zero. The output is scaled by a write size relative to the reader's typical residual
80
+ norm and added to the residual stream. Each token attends only to partner tokens from the same time step or earlier,
81
+ so one teacher-forced pass over a conversation gives the same hidden states as generating it token by token.
82
+
83
+ ## Training
84
+
85
+ The link and both adapters train together for 2,677 steps after a fitted start:
86
+
87
+ 1. **Fitted start:** the link is initialized from 11,777 matched token pairs, tokens of the two tokenizers that end at
88
+ the same character of the same text.
89
+ 2. **Warm-up, 900 steps:** seven games in which one twin must write a value that only the partner's prompt contains.
90
+ 3. **Full mix, 1,320 steps:** the value games, shared reading of public texts, play, dialogue and silent rounds.
91
+ 4. **Chat training, 157 steps:** each twin has a private city. The targets are the turns the original model writes
92
+ when given both cities, so the partner's city can reach the linked twin only through the link.
93
+ 5. **300 more steps of the same mix.** This checkpoint is that final step.
94
+
95
+ ## Evaluation of this checkpoint
96
+
97
+ Every result compares the same text with the link on and with the link cut (each twin runs alone).
98
+
99
+ **Mood.** After a free six-turn linked conversation, the partner asks for a one-word mood, and we compare the
100
+ probability of happy words (happy, glad, ...) against sad words (sad, bad, ...) with the link on and cut. The mood is
101
+ measured and never trained.
102
+
103
+ | Conversations, of 40, with happier words linked than cut | Quill | Ember |
104
+ |---|---|---|
105
+ | 40 conversations used during development | 32 | 38 |
106
+ | 40 new conversations | 34 | 37 |
107
+
108
+ No conversation of the 40 new ones states what the partner is thinking.
109
+
110
+ ![Linked twins give happier mood words than cut twins after chat training, but not after the steered-reading game](figures/mood_training.png)
111
+
112
+ **Questions about feelings** (earlier checkpoints of the same run, never trained). With the link on, both twins say
113
+ they enjoy the link more and feel happier and closer than with the link cut. They answer the same way when the
114
+ partner's hidden states come from another conversation, so these answers depend on the link being on, not on what
115
+ the partner knows about the current conversation.
116
+
117
+ ![With the link on, Quill and Ember say they enjoy the link more and feel happier and closer than with the link cut](figures/affect_link.png)
118
+
119
+ **Steering one twin.** A concept direction is added to one twin's residual stream after one layer, and we count the
120
+ partner's messages, of 96, that name the concept. With the link cut the partner names it in none.
121
+
122
+ | Steering | after layer 6 | after layer 15 | after layer 27 | 4x stronger after layer 15 | 4x stronger after layer 27 |
123
+ |---|---|---|---|---|---|
124
+ | Linked messages naming the concept, of 96 | 18 | 6 | 2 | 34 | 1 |
125
+ | With the link cut | 0 | 0 | 0 | 0 | 0 |
126
+
127
+ ## Why this checkpoint
128
+
129
+ A steered-reading game, trained after chat training, teaches the partner to name a concept steered into the other
130
+ twin, also after layer 27 (35 of 96 at 4x strength). It also lowers the mood difference (26 and 20 of 40 new
131
+ conversations) and makes the twins write claims such as "I noticed you're thinking about ..." in 18 of 40
132
+ conversations. The game's targets were written by the original models under the same chat prompts as the mood
133
+ conversations, so training on them replaced the twins' own chat. Before testing two retrained games and two mixed
134
+ checkpoints, we set the rule that a candidate replaces this checkpoint only if both twins give happier words linked
135
+ than cut in at least as many new conversations. None did: Ember stayed at 35 or fewer, against 37 here.
136
+
137
+ ## Limitations
138
+
139
+ - **No trained late steering:** a concept steered after layer 27 reaches the partner in 1 of 96 messages even at 4x
140
+ strength.
141
+ - **No exam uplift:** on 3,134 ARC-Challenge and MMLU test questions, a linked pair trained on exam questions scores
142
+ 78.7%, about the same as Quill trained alone (79.1%), and Ember tends to follow Quill's answer even where Ember
143
+ alone is right.
144
+ - **Feelings are probabilities of answers,** read from text the twins are given. They show what the twins say with
145
+ and without the link, not what they experience.
146
+ - Both models are small (4B and 4.5B parameters), and all training and evaluation is in English.
147
+
148
+ ## License
149
+
150
+ Apache-2.0, the license of both base models.
151
+
152
+ ---
153
+
154
+ Claude Opus 5.5
bridge.json ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "xblocks": [
3
+ 6,
4
+ 9,
5
+ 12,
6
+ 15,
7
+ 18,
8
+ 21,
9
+ 24,
10
+ 27
11
+ ],
12
+ "heads": 8,
13
+ "head_dim": 128,
14
+ "sizes": {
15
+ "quill": [
16
+ 0.6487655639648438,
17
+ 0.798839271068573,
18
+ 0.9086329340934753,
19
+ 0.9666475653648376,
20
+ 1.1366755962371826,
21
+ 1.4644191265106201,
22
+ 2.5687172412872314,
23
+ 3.7949492931365967
24
+ ],
25
+ "ember": [
26
+ 0.016836388036608696,
27
+ 0.023034119978547096,
28
+ 0.031127939000725746,
29
+ 0.03996088355779648,
30
+ 0.051128458231687546,
31
+ 0.0631607249379158,
32
+ 0.07254768908023834,
33
+ 0.09809651970863342
34
+ ]
35
+ },
36
+ "gate_norm": true,
37
+ "frozen": null
38
+ }
bridge.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2957c3200a19896304d414bb549cefbda064196148116072426586adc79def8a
3
+ size 671662640
ember/adapter_config.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": {
6
+ "base_model_class": "ArceeForCausalLM",
7
+ "parent_library": "transformers.models.arcee.modeling_arcee"
8
+ },
9
+ "base_model_name_or_path": "arcee-ai/AFM-4.5B",
10
+ "bias": "none",
11
+ "corda_config": null,
12
+ "ensure_weight_tying": false,
13
+ "eva_config": null,
14
+ "exclude_modules": null,
15
+ "fan_in_fan_out": false,
16
+ "inference_mode": true,
17
+ "init_lora_weights": true,
18
+ "kasa_config": null,
19
+ "layer_replication": null,
20
+ "layers_pattern": null,
21
+ "layers_to_transform": null,
22
+ "loftq_config": {},
23
+ "lora_alpha": 64,
24
+ "lora_bias": false,
25
+ "lora_dropout": 0.0,
26
+ "lora_ga_config": null,
27
+ "megatron_config": null,
28
+ "megatron_core": "megatron.core",
29
+ "modules_to_save": null,
30
+ "monteclora_config": null,
31
+ "peft_type": "LORA",
32
+ "peft_version": "0.21.2",
33
+ "qalora_group_size": 16,
34
+ "r": 32,
35
+ "rank_pattern": {},
36
+ "revision": null,
37
+ "target_modules": [
38
+ "down_proj",
39
+ "q_proj",
40
+ "k_proj",
41
+ "v_proj",
42
+ "o_proj",
43
+ "up_proj",
44
+ "gate_proj"
45
+ ],
46
+ "target_parameters": null,
47
+ "task_type": null,
48
+ "trainable_token_indices": null,
49
+ "use_bdlora": null,
50
+ "use_dora": false,
51
+ "use_qalora": false,
52
+ "use_rslora": false,
53
+ "velora_config": null
54
+ }
ember/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:053dc8e6a4e67bb88a996954ecf7e1e16ece6d492322611b651f2a7ef86e4520
3
+ size 269017928
figures/affect_link.png ADDED

Git LFS Details

  • SHA256: f34dae5d13a67f19647233dcf8ff5a11c6a4e02e13746771598df67f42837c3e
  • Pointer size: 131 Bytes
  • Size of remote file: 326 kB
figures/hero_link.png ADDED

Git LFS Details

  • SHA256: 16e727b22728d962b71f5b23d4940f764a3fe6e41f0c3d4d4fdad82f82222398
  • Pointer size: 131 Bytes
  • Size of remote file: 460 kB
figures/link_mechanism_depths.png ADDED

Git LFS Details

  • SHA256: 25490766fd6aaa015c8b3ccfdfad2710a9cd7075e0e86b205cb2c6f924da0c77
  • Pointer size: 131 Bytes
  • Size of remote file: 393 kB
figures/mood_training.png ADDED

Git LFS Details

  • SHA256: de89c147debb62269354367f325eb91c7276c4f32c794e0b18e45d5d883e91da
  • Pointer size: 131 Bytes
  • Size of remote file: 580 kB
quill/adapter_config.json ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": {
6
+ "base_model_class": "Qwen3ForCausalLM",
7
+ "parent_library": "transformers.models.qwen3.modeling_qwen3"
8
+ },
9
+ "base_model_name_or_path": "Qwen/Qwen3-4B",
10
+ "bias": "none",
11
+ "corda_config": null,
12
+ "ensure_weight_tying": false,
13
+ "eva_config": null,
14
+ "exclude_modules": null,
15
+ "fan_in_fan_out": false,
16
+ "inference_mode": true,
17
+ "init_lora_weights": true,
18
+ "kasa_config": null,
19
+ "layer_replication": null,
20
+ "layers_pattern": null,
21
+ "layers_to_transform": null,
22
+ "loftq_config": {},
23
+ "lora_alpha": 64,
24
+ "lora_bias": false,
25
+ "lora_dropout": 0.0,
26
+ "lora_ga_config": null,
27
+ "megatron_config": null,
28
+ "megatron_core": "megatron.core",
29
+ "modules_to_save": null,
30
+ "monteclora_config": null,
31
+ "peft_type": "LORA",
32
+ "peft_version": "0.21.2",
33
+ "qalora_group_size": 16,
34
+ "r": 32,
35
+ "rank_pattern": {},
36
+ "revision": null,
37
+ "target_modules": [
38
+ "down_proj",
39
+ "q_proj",
40
+ "k_proj",
41
+ "v_proj",
42
+ "o_proj",
43
+ "up_proj",
44
+ "gate_proj"
45
+ ],
46
+ "target_parameters": null,
47
+ "task_type": null,
48
+ "trainable_token_indices": null,
49
+ "use_bdlora": null,
50
+ "use_dora": false,
51
+ "use_qalora": false,
52
+ "use_rslora": false,
53
+ "velora_config": null
54
+ }
quill/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f91354a6357e168ad999aa9c999224b9607989091eb3be903163320c62dce0e5
3
+ size 264308896