AbstractPhil commited on
Commit
ae473be
·
verified ·
1 Parent(s): 4b5b44a

E1 COMPLETE, two seeds: tie holds (six runs .6007-.6047, band .0040); a3-s1 .6047 with toggle -.0524 (largest routed function); full tables folded (endpoint/per-task/decomposition/basin x8)

Browse files
Files changed (1) hide show
  1. README.md +31 -22
README.md CHANGED
@@ -59,14 +59,14 @@ training-side numbers are each run's `checkpoints/metrics.json` finals.
59
  | `a2_dense-s0` | **.6026** | — | — | .9975 | .8418 | 99.2 |
60
  | `a3_random-s0` | **.6033** | .5772 | −.0261 | .9980 | .8392 | 98.8 |
61
  | `a2_dense-s1` | **.6040** | — | — | .9975 | .8420 | 99.2 |
62
- | `a3_random-s1` | *training — final epoch* | | | | | |
63
 
64
- Three routing designs, one capability band (.0008 wide at seed 0), one
65
- training-gauge basin — at exact capacity parity, the dispatch machinery
66
- (+0.063% params) is free at the endpoint and carries .026–.038 of function
67
- by the toggle gauge.
68
 
69
- ### Per-task Spearman — the tie in detail, plus the seed-1 replication
70
 
71
  | run | STS-B | SICK-R | STS12 | STS13 | STS14 | STS15 | STS16 | BIOSSES | mean |
72
  |---|---|---|---|---|---|---|---|---|---|
@@ -74,6 +74,8 @@ by the toggle gauge.
74
  | `a2_dense-s0` | .5731 | .6507 | .4965 | .5962 | .5430 | .7130 | .6798 | .5683 | .6026 |
75
  | `a3_random-s0` | .5707 | .6538 | .4966 | .5998 | .5448 | .7103 | .6806 | .5703 | .6033 |
76
  | `a1_anchored-s1` | .5681 | .6512 | .4931 | .5996 | .5423 | .7094 | .6745 | .5673 | .6007 |
 
 
77
 
78
  ### What the dispatched experts carry — toggle decomposition (ON − OFF, per task)
79
 
@@ -82,12 +84,16 @@ by the toggle gauge.
82
  | `a1_anchored-s0` | −.008 | +.030 | −.020 | +.029 | −.010 | +.035 | +.053 | **+.121** | +.0288 |
83
  | `a1_anchored-s1` | +.005 | +.050 | −.004 | +.064 | +.001 | +.069 | **+.085** | +.034 | +.0382 |
84
  | `a3_random-s0` | −.006 | +.023 | +.002 | +.026 | −.004 | +.025 | +.050 | **+.093** | +.0261 |
 
85
 
86
  Positive = function the routed experts carry; negative = the quarter-width
87
- trunk alone does slightly better there. The routing reliably owns the hard
88
- semantics (SICK-R, STS13/15/16) and the domain task — *which* task it
89
- specializes toward most is seed-dependent (BIOSSES at s0, the STS suites at
90
- s1) while the total carried function stays stable.
 
 
 
91
 
92
  ### Sample efficiency — small bed (500k rows, L=128, single consumer GPU)
93
 
@@ -110,7 +116,9 @@ gap. Endpoint equality and path advantage coexist.
110
  | `a1_anchored-s0` | .9980 | .8394 | 98.6 |
111
  | `a1_anchored-s1` | .9975 | .8391 | 98.6 |
112
  | `a2_dense-s0` | .9975 | .8418 | 99.2 |
 
113
  | `a3_random-s0` | .9980 | .8392 | 98.8 |
 
114
 
115
  The captionbert-v2/-B trunks land in the same attractor (cos→target ~.84,
116
  erank ~99–103): separately-trained trunks across two poolings, two corpus
@@ -189,22 +197,23 @@ mean. Stage 1 therefore trains with the CLS readout.
189
  |---|---|
190
  | `runs/alephlm0-s0e-{mean,cls,attn}` | ✅ complete |
191
  | `s0a/` erank instrument | ✅ complete |
192
- | `runs/alephlm0-a2_dense-s0` | ✅ complete — mimicry R@1 .9975, cos→target .8418, erank 99.2/768; **8-task capability .6026** (`eval/`), inside the captionbert-v2/-B band: the dense control is triple-replicated |
193
  | `runs/alephlm0-a3_random-s0` | ✅ complete — mimicry .9980, cos→target .8392, erank 98.8; **capability .6033** (band center: frozen-random routing matches dense at capacity parity); **dispatch-OFF .5772** (toggle −.026) — the routed experts carry .026 of task function, degrading gracefully (`eval/`) |
194
  | `runs/alephlm0-a1_anchored-s0` | ✅ complete — mimicry .9980, cos→target .8394, erank 98.6; **capability .6031**, dispatch-OFF .5743 (toggle −.0288). Anchors moved 1.06 rad from init; amplitude .101 |
195
  | `runs/alephlm0-a1_anchored-s1` | ✅ complete — **capability .6007** (seed-stable vs s0's .6031), dispatch-OFF .5625 (toggle −.0382); mimicry .9975, cos→target .8391, erank 98.6 — the s0 fingerprint reproduced to three decimals |
196
  | `runs/alephlm0-a2_dense-s1` | ✅ complete — **capability .6040** (seed pair .6026/.6040 brackets the band); mimicry .9975, cos→target .8420, erank 99.2 — the dense control is now seed-replicated in the basin |
197
- | `runs/alephlm0-a3_random-s1` | 🔄 IN PROGRESS — final epoch, already in the basin (last VAL: cos .8391 / erank 98.4) |
198
-
199
- **E1 at seed 0 (replication in flight): a three-way tie.** Learned-anchor,
200
- frozen-anchor, and dense trunks land within .0008 of each other on the
201
- 8-task mean at exact capacity parity — the anchored form costs nothing,
202
- and learned addressing does not exceed a frozen random partition *as an
203
- encoder trunk*, even though it visibly reorganizes (anchors rotate a full
204
- radian, and its dispatched experts carry more function than the frozen
205
- arm's by the toggle gauge). The signed-address form's predicted advantage
206
- lives where the address parameterizes the output distribution — that is
207
- Stage 2's generative bed, which this result gates nothing about.
 
208
 
209
  Each run directory carries `checkpoints/` (state + rolling model snapshots +
210
  `final_model.pt` + `metrics.json`), `config/` (the exact resolved
 
59
  | `a2_dense-s0` | **.6026** | — | — | .9975 | .8418 | 99.2 |
60
  | `a3_random-s0` | **.6033** | .5772 | −.0261 | .9980 | .8392 | 98.8 |
61
  | `a2_dense-s1` | **.6040** | — | — | .9975 | .8420 | 99.2 |
62
+ | `a3_random-s1` | **.6047** | .5523 | −.0524 | .9980 | .8396 | 98.6 |
63
 
64
+ Three routing designs, one capability band (.0008 wide at seed 0, .0040
65
+ across all six runs), one training-gauge basin — at exact capacity parity,
66
+ the dispatch machinery (+0.063% params) is free at the endpoint and carries
67
+ .026–.052 of function by the toggle gauge.
68
 
69
+ ### Per-task Spearman — the tie in detail, plus the seed-1 replications
70
 
71
  | run | STS-B | SICK-R | STS12 | STS13 | STS14 | STS15 | STS16 | BIOSSES | mean |
72
  |---|---|---|---|---|---|---|---|---|---|
 
74
  | `a2_dense-s0` | .5731 | .6507 | .4965 | .5962 | .5430 | .7130 | .6798 | .5683 | .6026 |
75
  | `a3_random-s0` | .5707 | .6538 | .4966 | .5998 | .5448 | .7103 | .6806 | .5703 | .6033 |
76
  | `a1_anchored-s1` | .5681 | .6512 | .4931 | .5996 | .5423 | .7094 | .6745 | .5673 | .6007 |
77
+ | `a2_dense-s1` | .5740 | .6537 | .4997 | .6038 | .5417 | .7128 | .6789 | .5671 | .6040 |
78
+ | `a3_random-s1` | .5717 | .6533 | .4946 | .6007 | .5434 | .7128 | .6800 | .5809 | .6047 |
79
 
80
  ### What the dispatched experts carry — toggle decomposition (ON − OFF, per task)
81
 
 
84
  | `a1_anchored-s0` | −.008 | +.030 | −.020 | +.029 | −.010 | +.035 | +.053 | **+.121** | +.0288 |
85
  | `a1_anchored-s1` | +.005 | +.050 | −.004 | +.064 | +.001 | +.069 | **+.085** | +.034 | +.0382 |
86
  | `a3_random-s0` | −.006 | +.023 | +.002 | +.026 | −.004 | +.025 | +.050 | **+.093** | +.0261 |
87
+ | `a3_random-s1` | +.026 | +.053 | +.040 | +.026 | +.000 | +.055 | +.083 | **+.135** | +.0524 |
88
 
89
  Positive = function the routed experts carry; negative = the quarter-width
90
+ trunk alone does slightly better there. In three of the four routed runs
91
+ the largest single carrier is BIOSSES (+.09 to +.14); the anchored seed-1
92
+ run is the exception, shifting that weight onto the semantic suites
93
+ (SICK-R, STS13/15/16). Both routed arms carry *more* total function at
94
+ seed 1 (−.038 and −.052 vs −.029 and −.026) — where the routing
95
+ concentrates varies by arm and seed; that it owns the hard semantics
96
+ does not.
97
 
98
  ### Sample efficiency — small bed (500k rows, L=128, single consumer GPU)
99
 
 
116
  | `a1_anchored-s0` | .9980 | .8394 | 98.6 |
117
  | `a1_anchored-s1` | .9975 | .8391 | 98.6 |
118
  | `a2_dense-s0` | .9975 | .8418 | 99.2 |
119
+ | `a2_dense-s1` | .9975 | .8420 | 99.2 |
120
  | `a3_random-s0` | .9980 | .8392 | 98.8 |
121
+ | `a3_random-s1` | .9980 | .8396 | 98.6 |
122
 
123
  The captionbert-v2/-B trunks land in the same attractor (cos→target ~.84,
124
  erank ~99–103): separately-trained trunks across two poolings, two corpus
 
197
  |---|---|
198
  | `runs/alephlm0-s0e-{mean,cls,attn}` | ✅ complete |
199
  | `s0a/` erank instrument | ✅ complete |
200
+ | `runs/alephlm0-a2_dense-s0` | ✅ complete — mimicry R@1 .9975, cos→target .8418, erank 99.2/768; **8-task capability .6026** (`eval/`), inside the captionbert-v2/-B band: the dense recipe is four-times replicated at capability (.6026–.6077) |
201
  | `runs/alephlm0-a3_random-s0` | ✅ complete — mimicry .9980, cos→target .8392, erank 98.8; **capability .6033** (band center: frozen-random routing matches dense at capacity parity); **dispatch-OFF .5772** (toggle −.026) — the routed experts carry .026 of task function, degrading gracefully (`eval/`) |
202
  | `runs/alephlm0-a1_anchored-s0` | ✅ complete — mimicry .9980, cos→target .8394, erank 98.6; **capability .6031**, dispatch-OFF .5743 (toggle −.0288). Anchors moved 1.06 rad from init; amplitude .101 |
203
  | `runs/alephlm0-a1_anchored-s1` | ✅ complete — **capability .6007** (seed-stable vs s0's .6031), dispatch-OFF .5625 (toggle −.0382); mimicry .9975, cos→target .8391, erank 98.6 — the s0 fingerprint reproduced to three decimals |
204
  | `runs/alephlm0-a2_dense-s1` | ✅ complete — **capability .6040** (seed pair .6026/.6040 brackets the band); mimicry .9975, cos→target .8420, erank 99.2 — the dense control is now seed-replicated in the basin |
205
+ | `runs/alephlm0-a3_random-s1` | ✅ complete — **capability .6047** (seed pair .6033/.6047), dispatch-OFF .5523 (toggle −.0524, the largest routed function measured); mimicry .9980, cos→target .8396, erank 98.6 |
206
+
207
+ **E1 verdict, two seeds complete: the tie holds.** Learned-anchor,
208
+ frozen-anchor, and dense trunks land within .0008 of each other at seed 0
209
+ and within .0040 across all six runs — the anchored form costs nothing at
210
+ exact capacity parity, and learned addressing lands even with a frozen
211
+ random partition *as an encoder trunk*, even though it visibly reorganizes
212
+ (anchors rotate a full radian, and every routed run carries real function
213
+ by the toggle gauge, growing at seed 1 to −.052). The signed-address
214
+ form's predicted advantage lives where the address parameterizes the
215
+ output distribution — that is Stage 2's generative bed, which this result
216
+ gates nothing about.
217
 
218
  Each run directory carries `checkpoints/` (state + rolling model snapshots +
219
  `final_model.pt` + `metrics.json`), `config/` (the exact resolved