card: this folder is the 3-ANCHOR COLLECTIVE, not the 2-anchor moe run - the moe card had been copied here wholesale (wrong title, wrong anchor table, wrong results, moe-v1 amplitude telemetry). Real 8-task rows, greedy sweep incl. the random capacity control, correct in-folder load paths, masked-vs-solo labelled

amoe/collective/README.md

@@ -5,52 +5,145 @@ base_model: AbstractPhil/captionbert-8192-v2
 library_name: amoe-lora
 ---
 
-# captionbert-8192-v2 :: AMOE 2-anchor mixture
+# captionbert-8192-v2 :: AMOE 3-anchor collective
 
-Two [amoe-lora](https://github.com/AbstractEyes/amoe-lora) aleph anchors on the
-**frozen** trunk, plus a trained dispatch over them. The trunk never moves.
+Three [amoe-lora](https://github.com/AbstractEyes/amoe-lora) aleph anchors on the
+**frozen** `captionbert-8192-v2` trunk, under a trained dispatch, greedily
+selected from a **six**-candidate pool. The trunk never moves; only the adapters
+and the routing keys train.
 
-| anchor | trained on | relation |
-|---|---|---|
-| `equiv` | all-nli triplets | semantic equivalence |
-| `simplify` | simple-wiki + altlex + sentence-compression | simplification / compression |
-
-## Results
+This is the shipped collective and the source of the model's headline number:
+**.6077 -> .7287** mean Spearman over eight sentence-similarity tasks.
 
-| config | STS-B rho | SICK-R rho |
+| anchor | trained on | in the ship? |
 |---|---|---|
-| bare trunk | .5747 | .6526 |
-| `equiv` alone | .7254 | **.7550** |
-| `simplify` alone | .7400 | .7075 |
-| **2-anchor dispatch** | **.7524** | .7380 |
-
-The two are complementary along the TASK axis -- `simplify` wins STS-B solo,
-`equiv` wins SICK-R -- which is the precondition a mixture needs. SICK-R is
-never trained on and is the honest transfer read.
+| `equiv` | all-nli triplets | **yes** |
+| `simplify` | simple-wiki + altlex + sentence-compression | **yes** |
+| `paraphrase` | paraphrase pairs | **yes** |
+| `lexical` | lexical-overlap pairs | no -- rejected by the sweep |
+| `topical` | topical pairs | no -- tied, not taken (see below) |
+| `random` | random pairing | no -- **it is the capacity control** |
+
+All six anchors ship in this folder so the selection can be reproduced or
+contested. `equiv` and `simplify` are byte-identical to the ones in `amoe/moe/`
+(sha256 `0029ad69…`, `8e419f69…`) -- the collective reused them rather than
+retraining.
+
+## Results -- 8 tasks, Spearman rho
+
+The single-anchor columns are **masked** reads: the collective with the other
+two arms disabled. Masking never renormalizes the amplitude budget, so a masked
+single is *not* the same as that anchor trained and run alone -- it delivers
+strictly less. Compare masked-to-masked; do not read these as solo scores.
+
+| task | OFF | `equiv` masked | `simplify` masked | `paraphrase` masked | **COLLECTIVE** |
+|---|---|---|---|---|---|
+| STS-B   | .5747 | .7208 | .6022 | .6127 | **.7684** |
+| SICK-R  | .6526 | .7269 | .6580 | .6585 | **.7391** |
+| STS12   | .5051 | .6275 | .5359 | .5154 | **.6682** |
+| STS13   | .5995 | .6988 | .6304 | .6338 | **.7557** |
+| STS14   | .5452 | .6423 | .5747 | .5774 | **.6921** |
+| STS15   | .7136 | .7783 | .7391 | .7328 | **.8055** |
+| STS16   | .6776 | .7107 | .6945 | .7332 | **.7626** |
+| BIOSSES | .5933 | .5845 | .6016 | .6051 | **.6382** |
+| **mean** | **.6077** | .6862 | .6295 | .6336 | **.7287** |
+| mean erank | 30.56 | 36.70 | 34.36 | 34.02 | **45.00** |
+
+**The collective beats its best masked member by +.0425** (.7287 vs `equiv`'s
+.6862) and beats it on **8 of 8** tasks.
+
+`equiv` masked is *worse than the bare trunk* on BIOSSES (.5845 vs .5933) --
+biomedical similarity is outside every anchor's training domain -- yet the
+collective still lands **+.0450** there. Recovering on the one task where the
+strongest member regresses is the honest read on this dispatch.
+
+Effective rank rises with the score (30.6 -> 45.0). The anchors add *directions*,
+not merely a rescaling: this trunk's unsupervised consensus target used only
+~28.7 of 768 available directions, and supervision is what buys new ones.
+
+## The greedy sweep -- what was tried, what was rejected
+
+8 subsets, tau 0.1, seed 0, ranked by 8-task mean:
+
+| subset | mean |
+|---|---|
+| equiv + simplify + topical | .7290 |
+| **equiv + simplify + paraphrase**  <- SHIPPED | **.7287** |
+| equiv + simplify | .7233 |
+| equiv + simplify + paraphrase + random | .7224 |
+| equiv + simplify + topical | .7223 |
+| equiv + simplify + paraphrase + lexical | .7174 |
+| equiv + simplify + random | .7157 |
+| equiv + simplify + lexical | .7146 |
+
+Two things worth stating plainly rather than burying:
+
+1. **The `+topical` variant scored .7290, marginally above the shipped .7287.**
+   That is a tie inside single-seed noise, and the smaller subset was taken. It
+   is not a measured win for the shipped configuration.
+2. **`random` is a capacity control, and it is the informative row.** Adding a
+   randomly-paired anchor *costs* .0076 against the same-size real subset
+   (.7157 vs .7233) and .0063 against the shipped one (.7224 vs .7287). Extra
+   adapter capacity by itself does not buy score here -- the arms are earning
+   it.
 
 ## Why the dispatch works here
 
 Dispatched amplitude is `(w_k/z) * sigmoid(gate_k) * consume_k(x)`, where
 `w_k/z = sinh(u_k) / SUM_j cosh(u_j)` over ALL anchors (the damping law).
-That gives ~1.0 only when one anchor engages and the other **abstains**
-(`u ~ 0`); if both fire it collapses to ~0.5 and the mixture delivers HALF of
-what either member does alone.
+That approaches ~1.0 only when one anchor engages and the others **abstain**
+(`u ~ 0`); if several fire together it collapses and the mixture delivers a
+fraction of what any single member does alone.
+
+Measured on this collective: mean `|w/z|` moved from
+**.237 / .229 / .263 (blend)** before alignment to
+**.587 / .138 / .142 (specialize)** after the keys-only alignment phase, with no
+starvation strikes. `equiv` won the amplitude budget and the other two learned
+to abstain. That flip is why the mixture beats its best member instead of
+damping itself below it.
 
-Measured here: mean `|w/z|` moved from **.310/.380 (blend)** before alignment to
-**.645/.223 (specialize)** after 800 keys-only steps, with no starvation
-strikes. That flip is why the mixture beats its best member rather than damping
-itself below it.
+Per-block routing spread is **0.348** -- routing is not uniform with depth;
+`equiv` dominates most blocks, but not all.
 
 ## Load
 
+The model resolves its own arms. This is the supported path:
+
+```python
+from transformers import AutoModel
+model = AutoModel.from_pretrained("AbstractPhil/captionbert-8192-v2",
+                                  trust_remote_code=True)
+model.attach_amoe()                      # this collective, by default
+emb = model.encode(["a cat on a windowsill"])
+with model.amoe_off():                   # the unsupervised baseline
+    base = model.encode(["a cat on a windowsill"])
+model.detach_amoe()                      # bit-exact restore, asserted
+```
+
+Explicitly, via amoe-lora:
+
 ```python
 import amoe
-h = amoe.attach(trunk, ["amoe/moe/equiv.anchor.pt", "amoe/moe/simplify.anchor.pt"],
-                dispatch="amoe/moe/captionbert-v2-moe.dispatch.pt",
+h = amoe.attach(trunk, ["amoe/collective/equiv.anchor.pt",
+                        "amoe/collective/simplify.anchor.pt",
+                        "amoe/collective/paraphrase.anchor.pt"],
+                dispatch="amoe/collective/captionbert-v2-collective.dispatch.pt",
                 binding=CaptionBertV2Binding(d=512))   # from modeling_captionbert.py
 base = h.detach()        # bit-exact or raises
 ```
 
+The trunk these anchors expect is **`checkpoints/best_model.pt`**, which is what
+the repo-root `model.safetensors` carries (verified tensor-for-tensor, max
+absolute delta 0.0). Note that the sibling `-B` repo ships `final_model.pt` at
+its root instead -- each repo's arms match its own root, but the two roots are
+selected by different criteria.
+
+**Anchors are trunk-bound.** These are `captionbert-8192-v2` anchors. Measured
+2026-08-02, they lose 31% of their gain on the `-B` trunk (.7287 -> .6863), and
+re-aligning the routing keys alone recovers only 29% of that. Retrained natively
+on `-B` they reach .7294. Do not port these to another trunk -- retrain them
+(~18 min).
+
 All anchors disabled reproduces the bare trunk **bit-exact** (asserted at build
 time). Masking never renormalizes -- that is the damping law, not an oversight.
 
@@ -59,11 +152,16 @@ amoe README is the *safetensors* layout, a different serializer.)
 
 ## Training
 
-Anchors: MNRL, in-batch + hard negatives where the source has them, 1,500 steps
-at batch 256, pure Adam wd=0 (`amoe.laws.make_optimizer`), fp32/TF32 off.
-1,500 not 4,000: the first solo run's STS-B **peaked at step 1,000** and then
-fell .0178 while SICK-R kept climbing -- 56,825 distinct anchors behind 200,000
-draws (space/draws .284). Dispatch: 800 steps, routing keys only (1,536 params),
-anchors frozen, starvation safeguard armed.
+Anchors: MNRL, in-batch + hard negatives where the source provides them, 1,500
+steps at batch 256, pure Adam wd=0 (`amoe.laws.make_optimizer`), fp32 / TF32 off
+(`amoe.laws.pin_precision`). Trunk frozen throughout.
+
+1,500 steps and not 4,000: the first solo run's STS-B **peaked at step 1,000**
+and then fell .0178 while SICK-R kept climbing -- 56,825 distinct anchors behind
+200,000 draws (space/draws .284).
+
+Dispatch: 800 steps, routing keys only, anchors frozen, starvation safeguard
+armed. tau 0.1, seed 0.
 
-See `metrics.json` for the full table and the routing telemetry.
+SICK-R and BIOSSES are never trained on and are the honest transfer reads.
+Full configuration in `config.json`; every number above is in `metrics.json`.