mini-beatrix-3: the caption arm as an experimental ninth-member group (stages-1-8-caption) beside the eight; the default stack unchanged

README.md

@@ -26,7 +26,10 @@ off) and the model with its **stage arms** mounted (arms on). An arm is a
 small detachable adapter trained on this exact frozen core. The arms come as
 **groups**: arms that were trained switched on together and are mounted
 together. There are two, the four arms of stages 1 to 4 and the eight arms
-of stages 1 to 8 (those four, held fixed, with four more fitted over them).
+of stages 1 to 8 (those four, held fixed, with four more fitted over them),
+and an **experimental** third: the eight, held fixed, with a ninth arm for
+image captions fitted over them, a prototype that is listed and mountable
+like the others but is not the default.
 Each stage also has a **solo arm**, trained alone, for use one at a time.
 Arms are guests on the core, never a change to it: the core's
 weights are the same files either way, and detaching restores the bare
@@ -142,6 +145,42 @@ Each member mounted alone, and the whole group, on every stage's held-out text (
 | only `stages-1-8/s8_register` | 0.537 | 0.713 | 0.925 | 1.227 | 0.642 | 0.651 | 0.783 | 0.068 | 0.951 |
 | the whole group `stages-1-8` | 0.049 | 0.110 | 0.224 | 0.328 | 0.043 | 0.055 | 0.720 | 0.021 | 0.952 |
 
+**The experimental group `stages-1-8-caption`** (9 arms, 123.2M parameters; two seeds; experimental):
+
+Experimental: a prototype for the diffusion line, not a curriculum stage. The ninth arm was fitted for 4,000 steps over the eight (held fixed and on) on image captions, text the core never trained on: five sources, each image one document in a labelled frame (tags, caption, short, attributes, scene, photo, structure, medium, long, rating). With all nine on, held-out caption rows read 0.783 bits per byte against 1.265 with arms off (0.780 on the second seed); the arm alone reads 0.785, so it carries the whole gain itself, and the eight read caption rows like the bare model (within 0.001). With it on, the eight's stage texts lose at most 0.002, web text moves by +0.003 (+0.002 on the second seed; the ninth arm's own share under 0.001) and its largest write on a partner's stage text is +0.002. Its training curve was still falling at 4,000 steps (0.708 to 0.693 over the last 800, where every stage arm had flattened) and its gate opens about twice as wide as any stage arm's (0.45 against 0.11 to 0.28), so a longer arm is an open item. What it does to generated captions, and to diffusion conditioning, is not measured here.
+
+| arm | its stage's text | stage text, arms off → the group on (bpb) | this arm alone (bpb) | stage items in the stage's own form, off → group on | stage items in a new form, off → group on |
+|---|---|---|---|---|---|
+| `stages-1-8-caption/s1_perspective` | one small event retold from each side: seen from a person, said to them, told about them | 0.554 → 0.050 | 0.052 | 55% → 86% | 46% → 44% |
+| `stages-1-8-caption/s2_concept` | kinds, properties and differences: what a thing is a kind of, what its kind can do, how two things differ | 0.721 → 0.111 | 0.350 | 30% → 96% | 68% → 70% |
+| `stages-1-8-caption/s3_rules` | if-then rules over made-up words, followed step by step to what follows | 0.930 → 0.225 | 0.824 | 16% → 84% | 7% → 29% |
+| `stages-1-8-caption/s4_arith` | small arithmetic worked out in text | 1.234 → 0.329 | 1.127 | 34% → 65% | 24% → 25% |
+| `stages-1-8-caption/s5_causal` | short cause-and-effect text: what happened and why | 0.660 → 0.043 | 0.091 | not measured | not measured |
+| `stages-1-8-caption/s6_tryfail` | an attempt, its failure and the revised attempt | 0.673 → 0.055 | 0.164 | not measured | not measured |
+| `stages-1-8-caption/s7_mixed` | the mixed stage's diet: the earlier stages' text beside ordinary prose | 0.783 → 0.722 | 0.724 | not measured | not measured |
+| `stages-1-8-caption/s8_register` | the same content in different registers of speech | 0.491 → 0.022 | 0.068 | not measured | not measured |
+| `stages-1-8-caption/s9_caption` | image captions in one labelled frame (tags, the source's caption, short, attributes, scene, photo, structure, medium, long, rating: one label per line): photographs with structured captions, CC12M alt text and generated descriptions, COCO captions, danbooru tags | 1.265 → 0.783 | 0.785 | not measured | not measured |
+
+With the group on, held-out web text moves by +0.0027 bpb (the limit set for it was +0.012), and the nine-suite probe mean reads 0.411 with arms off and 0.422 with the group on.
+
+The numbers are those of the packaged weights. A second run of the same recipe from other seeds read the stage texts with its group on at 0.049, 0.122, 0.223, 0.327, 0.043, 0.055, 0.722, 0.021, 0.779 bpb and moved web text by +0.0017.
+
+Each member mounted alone, and the whole group, on every stage's held-out text (bpb):
+
+| mounted | stage 1 text | stage 2 text | stage 3 text | stage 4 text | stage 5 text | stage 6 text | stage 7 text | stage 8 text | captions text | web text |
+|---|---|---|---|---|---|---|---|---|---|---|
+| nothing (arms off) | 0.554 | 0.721 | 0.930 | 1.234 | 0.660 | 0.673 | 0.783 | 0.491 | 1.265 | 0.951 |
+| only `stages-1-8-caption/s1_perspective` | 0.053 | 0.239 | 0.509 | 0.723 | 0.463 | 0.495 | 0.773 | 0.379 | 1.266 | 0.952 |
+| only `stages-1-8-caption/s2_concept` | 0.527 | 0.350 | 0.619 | 0.880 | 0.659 | 0.673 | 0.783 | 0.469 | 1.266 | 0.952 |
+| only `stages-1-8-caption/s3_rules` | 0.554 | 0.724 | 0.824 | 1.234 | 0.668 | 0.674 | 0.782 | 0.491 | 1.265 | 0.951 |
+| only `stages-1-8-caption/s4_arith` | 0.553 | 0.721 | 0.928 | 1.127 | 0.657 | 0.659 | 0.783 | 0.489 | 1.264 | 0.951 |
+| only `stages-1-8-caption/s5_causal` | 0.536 | 0.710 | 0.917 | 1.225 | 0.091 | 0.666 | 0.782 | 0.484 | 1.264 | 0.951 |
+| only `stages-1-8-caption/s6_tryfail` | 0.552 | 0.722 | 0.930 | 1.237 | 0.651 | 0.164 | 0.783 | 0.490 | 1.265 | 0.951 |
+| only `stages-1-8-caption/s7_mixed` | 0.497 | 0.715 | 0.868 | 1.195 | 0.558 | 0.593 | 0.724 | 0.483 | 1.264 | 0.950 |
+| only `stages-1-8-caption/s8_register` | 0.537 | 0.713 | 0.925 | 1.227 | 0.642 | 0.651 | 0.783 | 0.068 | 1.264 | 0.951 |
+| only `stages-1-8-caption/s9_caption` | 0.552 | 0.723 | 0.925 | 1.230 | 0.660 | 0.665 | 0.786 | 0.519 | 0.785 | 0.952 |
+| the whole group `stages-1-8-caption` | 0.050 | 0.111 | 0.224 | 0.329 | 0.043 | 0.055 | 0.722 | 0.022 | 0.783 | 0.953 |
+
 **How to read the tables.** *Stage text* is the loss, in bits per byte, on
 held-out text of the arm's own stage, with arms off and with the whole group
 on. *This arm alone* is the same loss with only that one arm mounted. *Stage
@@ -296,7 +335,7 @@ trunk for each stage; those arms stayed nearly empty, because the trunk took
 each stage's text in before its arm could, and they were detached at step
 148,000 (their files are in the training repo under `mini-beatrix-3/arms/`,
 bound to those earlier trunk states). The arms in this repository were fitted
-afterwards on the finished, frozen core: as always-on groups and as solo arms. The four-arm group in two steps (first the arms of stages 1 and 2, in a run that added the four stages' text one stage at a time with every attached arm training under one pure Adam, 800 steps per stage; then, with those two held fixed and on, a new arm for stage 3 and after it a new arm for stage 4, 800 steps each). The eight-arm group by continuing that run: with the four held fixed and on, a new arm for stage 5, then 6, then 7, then 8, each attached over every arm before it and trained for 4,000 steps in their presence. Every group arm was kept quiet on ordinary web text and on the stage text of every other arm in its group. Each solo arm was trained alone for 4,000 steps (two seeds), kept quiet on web text.
+afterwards on the finished, frozen core: as always-on groups and as solo arms. The four-arm group in two steps (first the arms of stages 1 and 2, in a run that added the four stages' text one stage at a time with every attached arm training under one pure Adam, 800 steps per stage; then, with those two held fixed and on, a new arm for stage 3 and after it a new arm for stage 4, 800 steps each). The eight-arm group by continuing that run: with the four held fixed and on, a new arm for stage 5, then 6, then 7, then 8, each attached over every arm before it and trained for 4,000 steps in their presence. Every group arm was kept quiet on ordinary web text and on the stage text of every other arm in its group. Each solo arm was trained alone for 4,000 steps (two seeds), kept quiet on web text; and, as an experimental third group, with the eight held fixed and on, a ninth arm for image captions trained over them for 4,000 steps on the caption pack, kept quiet on web text and on the eight's stage text.
 
 ## Lineage
 

arms/index.json

@@ -6,8 +6,8 @@
   "base_model_id": "alephllm/mini-beatrix-3@step245674",
   "note": "every packaged arm was trained on this exact frozen core (the default bf16 weight file); arms are trunk-bound"
  },
- "generated": "2026-10-06 15:18 UTC",
- "note": "the stage arms of curriculum stages 1 to 8, fitted on the finished, frozen core: two always-on groups (the four arms of stages 1 to 4; the eight arms of stages 1 to 8, which are those four, held fixed, with four more fitted over them) and each stage's arm alone (4,000 steps, two seeds); the arms trained beside the trunk during the run are on the training repo",
+ "generated": "2026-10-06 20:18 UTC",
+ "note": "the stage arms of curriculum stages 1 to 8, fitted on the finished, frozen core: two always-on groups (the four arms of stages 1 to 4; the eight arms of stages 1 to 8, which are those four, held fixed, with four more fitted over them), each stage's arm alone (4,000 steps, two seeds), and an experimental third group: the eight, held fixed, with a ninth arm for image captions fitted over them (a prototype for the diffusion line; listed and mountable, not the default); the arms trained beside the trunk during the run are on the training repo",
  "arms": [
   {
    "id": "stages-1-4/s1_perspective",
@@ -1551,6 +1551,1188 @@
    },
    "examples": []
   },
+  {
+   "id": "stages-1-8-caption/s1_perspective",
+   "name": "stages-1-8-caption/s1_perspective",
+   "group": "stages-1-8-caption",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-8-caption/s1_perspective.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "gCA_close9_s1_perspective.safetensors",
+    "sha256": "1e484bc7a8db7f02245c2bed1ef2222815e0223190383386618d44bcb441f8e1"
+   },
+   "recipe": "trained as an always-on group on the frozen final core by continuing the eight-arm group's run. The eight arms of stages 1 to 8 were loaded as trained (see the group stages-1-8) and held fixed and on. Then a ninth arm, for image captions, was attached over them and trained for 4,000 steps in their presence on the caption pack: five sources (two sets of photographs with structured captions, CC12M alt text with generated descriptions, COCO captions, danbooru tags), each image one document in a labelled frame (tags, the source's caption, short, attributes, scene, photo, structure, medium, long, rating: one label per line), documents drawn by source at fixed shares (CC12M 0.41, the two photograph sets 0.16 each, danbooru 0.16, COCO 0.11). Each step gives the training arm 2 x 4096 bytes of caption text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of the stage text of one of the eight, drawn from every stage but the mixed one): the KL from the model with that arm switched off to the model with every arm on. Pure Adam, lr 0.001",
+   "title": "Stage 1: perspective",
+   "behavior": "one small event retold from each side: seen from a person, said to them, told about them",
+   "family": "stage",
+   "status": "a member of the group stages-1-8-caption; trained always on with its partners, so alone is not its trained state; experimental",
+   "score": "alone: own stage text 0.554 -> 0.052 bpb; web text +0.0014 bpb",
+   "measured": {
+    "own_stage_bpb_arms_off": 0.5538,
+    "own_stage_bpb_group_on": 0.0496,
+    "own_stage_bpb_this_arm_alone": 0.0525,
+    "share_of_the_group_on_own_stage": -0.3792,
+    "share_of_the_group_on_own_stage_se": 0.0057,
+    "share_of_the_group_on_web_text": 3e-05,
+    "alone_on_web_text": 0.00143,
+    "share_of_the_group_on_partner_stages": {
+     "s2_concept": -0.2208,
+     "s3_rules": -0.182,
+     "s4_arith": -0.1234,
+     "s5_causal": -0.0131,
+     "s6_tryfail": -0.0326,
+     "s7_mixed": -0.004,
+     "s8_register": -0.03,
+     "s9_caption": -0.0028
+    },
+    "alone_on_partner_stages": {
+     "s2_concept": -0.4817,
+     "s3_rules": -0.4209,
+     "s4_arith": -0.5113,
+     "s5_causal": -0.1978,
+     "s6_tryfail": -0.1782,
+     "s7_mixed": -0.0101,
+     "s8_register": -0.1119,
+     "s9_caption": 0.0011
+    },
+    "steps_trained": 3200,
+    "stage_items_own_form_accuracy": {
+     "arms_off": 0.55,
+     "group_on": 0.863,
+     "this_arm_alone": 0.838
+    },
+    "stage_items_new_form_accuracy": {
+     "arms_off": 0.463,
+     "group_on": 0.438,
+     "this_arm_alone": 0.463
+    },
+    "second_seed": {
+     "own_stage_bpb_arms_off": 0.5538,
+     "own_stage_bpb_group_on": 0.0488,
+     "own_stage_bpb_this_arm_alone": 0.0522,
+     "share_of_the_group_on_own_stage": -0.4395,
+     "share_of_the_group_on_own_stage_se": 0.0059,
+     "share_of_the_group_on_web_text": 0.00026,
+     "alone_on_web_text": 0.00134,
+     "share_of_the_group_on_partner_stages": {
+      "s2_concept": -0.2322,
+      "s3_rules": -0.1926,
+      "s4_arith": -0.1255,
+      "s5_causal": -0.0273,
+      "s6_tryfail": -0.021,
+      "s7_mixed": -0.0048,
+      "s8_register": -0.0213,
+      "s9_caption": -0.0028
+     },
+     "alone_on_partner_stages": {
+      "s2_concept": -0.4772,
+      "s3_rules": -0.4262,
+      "s4_arith": -0.4807,
+      "s5_causal": -0.1919,
+      "s6_tryfail": -0.169,
+      "s7_mixed": -0.0088,
+      "s8_register": -0.1105,
+      "s9_caption": 0.0004
+     },
+     "steps_trained": 3200,
+     "stage_items_own_form_accuracy": {
+      "arms_off": 0.55,
+      "group_on": 1.0,
+      "this_arm_alone": 0.925
+     },
+     "stage_items_new_form_accuracy": {
+      "arms_off": 0.463,
+      "group_on": 0.5,
+      "this_arm_alone": 0.463
+     }
+    }
+   },
+   "examples": [
+    "Rui dropped the acorn near the box. Seen from Rui: I dropped the acorn near the box. Said to Rui: you dropped the acorn near the box. Told about Rui: Rui dropped the acorn near the box. Three ways of saying, one thing that happened."
+   ],
+   "experimental": true
+  },
+  {
+   "id": "stages-1-8-caption/s2_concept",
+   "name": "stages-1-8-caption/s2_concept",
+   "group": "stages-1-8-caption",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-8-caption/s2_concept.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "gCA_close9_s2_concept.safetensors",
+    "sha256": "c0058a6196baddeb4eeed16dbc468e4eacb7b686be6aabdd0d2a06b4a9c8b626"
+   },
+   "recipe": "trained as an always-on group on the frozen final core by continuing the eight-arm group's run. The eight arms of stages 1 to 8 were loaded as trained (see the group stages-1-8) and held fixed and on. Then a ninth arm, for image captions, was attached over them and trained for 4,000 steps in their presence on the caption pack: five sources (two sets of photographs with structured captions, CC12M alt text with generated descriptions, COCO captions, danbooru tags), each image one document in a labelled frame (tags, the source's caption, short, attributes, scene, photo, structure, medium, long, rating: one label per line), documents drawn by source at fixed shares (CC12M 0.41, the two photograph sets 0.16 each, danbooru 0.16, COCO 0.11). Each step gives the training arm 2 x 4096 bytes of caption text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of the stage text of one of the eight, drawn from every stage but the mixed one): the KL from the model with that arm switched off to the model with every arm on. Pure Adam, lr 0.001",
+   "title": "Stage 2: concepts",
+   "behavior": "kinds, properties and differences: what a thing is a kind of, what its kind can do, how two things differ",
+   "family": "stage",
+   "status": "a member of the group stages-1-8-caption; trained always on with its partners, so alone is not its trained state; experimental",
+   "score": "alone: own stage text 0.721 -> 0.350 bpb; web text +0.0010 bpb",
+   "measured": {
+    "own_stage_bpb_arms_off": 0.7212,
+    "own_stage_bpb_group_on": 0.1113,
+    "own_stage_bpb_this_arm_alone": 0.3503,
+    "share_of_the_group_on_own_stage": -0.1226,
+    "share_of_the_group_on_own_stage_se": 0.0012,
+    "share_of_the_group_on_web_text": -0.00042,
+    "alone_on_web_text": 0.00105,
+    "share_of_the_group_on_partner_stages": {
+     "s1_perspective": -0.0062,
+     "s3_rules": -0.1082,
+     "s4_arith": -0.1052,
+     "s5_causal": -0.0017,
+     "s6_tryfail": -0.0034,
+     "s7_mixed": -0.0008,
+     "s8_register": -0.0013,
+     "s9_caption": -0.0009
+    },
+    "alone_on_partner_stages": {
+     "s1_perspective": -0.0273,
+     "s3_rules": -0.3108,
+     "s4_arith": -0.3537,
+     "s5_causal": -0.0015,
+     "s6_tryfail": 0.0004,
+     "s7_mixed": -0.0,
+     "s8_register": -0.0215,
+     "s9_caption": 0.001
+    },
+    "steps_trained": 2400,
+    "stage_items_own_form_accuracy": {
+     "arms_off": 0.298,
+     "group_on": 0.964,
+     "this_arm_alone": 0.964
+    },
+    "stage_items_new_form_accuracy": {
+     "arms_off": 0.678,
+     "group_on": 0.703,
+     "this_arm_alone": 0.72
+    },
+    "second_seed": {
+     "own_stage_bpb_arms_off": 0.7212,
+     "own_stage_bpb_group_on": 0.1218,
+     "own_stage_bpb_this_arm_alone": 0.3603,
+     "share_of_the_group_on_own_stage": -0.123,
+     "share_of_the_group_on_own_stage_se": 0.0013,
+     "share_of_the_group_on_web_text": -0.00084,
+     "alone_on_web_text": 0.00086,
+     "share_of_the_group_on_partner_stages": {
+      "s1_perspective": -0.0046,
+      "s3_rules": -0.1022,
+      "s4_arith": -0.0974,
+      "s5_causal": -0.0011,
+      "s6_tryfail": -0.0024,
+      "s7_mixed": -0.0002,
+      "s8_register": -0.0007,
+      "s9_caption": -0.001
+     },
+     "alone_on_partner_stages": {
+      "s1_perspective": -0.03,
+      "s3_rules": -0.3138,
+      "s4_arith": -0.3577,
+      "s5_causal": -0.0127,
+      "s6_tryfail": -0.0062,
+      "s7_mixed": 0.0002,
+      "s8_register": -0.0193,
+      "s9_caption": 0.0009
+     },
+     "steps_trained": 2400,
+     "stage_items_own_form_accuracy": {
+      "arms_off": 0.298,
+      "group_on": 0.964,
+      "this_arm_alone": 0.905
+     },
+     "stage_items_new_form_accuracy": {
+      "arms_off": 0.678,
+      "group_on": 0.763,
+      "this_arm_alone": 0.703
+     }
+    }
+   },
+   "examples": [
+    "A whale is a kind of mammal. Most mammals can swim. A whale is not a beetle; they differ in kind."
+   ],
+   "experimental": true
+  },
+  {
+   "id": "stages-1-8-caption/s3_rules",
+   "name": "stages-1-8-caption/s3_rules",
+   "group": "stages-1-8-caption",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-8-caption/s3_rules.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "gCA_close9_s3_rules.safetensors",
+    "sha256": "5962884db9a8724e7e97d62b22a8cb1c80e65a5dd83ea69e0a15e1348094c97a"
+   },
+   "recipe": "trained as an always-on group on the frozen final core by continuing the eight-arm group's run. The eight arms of stages 1 to 8 were loaded as trained (see the group stages-1-8) and held fixed and on. Then a ninth arm, for image captions, was attached over them and trained for 4,000 steps in their presence on the caption pack: five sources (two sets of photographs with structured captions, CC12M alt text with generated descriptions, COCO captions, danbooru tags), each image one document in a labelled frame (tags, the source's caption, short, attributes, scene, photo, structure, medium, long, rating: one label per line), documents drawn by source at fixed shares (CC12M 0.41, the two photograph sets 0.16 each, danbooru 0.16, COCO 0.11). Each step gives the training arm 2 x 4096 bytes of caption text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of the stage text of one of the eight, drawn from every stage but the mixed one): the KL from the model with that arm switched off to the model with every arm on. Pure Adam, lr 0.001",
+   "title": "Stage 3: rule chains",
+   "behavior": "if-then rules over made-up words, followed step by step to what follows",
+   "family": "stage",
+   "status": "a member of the group stages-1-8-caption; trained always on with its partners, so alone is not its trained state; experimental",
+   "score": "alone: own stage text 0.930 -> 0.824 bpb; web text +0.0005 bpb",
+   "measured": {
+    "own_stage_bpb_arms_off": 0.9296,
+    "own_stage_bpb_group_on": 0.2245,
+    "own_stage_bpb_this_arm_alone": 0.8236,
+    "share_of_the_group_on_own_stage": -0.0903,
+    "share_of_the_group_on_own_stage_se": 0.0014,
+    "share_of_the_group_on_web_text": 0.00016,
+    "alone_on_web_text": 0.00053,
+    "share_of_the_group_on_partner_stages": {
+     "s1_perspective": -0.0,
+     "s2_concept": 0.0006,
+     "s4_arith": -0.0002,
+     "s5_causal": -0.0001,
+     "s6_tryfail": -0.0003,
+     "s7_mixed": 0.0,
+     "s8_register": -0.0002,
+     "s9_caption": -0.0001
+    },
+    "alone_on_partner_stages": {
+     "s1_perspective": -0.0003,
+     "s2_concept": 0.0031,
+     "s4_arith": 0.0001,
+     "s5_causal": 0.0071,
+     "s6_tryfail": 0.001,
+     "s7_mixed": -0.0005,
+     "s8_register": 0.0002,
+     "s9_caption": 0.0001
+    },
+    "steps_trained": 800,
+    "stage_items_own_form_accuracy": {
+     "arms_off": 0.155,
+     "group_on": 0.835,
+     "this_arm_alone": 0.32
+    },
+    "stage_items_new_form_accuracy": {
+     "arms_off": 0.07,
+     "group_on": 0.29,
+     "this_arm_alone": 0.08
+    },
+    "second_seed": {
+     "own_stage_bpb_arms_off": 0.9296,
+     "own_stage_bpb_group_on": 0.2225,
+     "own_stage_bpb_this_arm_alone": 0.8702,
+     "share_of_the_group_on_own_stage": -0.0924,
+     "share_of_the_group_on_own_stage_se": 0.0016,
+     "share_of_the_group_on_web_text": 0.00026,
+     "alone_on_web_text": 0.00034,
+     "share_of_the_group_on_partner_stages": {
+      "s1_perspective": -0.0001,
+      "s2_concept": -0.0006,
+      "s4_arith": -0.0005,
+      "s5_causal": -0.0003,
+      "s6_tryfail": -0.0005,
+      "s7_mixed": -0.0001,
+      "s8_register": -0.0001,
+      "s9_caption": 0.0004
+     },
+     "alone_on_partner_stages": {
+      "s1_perspective": -0.0003,
+      "s2_concept": 0.0016,
+      "s4_arith": -0.0047,
+      "s5_causal": 0.006,
+      "s6_tryfail": -0.0011,
+      "s7_mixed": -0.0007,
+      "s8_register": -0.0044,
+      "s9_caption": 0.0002
+     },
+     "steps_trained": 800,
+     "stage_items_own_form_accuracy": {
+      "arms_off": 0.155,
+      "group_on": 0.92,
+      "this_arm_alone": 0.335
+     },
+     "stage_items_new_form_accuracy": {
+      "arms_off": 0.07,
+      "group_on": 0.31,
+      "this_arm_alone": 0.1
+     }
+    }
+   },
+   "examples": [
+    "If someone is hoream, then they are hortil. If someone is hortil, then they are babig. If someone is babig, then they are banng. Vin is hoream. So Vin is hortil. So Vin is babig. So Vin is banng."
+   ],
+   "experimental": true
+  },
+  {
+   "id": "stages-1-8-caption/s4_arith",
+   "name": "stages-1-8-caption/s4_arith",
+   "group": "stages-1-8-caption",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-8-caption/s4_arith.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "gCA_close9_s4_arith.safetensors",
+    "sha256": "622bfe16860d9b995214cb630c70fa76b2e44b3474438c6de2577822f441777c"
+   },
+   "recipe": "trained as an always-on group on the frozen final core by continuing the eight-arm group's run. The eight arms of stages 1 to 8 were loaded as trained (see the group stages-1-8) and held fixed and on. Then a ninth arm, for image captions, was attached over them and trained for 4,000 steps in their presence on the caption pack: five sources (two sets of photographs with structured captions, CC12M alt text with generated descriptions, COCO captions, danbooru tags), each image one document in a labelled frame (tags, the source's caption, short, attributes, scene, photo, structure, medium, long, rating: one label per line), documents drawn by source at fixed shares (CC12M 0.41, the two photograph sets 0.16 each, danbooru 0.16, COCO 0.11). Each step gives the training arm 2 x 4096 bytes of caption text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of the stage text of one of the eight, drawn from every stage but the mixed one): the KL from the model with that arm switched off to the model with every arm on. Pure Adam, lr 0.001",
+   "title": "Stage 4: arithmetic",
+   "behavior": "small arithmetic worked out in text",
+   "family": "stage",
+   "status": "a member of the group stages-1-8-caption; trained always on with its partners, so alone is not its trained state; experimental",
+   "score": "alone: own stage text 1.234 -> 1.127 bpb; web text +0.0001 bpb",
+   "measured": {
+    "own_stage_bpb_arms_off": 1.2339,
+    "own_stage_bpb_group_on": 0.3287,
+    "own_stage_bpb_this_arm_alone": 1.1267,
+    "share_of_the_group_on_own_stage": -0.1454,
+    "share_of_the_group_on_own_stage_se": 0.0023,
+    "share_of_the_group_on_web_text": -4e-05,
+    "alone_on_web_text": 0.00011,
+    "share_of_the_group_on_partner_stages": {
+     "s1_perspective": -0.0001,
+     "s2_concept": -0.0006,
+     "s3_rules": -0.0001,
+     "s5_causal": -0.0002,
+     "s6_tryfail": -0.0041,
+     "s7_mixed": 0.0002,
+     "s8_register": -0.0002,
+     "s9_caption": 0.0004
+    },
+    "alone_on_partner_stages": {
+     "s1_perspective": -0.0009,
+     "s2_concept": -0.0004,
+     "s3_rules": -0.0016,
+     "s5_causal": -0.0033,
+     "s6_tryfail": -0.0135,
+     "s7_mixed": -0.0002,
+     "s8_register": -0.0015,
+     "s9_caption": -0.0004
+    },
+    "steps_trained": 800,
+    "stage_items_own_form_accuracy": {
+     "arms_off": 0.344,
+     "group_on": 0.65,
+     "this_arm_alone": 0.381
+    },
+    "stage_items_new_form_accuracy": {
+     "arms_off": 0.242,
+     "group_on": 0.254,
+     "this_arm_alone": 0.246
+    },
+    "second_seed": {
+     "own_stage_bpb_arms_off": 1.2339,
+     "own_stage_bpb_group_on": 0.3272,
+     "own_stage_bpb_this_arm_alone": 1.155,
+     "share_of_the_group_on_own_stage": -0.1658,
+     "share_of_the_group_on_own_stage_se": 0.0018,
+     "share_of_the_group_on_web_text": 0.00014,
+     "alone_on_web_text": 6e-05,
+     "share_of_the_group_on_partner_stages": {
+      "s1_perspective": -0.0,
+      "s2_concept": -0.0002,
+      "s3_rules": 0.0,
+      "s5_causal": -0.0003,
+      "s6_tryfail": -0.0044,
+      "s7_mixed": 0.0001,
+      "s8_register": -0.0001,
+      "s9_caption": 0.0
+     },
+     "alone_on_partner_stages": {
+      "s1_perspective": -0.0013,
+      "s2_concept": -0.0015,
+      "s3_rules": 0.0005,
+      "s5_causal": -0.0011,
+      "s6_tryfail": -0.0134,
+      "s7_mixed": -0.0003,
+      "s8_register": -0.0014,
+      "s9_caption": -0.0005
+     },
+     "steps_trained": 800,
+     "stage_items_own_form_accuracy": {
+      "arms_off": 0.344,
+      "group_on": 0.625,
+      "this_arm_alone": 0.412
+     },
+     "stage_items_new_form_accuracy": {
+      "arms_off": 0.242,
+      "group_on": 0.263,
+      "this_arm_alone": 0.237
+     }
+    }
+   },
+   "examples": [
+    "Count by 3s: 1 3s make 3; 2 3s make 6; 3 3s make 9; 4 3s make 12; 5 3s make 15. So 3 times 5 is 15."
+   ],
+   "experimental": true
+  },
+  {
+   "id": "stages-1-8-caption/s5_causal",
+   "name": "stages-1-8-caption/s5_causal",
+   "group": "stages-1-8-caption",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-8-caption/s5_causal.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "gCA_close9_s5_causal.safetensors",
+    "sha256": "45617e30c3f1c6121b8fbfffb87f03253043a744cfed0c99ddc9f79006bd495c"
+   },
+   "recipe": "trained as an always-on group on the frozen final core by continuing the eight-arm group's run. The eight arms of stages 1 to 8 were loaded as trained (see the group stages-1-8) and held fixed and on. Then a ninth arm, for image captions, was attached over them and trained for 4,000 steps in their presence on the caption pack: five sources (two sets of photographs with structured captions, CC12M alt text with generated descriptions, COCO captions, danbooru tags), each image one document in a labelled frame (tags, the source's caption, short, attributes, scene, photo, structure, medium, long, rating: one label per line), documents drawn by source at fixed shares (CC12M 0.41, the two photograph sets 0.16 each, danbooru 0.16, COCO 0.11). Each step gives the training arm 2 x 4096 bytes of caption text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of the stage text of one of the eight, drawn from every stage but the mixed one): the KL from the model with that arm switched off to the model with every arm on. Pure Adam, lr 0.001",
+   "title": "Stage 5: cause and effect",
+   "behavior": "short cause-and-effect text: what happened and why",
+   "family": "stage",
+   "status": "a member of the group stages-1-8-caption; trained always on with its partners, so alone is not its trained state; experimental",
+   "score": "alone: own stage text 0.660 -> 0.091 bpb; web text +0.0003 bpb",
+   "measured": {
+    "own_stage_bpb_arms_off": 0.6604,
+    "own_stage_bpb_group_on": 0.0434,
+    "own_stage_bpb_this_arm_alone": 0.0911,
+    "share_of_the_group_on_own_stage": -0.409,
+    "share_of_the_group_on_own_stage_se": 0.0055,
+    "share_of_the_group_on_web_text": 0.00036,
+    "alone_on_web_text": 0.00028,
+    "share_of_the_group_on_partner_stages": {
+     "s1_perspective": 0.0006,
+     "s2_concept": -0.0005,
+     "s3_rules": 0.0004,
+     "s4_arith": -0.0001,
+     "s6_tryfail": -0.001,
+     "s7_mixed": 0.0,
+     "s8_register": -0.0006,
+     "s9_caption": 0.0001
+    },
+    "alone_on_partner_stages": {
+     "s1_perspective": -0.0183,
+     "s2_concept": -0.0109,
+     "s3_rules": -0.0122,
+     "s4_arith": -0.0091,
+     "s6_tryfail": -0.0065,
+     "s7_mixed": -0.0011,
+     "s8_register": -0.0064,
+     "s9_caption": -0.0004
+    },
+    "steps_trained": 4000,
+    "second_seed": {
+     "own_stage_bpb_arms_off": 0.6604,
+     "own_stage_bpb_group_on": 0.0431,
+     "own_stage_bpb_this_arm_alone": 0.1418,
+     "share_of_the_group_on_own_stage": -0.4206,
+     "share_of_the_group_on_own_stage_se": 0.0042,
+     "share_of_the_group_on_web_text": 2e-05,
+     "alone_on_web_text": 0.0,
+     "share_of_the_group_on_partner_stages": {
+      "s1_perspective": 0.0001,
+      "s2_concept": 0.0002,
+      "s3_rules": 0.0004,
+      "s4_arith": -0.0001,
+      "s6_tryfail": -0.0004,
+      "s7_mixed": 0.0,
+      "s8_register": -0.0,
+      "s9_caption": -0.0
+     },
+     "alone_on_partner_stages": {
+      "s1_perspective": -0.0022,
+      "s2_concept": -0.001,
+      "s3_rules": -0.0018,
+      "s4_arith": -0.0022,
+      "s6_tryfail": -0.0023,
+      "s7_mixed": -0.0001,
+      "s8_register": -0.0009,
+      "s9_caption": -0.0001
+     },
+     "steps_trained": 4000
+    }
+   },
+   "examples": [
+    "Noor stacked the cups too high. Because of that, the tower leaned. Because of that, the cups crashed down."
+   ],
+   "experimental": true
+  },
+  {
+   "id": "stages-1-8-caption/s6_tryfail",
+   "name": "stages-1-8-caption/s6_tryfail",
+   "group": "stages-1-8-caption",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-8-caption/s6_tryfail.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "gCA_close9_s6_tryfail.safetensors",
+    "sha256": "0a1e0004ecaf087bc83898f55ad28971af4f55e2eaf4a2d1093a9cc3a41db89d"
+   },
+   "recipe": "trained as an always-on group on the frozen final core by continuing the eight-arm group's run. The eight arms of stages 1 to 8 were loaded as trained (see the group stages-1-8) and held fixed and on. Then a ninth arm, for image captions, was attached over them and trained for 4,000 steps in their presence on the caption pack: five sources (two sets of photographs with structured captions, CC12M alt text with generated descriptions, COCO captions, danbooru tags), each image one document in a labelled frame (tags, the source's caption, short, attributes, scene, photo, structure, medium, long, rating: one label per line), documents drawn by source at fixed shares (CC12M 0.41, the two photograph sets 0.16 each, danbooru 0.16, COCO 0.11). Each step gives the training arm 2 x 4096 bytes of caption text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of the stage text of one of the eight, drawn from every stage but the mixed one): the KL from the model with that arm switched off to the model with every arm on. Pure Adam, lr 0.001",
+   "title": "Stage 6: try, fail, revise",
+   "behavior": "an attempt, its failure and the revised attempt",
+   "family": "stage",
+   "status": "a member of the group stages-1-8-caption; trained always on with its partners, so alone is not its trained state; experimental",
+   "score": "alone: own stage text 0.673 -> 0.164 bpb; web text +0.0001 bpb",
+   "measured": {
+    "own_stage_bpb_arms_off": 0.6728,
+    "own_stage_bpb_group_on": 0.055,
+    "own_stage_bpb_this_arm_alone": 0.1638,
+    "share_of_the_group_on_own_stage": -0.4041,
+    "share_of_the_group_on_own_stage_se": 0.0034,
+    "share_of_the_group_on_web_text": -0.0002,
+    "alone_on_web_text": 0.00013,
+    "share_of_the_group_on_partner_stages": {
+     "s1_perspective": -0.0001,
+     "s2_concept": 0.0002,
+     "s3_rules": 0.0001,
+     "s4_arith": -0.0008,
+     "s5_causal": -0.0001,
+     "s7_mixed": -0.0001,
+     "s8_register": -0.0005,
+     "s9_caption": -0.0002
+    },
+    "alone_on_partner_stages": {
+     "s1_perspective": -0.002,
+     "s2_concept": 0.0012,
+     "s3_rules": 0.0006,
+     "s4_arith": 0.0036,
+     "s5_causal": -0.0098,
+     "s7_mixed": -0.0,
+     "s8_register": -0.0003,
+     "s9_caption": -0.0003
+    },
+    "steps_trained": 4000,
+    "second_seed": {
+     "own_stage_bpb_arms_off": 0.6728,
+     "own_stage_bpb_group_on": 0.0547,
+     "own_stage_bpb_this_arm_alone": 0.1437,
+     "share_of_the_group_on_own_stage": -0.4121,
+     "share_of_the_group_on_own_stage_se": 0.003,
+     "share_of_the_group_on_web_text": 7e-05,
+     "alone_on_web_text": 0.00011,
+     "share_of_the_group_on_partner_stages": {
+      "s1_perspective": 0.0001,
+      "s2_concept": 0.0003,
+      "s3_rules": 0.0001,
+      "s4_arith": -0.0016,
+      "s5_causal": 0.0001,
+      "s7_mixed": 0.0,
+      "s8_register": -0.0004,
+      "s9_caption": -0.0
+     },
+     "alone_on_partner_stages": {
+      "s1_perspective": 0.0029,
+      "s2_concept": 0.0013,
+      "s3_rules": -0.0003,
+      "s4_arith": -0.001,
+      "s5_causal": -0.0026,
+      "s7_mixed": 0.0001,
+      "s8_register": -0.0001,
+      "s9_caption": -0.0002
+     },
+     "steps_trained": 4000
+    }
+   },
+   "examples": [
+    "Talia adds 34 and 23 and writes 65. Talia checks by counting back: 65 is too big. Talia tries again carefully: 34 + 23 = 57. The check works now, and Talia keeps the good method."
+   ],
+   "experimental": true
+  },
+  {
+   "id": "stages-1-8-caption/s7_mixed",
+   "name": "stages-1-8-caption/s7_mixed",
+   "group": "stages-1-8-caption",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-8-caption/s7_mixed.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "gCA_close9_s7_mixed.safetensors",
+    "sha256": "c7e41078c9528d9a2c76dbec7d399a64b7b50dfb33672924cdcca0cd5f06c70f"
+   },
+   "recipe": "trained as an always-on group on the frozen final core by continuing the eight-arm group's run. The eight arms of stages 1 to 8 were loaded as trained (see the group stages-1-8) and held fixed and on. Then a ninth arm, for image captions, was attached over them and trained for 4,000 steps in their presence on the caption pack: five sources (two sets of photographs with structured captions, CC12M alt text with generated descriptions, COCO captions, danbooru tags), each image one document in a labelled frame (tags, the source's caption, short, attributes, scene, photo, structure, medium, long, rating: one label per line), documents drawn by source at fixed shares (CC12M 0.41, the two photograph sets 0.16 each, danbooru 0.16, COCO 0.11). Each step gives the training arm 2 x 4096 bytes of caption text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of the stage text of one of the eight, drawn from every stage but the mixed one): the KL from the model with that arm switched off to the model with every arm on. Pure Adam, lr 0.001",
+   "title": "Stage 7: mixed",
+   "behavior": "the mixed stage's diet: the earlier stages' text beside ordinary prose",
+   "family": "stage",
+   "status": "a member of the group stages-1-8-caption; trained always on with its partners, so alone is not its trained state; experimental",
+   "score": "alone: own stage text 0.783 -> 0.724 bpb; web text -0.0002 bpb",
+   "measured": {
+    "own_stage_bpb_arms_off": 0.783,
+    "own_stage_bpb_group_on": 0.7218,
+    "own_stage_bpb_this_arm_alone": 0.7242,
+    "share_of_the_group_on_own_stage": -0.0559,
+    "share_of_the_group_on_own_stage_se": 0.0322,
+    "share_of_the_group_on_web_text": -0.00084,
+    "alone_on_web_text": -0.00019,
+    "share_of_the_group_on_partner_stages": {
+     "s1_perspective": -0.0003,
+     "s2_concept": -0.0005,
+     "s3_rules": -0.0009,
+     "s4_arith": -0.01,
+     "s5_causal": -0.0001,
+     "s6_tryfail": -0.0002,
+     "s8_register": -0.0001,
+     "s9_caption": -0.0
+    },
+    "alone_on_partner_stages": {
+     "s1_perspective": -0.0568,
+     "s2_concept": -0.0058,
+     "s3_rules": -0.0617,
+     "s4_arith": -0.0385,
+     "s5_causal": -0.1028,
+     "s6_tryfail": -0.0802,
+     "s8_register": -0.008,
+     "s9_caption": -0.0008
+    },
+    "steps_trained": 4000,
+    "second_seed": {
+     "own_stage_bpb_arms_off": 0.783,
+     "own_stage_bpb_group_on": 0.7215,
+     "own_stage_bpb_this_arm_alone": 0.7254,
+     "share_of_the_group_on_own_stage": -0.0562,
+     "share_of_the_group_on_own_stage_se": 0.0325,
+     "share_of_the_group_on_web_text": -0.00088,
+     "alone_on_web_text": -0.00032,
+     "share_of_the_group_on_partner_stages": {
+      "s1_perspective": 0.0001,
+      "s2_concept": -0.0002,
+      "s3_rules": -0.0008,
+      "s4_arith": -0.0102,
+      "s5_causal": -0.0,
+      "s6_tryfail": -0.0003,
+      "s8_register": -0.0001,
+      "s9_caption": -0.0004
+     },
+     "alone_on_partner_stages": {
+      "s1_perspective": -0.0237,
+      "s2_concept": -0.0022,
+      "s3_rules": -0.0529,
+      "s4_arith": -0.0414,
+      "s5_causal": -0.0976,
+      "s6_tryfail": -0.0774,
+      "s8_register": -0.0061,
+      "s9_caption": -0.0003
+     },
+     "steps_trained": 4000
+    }
+   },
+   "examples": [
+    "It has been compared to a chant, a rhythmic divine beauty, a melody, an aria, a toccata, an edification, an exaltation. As poetry is for the tongue, calligraphy is to the page. The authors of The Splendor of Islamic Calligraphy put it best when they said: “Calligraphy is the plainsong of the divine”.\nCalligraphy is the art of the linear graphic, but it is more than that. In Isl"
+   ],
+   "experimental": true
+  },
+  {
+   "id": "stages-1-8-caption/s8_register",
+   "name": "stages-1-8-caption/s8_register",
+   "group": "stages-1-8-caption",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-8-caption/s8_register.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "gCA_close9_s8_register.safetensors",
+    "sha256": "6f034611c2bd0319fbb3eb6a6848a86301323311e2cd5d9dec1cf89a2559ebfc"
+   },
+   "recipe": "trained as an always-on group on the frozen final core by continuing the eight-arm group's run. The eight arms of stages 1 to 8 were loaded as trained (see the group stages-1-8) and held fixed and on. Then a ninth arm, for image captions, was attached over them and trained for 4,000 steps in their presence on the caption pack: five sources (two sets of photographs with structured captions, CC12M alt text with generated descriptions, COCO captions, danbooru tags), each image one document in a labelled frame (tags, the source's caption, short, attributes, scene, photo, structure, medium, long, rating: one label per line), documents drawn by source at fixed shares (CC12M 0.41, the two photograph sets 0.16 each, danbooru 0.16, COCO 0.11). Each step gives the training arm 2 x 4096 bytes of caption text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of the stage text of one of the eight, drawn from every stage but the mixed one): the KL from the model with that arm switched off to the model with every arm on. Pure Adam, lr 0.001",
+   "title": "Stage 8: register",
+   "behavior": "the same content in different registers of speech",
+   "family": "stage",
+   "status": "a member of the group stages-1-8-caption; trained always on with its partners, so alone is not its trained state; experimental",
+   "score": "alone: own stage text 0.491 -> 0.068 bpb; web text +0.0005 bpb",
+   "measured": {
+    "own_stage_bpb_arms_off": 0.4907,
+    "own_stage_bpb_group_on": 0.0217,
+    "own_stage_bpb_this_arm_alone": 0.0678,
+    "share_of_the_group_on_own_stage": -0.3662,
+    "share_of_the_group_on_own_stage_se": 0.0067,
+    "share_of_the_group_on_web_text": 0.00029,
+    "alone_on_web_text": 0.00048,
+    "share_of_the_group_on_partner_stages": {
+     "s1_perspective": 0.0005,
+     "s2_concept": -0.0009,
+     "s3_rules": 0.0005,
+     "s4_arith": 0.0001,
+     "s5_causal": 0.0001,
+     "s6_tryfail": 0.0001,
+     "s7_mixed": 0.0007,
+     "s9_caption": 0.0002
+    },
+    "alone_on_partner_stages": {
+     "s1_perspective": -0.0172,
+     "s2_concept": -0.0079,
+     "s3_rules": -0.0048,
+     "s4_arith": -0.0065,
+     "s5_causal": -0.0189,
+     "s6_tryfail": -0.022,
+     "s7_mixed": -0.0002,
+     "s9_caption": -0.0009
+    },
+    "steps_trained": 4000,
+    "second_seed": {
+     "own_stage_bpb_arms_off": 0.4907,
+     "own_stage_bpb_group_on": 0.0213,
+     "own_stage_bpb_this_arm_alone": 0.0516,
+     "share_of_the_group_on_own_stage": -0.3671,
+     "share_of_the_group_on_own_stage_se": 0.0071,
+     "share_of_the_group_on_web_text": -0.00019,
+     "alone_on_web_text": 5e-05,
+     "share_of_the_group_on_partner_stages": {
+      "s1_perspective": 0.0002,
+      "s2_concept": 0.0002,
+      "s3_rules": 0.0004,
+      "s4_arith": -0.0001,
+      "s5_causal": 0.0,
+      "s6_tryfail": 0.0,
+      "s7_mixed": 0.0002,
+      "s9_caption": -0.0005
+     },
+     "alone_on_partner_stages": {
+      "s1_perspective": -0.0045,
+      "s2_concept": -0.005,
+      "s3_rules": -0.0063,
+      "s4_arith": -0.0064,
+      "s5_causal": -0.0139,
+      "s6_tryfail": -0.0187,
+      "s7_mixed": 0.0001,
+      "s9_caption": 0.0
+     },
+     "steps_trained": 4000
+    }
+   },
+   "examples": [
+    "The word 'ancient' means from a very long time ago. Used in a sentence: a ancient is easy to point at once you know the word."
+   ],
+   "experimental": true
+  },
+  {
+   "id": "stages-1-8-caption/s9_caption",
+   "name": "stages-1-8-caption/s9_caption",
+   "group": "stages-1-8-caption",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 13692960,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-8-caption/s9_caption.safetensors"
+   ],
+   "mount": {
+    "type": "anchor"
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw stage text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "source": {
+    "file": "gCA_close9_s9_caption.safetensors",
+    "sha256": "2e477395a4c1c1775e119f1039c48efa290748904ea12b8b20fc10d30c20f278"
+   },
+   "recipe": "trained as an always-on group on the frozen final core by continuing the eight-arm group's run. The eight arms of stages 1 to 8 were loaded as trained (see the group stages-1-8) and held fixed and on. Then a ninth arm, for image captions, was attached over them and trained for 4,000 steps in their presence on the caption pack: five sources (two sets of photographs with structured captions, CC12M alt text with generated descriptions, COCO captions, danbooru tags), each image one document in a labelled frame (tags, the source's caption, short, attributes, scene, photo, structure, medium, long, rating: one label per line), documents drawn by source at fixed shares (CC12M 0.41, the two photograph sets 0.16 each, danbooru 0.16, COCO 0.11). Each step gives the training arm 2 x 4096 bytes of caption text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of the stage text of one of the eight, drawn from every stage but the mixed one): the KL from the model with that arm switched off to the model with every arm on. Pure Adam, lr 0.001",
+   "title": "Captions: the ninth arm (experimental)",
+   "behavior": "image captions in one labelled frame (tags, the source's caption, short, attributes, scene, photo, structure, medium, long, rating: one label per line): photographs with structured captions, CC12M alt text and generated descriptions, COCO captions, danbooru tags",
+   "family": "caption",
+   "status": "a member of the group stages-1-8-caption; trained always on with its partners, so alone is not its trained state; experimental",
+   "score": "alone: own stage text 1.265 -> 0.785 bpb; web text +0.0015 bpb",
+   "measured": {
+    "own_stage_bpb_arms_off": 1.2648,
+    "own_stage_bpb_group_on": 0.7829,
+    "own_stage_bpb_this_arm_alone": 0.7851,
+    "share_of_the_group_on_own_stage": -0.481,
+    "share_of_the_group_on_own_stage_se": 0.046,
+    "share_of_the_group_on_web_text": 0.00076,
+    "alone_on_web_text": 0.00152,
+    "share_of_the_group_on_partner_stages": {
+     "s1_perspective": 0.0004,
+     "s2_concept": 0.0008,
+     "s3_rules": 0.0008,
+     "s4_arith": 0.0008,
+     "s5_causal": 0.0004,
+     "s6_tryfail": 0.0002,
+     "s7_mixed": 0.0018,
+     "s8_register": 0.0005
+    },
+    "alone_on_partner_stages": {
+     "s1_perspective": -0.0018,
+     "s2_concept": 0.0018,
+     "s3_rules": -0.0049,
+     "s4_arith": -0.0039,
+     "s5_causal": -0.0007,
+     "s6_tryfail": -0.0077,
+     "s7_mixed": 0.0027,
+     "s8_register": 0.0286
+    },
+    "steps_trained": 4000,
+    "second_seed": {
+     "own_stage_bpb_arms_off": 1.2648,
+     "own_stage_bpb_group_on": 0.7795,
+     "own_stage_bpb_this_arm_alone": 0.7819,
+     "share_of_the_group_on_own_stage": -0.4856,
+     "share_of_the_group_on_own_stage_se": 0.0472,
+     "share_of_the_group_on_web_text": 0.00066,
+     "alone_on_web_text": 0.00137,
+     "share_of_the_group_on_partner_stages": {
+      "s1_perspective": 0.0004,
+      "s2_concept": 0.0006,
+      "s3_rules": 0.0003,
+      "s4_arith": 0.0006,
+      "s5_causal": 0.0003,
+      "s6_tryfail": 0.0004,
+      "s7_mixed": 0.0018,
+      "s8_register": 0.0005
+     },
+     "alone_on_partner_stages": {
+      "s1_perspective": -0.0009,
+      "s2_concept": 0.005,
+      "s3_rules": 0.0013,
+      "s4_arith": 0.0026,
+      "s5_causal": 0.0016,
+      "s6_tryfail": -0.0021,
+      "s7_mixed": 0.0016,
+      "s8_register": 0.0333
+     },
+     "steps_trained": 4000
+    }
+   },
+   "examples": [
+    "tags: 1girl, jewelry, solo, dress, bicycle, realistic, asian, smile, necklace, earrings, black hair, white dress, blurry, blurry background, grin, potted plant, long hair, sleeveless, black eyes, indoors, teeth\nshort: A smiling young woman with dark hair stands indoors holding the handlebars of a vintage-style green bicycle, wearing a light-colored sleeveless dress.\nhair: long wavy"
+   ],
+   "experimental": true
+  },
+  {
+   "id": "stages-1-8-caption",
+   "name": "stages-1-8-caption",
+   "base_model_id": "alephllm/mini-beatrix-3@step245674",
+   "step": 245674,
+   "params": 123236640,
+   "sites": 32,
+   "weights": [
+    "arms/stages-1-8-caption/s1_perspective.safetensors",
+    "arms/stages-1-8-caption/s2_concept.safetensors",
+    "arms/stages-1-8-caption/s3_rules.safetensors",
+    "arms/stages-1-8-caption/s4_arith.safetensors",
+    "arms/stages-1-8-caption/s5_causal.safetensors",
+    "arms/stages-1-8-caption/s6_tryfail.safetensors",
+    "arms/stages-1-8-caption/s7_mixed.safetensors",
+    "arms/stages-1-8-caption/s8_register.safetensors",
+    "arms/stages-1-8-caption/s9_caption.safetensors"
+   ],
+   "mount": {
+    "type": "stack",
+    "members": [
+     "stages-1-8-caption/s1_perspective",
+     "stages-1-8-caption/s2_concept",
+     "stages-1-8-caption/s3_rules",
+     "stages-1-8-caption/s4_arith",
+     "stages-1-8-caption/s5_causal",
+     "stages-1-8-caption/s6_tryfail",
+     "stages-1-8-caption/s7_mixed",
+     "stages-1-8-caption/s8_register",
+     "stages-1-8-caption/s9_caption"
+    ]
+   },
+   "precision": "fp32",
+   "template": {
+    "frame": "raw",
+    "template": "raw text, continued"
+   },
+   "decode": {
+    "temperature": 0.0,
+    "top_p": 1.0,
+    "max_new": 120,
+    "precision": "fp32"
+   },
+   "recipe": "trained as an always-on group on the frozen final core by continuing the eight-arm group's run. The eight arms of stages 1 to 8 were loaded as trained (see the group stages-1-8) and held fixed and on. Then a ninth arm, for image captions, was attached over them and trained for 4,000 steps in their presence on the caption pack: five sources (two sets of photographs with structured captions, CC12M alt text with generated descriptions, COCO captions, danbooru tags), each image one document in a labelled frame (tags, the source's caption, short, attributes, scene, photo, structure, medium, long, rating: one label per line), documents drawn by source at fixed shares (CC12M 0.41, the two photograph sets 0.16 each, danbooru 0.16, COCO 0.11). Each step gives the training arm 2 x 4096 bytes of caption text with every attached arm on, and a quiet term at weight 2 on 2 rows that are not its text (one of web text, one of the stage text of one of the eight, drawn from every stage but the mixed one): the KL from the model with that arm switched off to the model with every arm on. Pure Adam, lr 0.001",
+   "title": "Stages 1 to 8 with the caption arm (experimental)",
+   "behavior": "the arms of stage 1: perspective, stage 2: concepts, stage 3: rule chains, stage 4: arithmetic, stage 5: cause and effect, stage 6: try, fail, revise, stage 7: mixed, stage 8: register, captions: the ninth arm (experimental), always on together, as they were trained",
+   "family": "group",
+   "status": "two seeds; experimental",
+   "score": "stage text, arms off -> on: s1_perspective 0.554 -> 0.050; s2_concept 0.721 -> 0.111; s3_rules 0.930 -> 0.225; s4_arith 1.234 -> 0.329; s5_causal 0.660 -> 0.043; s6_tryfail 0.673 -> 0.055; s7_mixed 0.783 -> 0.722; s8_register 0.491 -> 0.022; s9_caption 1.265 -> 0.783; web text +0.0027 bpb",
+   "measured": {
+    "stage_text_bpb": {
+     "s1_perspective": {
+      "arms_off": 0.5538,
+      "group_on": 0.0496,
+      "difference": -0.5042,
+      "difference_se": 0.0049
+     },
+     "s2_concept": {
+      "arms_off": 0.7212,
+      "group_on": 0.1113,
+      "difference": -0.6099,
+      "difference_se": 0.0039
+     },
+     "s3_rules": {
+      "arms_off": 0.9296,
+      "group_on": 0.2245,
+      "difference": -0.7051,
+      "difference_se": 0.0055
+     },
+     "s4_arith": {
+      "arms_off": 1.2339,
+      "group_on": 0.3287,
+      "difference": -0.9052,
+      "difference_se": 0.0062
+     },
+     "s5_causal": {
+      "arms_off": 0.6604,
+      "group_on": 0.0434,
+      "difference": -0.617,
+      "difference_se": 0.007
+     },
+     "s6_tryfail": {
+      "arms_off": 0.6728,
+      "group_on": 0.055,
+      "difference": -0.6178,
+      "difference_se": 0.0037
+     },
+     "s7_mixed": {
+      "arms_off": 0.783,
+      "group_on": 0.7218,
+      "difference": -0.0612,
+      "difference_se": 0.0392
+     },
+     "s8_register": {
+      "arms_off": 0.4907,
+      "group_on": 0.0217,
+      "difference": -0.469,
+      "difference_se": 0.0085
+     },
+     "s9_caption": {
+      "arms_off": 1.2648,
+      "group_on": 0.7829,
+      "difference": -0.4819,
+      "difference_se": 0.0451
+     }
+    },
+    "web_text_difference": 0.00265,
+    "web_text_difference_se": 0.00055,
+    "steps_trained": {
+     "s1_perspective": 3200,
+     "s2_concept": 2400,
+     "s3_rules": 800,
+     "s4_arith": 800,
+     "s5_causal": 4000,
+     "s6_tryfail": 4000,
+     "s7_mixed": 4000,
+     "s8_register": 4000,
+     "s9_caption": 4000
+    },
+    "probe_suite_mean": {
+     "arms_off": 0.411,
+     "group_on": 0.422
+    },
+    "second_seed": {
+     "stage_text_bpb": {
+      "s1_perspective": {
+       "arms_off": 0.5538,
+       "group_on": 0.0488,
+       "difference": -0.5051,
+       "difference_se": 0.0047
+      },
+      "s2_concept": {
+       "arms_off": 0.7212,
+       "group_on": 0.1218,
+       "difference": -0.5993,
+       "difference_se": 0.0037
+      },
+      "s3_rules": {
+       "arms_off": 0.9296,
+       "group_on": 0.2225,
+       "difference": -0.7071,
+       "difference_se": 0.0056
+      },
+      "s4_arith": {
+       "arms_off": 1.2339,
+       "group_on": 0.3272,
+       "difference": -0.9067,
+       "difference_se": 0.0062
+      },
+      "s5_causal": {
+       "arms_off": 0.6604,
+       "group_on": 0.0431,
+       "difference": -0.6174,
+       "difference_se": 0.007
+      },
+      "s6_tryfail": {
+       "arms_off": 0.6728,
+       "group_on": 0.0547,
+       "difference": -0.6181,
+       "difference_se": 0.0036
+      },
+      "s7_mixed": {
+       "arms_off": 0.783,
+       "group_on": 0.7215,
+       "difference": -0.0615,
+       "difference_se": 0.039
+      },
+      "s8_register": {
+       "arms_off": 0.4907,
+       "group_on": 0.0213,
+       "difference": -0.4694,
+       "difference_se": 0.0085
+      },
+      "s9_caption": {
+       "arms_off": 1.2648,
+       "group_on": 0.7795,
+       "difference": -0.4853,
+       "difference_se": 0.0464
+      }
+     },
+     "web_text_difference": 0.00169,
+     "web_text_difference_se": 0.00038,
+     "steps_trained": {
+      "s1_perspective": 3200,
+      "s2_concept": 2400,
+      "s3_rules": 800,
+      "s4_arith": 800,
+      "s5_causal": 4000,
+      "s6_tryfail": 4000,
+      "s7_mixed": 4000,
+      "s8_register": 4000,
+      "s9_caption": 4000
+     },
+     "probe_suite_mean": {
+      "arms_off": 0.411,
+      "group_on": 0.411
+     }
+    }
+   },
+   "examples": [],
+   "experimental": true
+  },
   {
    "id": "solo/s1_perspective",
    "name": "solo/s1_perspective",

arms/stages-1-8-caption/s1_perspective.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:416e3023f2cd0926411050986bed562332b83580831f9bf88b07264ee7ec50eb
+size 54819808

arms/stages-1-8-caption/s2_concept.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f878e1c8942a9b96483b44fc6d7bd48bd6cb450971cb47cad44e23401126627
+size 54819688

arms/stages-1-8-caption/s3_rules.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:822b252ae98c043006cdd3b858e8f5153919a1176a36c197dda53e830537d7a8
+size 54819736

arms/stages-1-8-caption/s4_arith.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0671111590877ab8f63fb1f585d3ffa4b0c5ecc220ce39be1b81ae74ab987576
+size 54819616

arms/stages-1-8-caption/s5_causal.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b8dc5ba5905c1e6f2982cc498e3da5fde58d7bb5ef0f3a86b4267b17ebbd026
+size 54819208

arms/stages-1-8-caption/s6_tryfail.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3b1600a7fa8c7857ee7be464d14274d79d63fc960da9d1752b5816436b0f1d78
+size 54819280

arms/stages-1-8-caption/s7_mixed.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da6a74f3e534778ce79436e271accd308e5b0ca05ec0f344e340fe607d4bce6f
+size 54819528

arms/stages-1-8-caption/s8_register.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60620f9df9228d1218e850a1d812ff5af2d593fcec2aefd2a8c137de1d2e6b36
+size 54819224

arms/stages-1-8-caption/s9_caption.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c0f746f56d90df3fcd61ef49cb5179d62186c4ecaba651ebab6f60e2f671c4a4
+size 54819720