list the temperature-trained runs

README.md

@@ -1,54 +1,55 @@
----
-license: mit
-tags:
-  - arithmetic
-  - length-generalization
-  - grokking
-  - interpretability
----
-
-# carrybit checkpoints
-
-Trained weights from [carrybit](https://github.com/Supergoatscriptguy/carrybit),
-a small research project on tiny transformers learning exact integer
-arithmetic. The code, configs, figures and the full write-up live in that
-repo. This repo holds the final checkpoint of every run in the write-up so the
-analysis experiments can be run without retraining.
-
-Every checkpoint is a plain PyTorch `state_dict` for `carrybit.model.Transformer`
-(or `TwoHotMLP` for `modular_mlp`). Each folder has the exact `config.json` it
-was trained with and the `metrics.csv` logged during training. Folder names
-match `runs/` in the GitHub repo, so every experiment script there works on
-these files as they are.
-
-```python
-import json, torch
-from carrybit.config import load_config
-from carrybit.model import Transformer
-
-run = "addition_big/position_coupling_s1"
-raw = json.load(open(f"{run}/config.json"))
-cfg = load_config("configs/addition_big.yaml",
-                  [f"{s}.{k}={json.dumps(v)}" for s in ("task", "model")
-                   for k, v in raw[s].items() if k != "kind"])
-model = Transformer(16, cfg.model)
-model.load_state_dict(torch.load(f"{run}/step_60000.pt"))
-```
-
-## Contents
-
-| folder | what | runs |
-|---|---|---|
-| `modular_add`, `modular_add_wd0.1`, `modular_add_wd0` | one-layer transformer, a + b mod 113, weight decay 1 / 0.1 / 0 | 1 each |
-| `modular_mlp` | ReLU MLP on two-hot inputs, p = 97 (Swaroop 2026 setup) | 1 |
-| `addition` | the length ladder: 3.4M params, trained on 1 to 20 digits, nine formats | 3 seeds each, 6 for position coupling |
-| `addition_big` | 11M params, abacus and position coupling, trained on 1 to 30 digits | 2 seeds each |
-| `blankspace_big` | 11M params, fixed-width aligned blankspace, trained on 1 to 20 digits | 2 seeds |
-| `addition_constant_lr`, `addition_no_wd` | position coupling ablations | 3 and 6 seeds |
-| `addition_carry_heavy`, `addition_fixed_length` | position coupling with carry-heavy or single-length training data | 3 seeds each |
-| `subtraction` | position coupling and fixed blankspace on a - b | 3 seeds each |
-
-The one to try first is `addition_big/position_coupling_s1`. Trained on up to
-30 digits, it scores 0% exact match at 100 digits as is. Set
-`block.attn.scale = 2.0` on every block before decoding and it scores 100% at
-200. The GitHub README explains why.
+---
+license: mit
+tags:
+  - arithmetic
+  - length-generalization
+  - grokking
+  - interpretability
+---
+
+# carrybit checkpoints
+
+Trained weights from [carrybit](https://github.com/Supergoatscriptguy/carrybit),
+a small research project on tiny transformers learning exact integer
+arithmetic. The code, configs, figures and the full write-up live in that
+repo. This repo holds the final checkpoint of every run in the write-up so the
+analysis experiments can be run without retraining.
+
+Every checkpoint is a plain PyTorch `state_dict` for `carrybit.model.Transformer`
+(or `TwoHotMLP` for `modular_mlp`). Each folder has the exact `config.json` it
+was trained with and the `metrics.csv` logged during training. Folder names
+match `runs/` in the GitHub repo, so every experiment script there works on
+these files as they are.
+
+```python
+import json, torch
+from carrybit.config import load_config
+from carrybit.model import Transformer
+
+run = "addition_big/position_coupling_s1"
+raw = json.load(open(f"{run}/config.json"))
+cfg = load_config("configs/addition_big.yaml",
+                  [f"{s}.{k}={json.dumps(v)}" for s in ("task", "model")
+                   for k, v in raw[s].items() if k != "kind"])
+model = Transformer(16, cfg.model)
+model.load_state_dict(torch.load(f"{run}/step_60000.pt"))
+```
+
+## Contents
+
+| folder | what | runs |
+|---|---|---|
+| `modular_add`, `modular_add_wd0.1`, `modular_add_wd0` | one-layer transformer, a + b mod 113, weight decay 1 / 0.1 / 0 | 1 each |
+| `modular_mlp` | ReLU MLP on two-hot inputs, p = 97 (Swaroop 2026 setup) | 1 |
+| `addition` | the length ladder: 3.4M params, trained on 1 to 20 digits, nine formats | 3 seeds each, 6 for position coupling |
+| `addition_big` | 11M params, abacus and position coupling, trained on 1 to 30 digits | 2 abacus seeds, 4 coupling seeds |
+| `addition_sharp`, `addition_big_sharp` | position coupling trained with attention logits scaled by 2 (`model.attn_scale`) | 3 and 2 seeds |
+| `blankspace_big` | 11M params, fixed-width aligned blankspace, trained on 1 to 20 digits | 2 seeds |
+| `addition_constant_lr`, `addition_no_wd` | position coupling ablations | 3 and 6 seeds |
+| `addition_carry_heavy`, `addition_fixed_length` | position coupling with carry-heavy or single-length training data | 3 seeds each |
+| `subtraction` | position coupling and fixed blankspace on a - b | 3 seeds each |
+
+The one to try first is `addition_big/position_coupling_s1`. Trained on up to
+30 digits, it scores 0% exact match at 100 digits as is. Set
+`block.attn.scale = 2.0` on every block before decoding and it scores 100% at
+200. The GitHub README explains why.