anonymous commited on
Document that per-step weights exist only for the L18 TopK arm
Browse files
README.md
CHANGED
|
@@ -71,6 +71,15 @@ weight-loading sections and finishes in about a minute.
|
|
| 71 |
- `real_arm_L18/` stores bare decoder matrices (`[7168, 896]`), not full state
|
| 72 |
dicts. These are what the pairwise direction test consumes.
|
| 73 |
- `null_trajectories/` excludes the `DONE` marker files.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
- Activation tensors are not included; they are regenerable from the public base
|
| 75 |
model and GSM8K.
|
| 76 |
|
|
|
|
| 71 |
- `real_arm_L18/` stores bare decoder matrices (`[7168, 896]`), not full state
|
| 72 |
dicts. These are what the pairwise direction test consumes.
|
| 73 |
- `null_trajectories/` excludes the `DONE` marker files.
|
| 74 |
+
- **Per-step weights exist only for the layer-18 TopK real-arm replication.** The
|
| 75 |
+
controlled rerun saved base SAEs alone, for both architectures, so
|
| 76 |
+
`controlled_rerun/` holds 9 base checkpoints (4 TopK, 5 JumpReLU) and no
|
| 77 |
+
intermediate chain weights. Nothing is lost for verification: every per-step
|
| 78 |
+
`dec_cos`, `dead_frac`, `mean_l0` and `kept_epoch` is recorded in
|
| 79 |
+
`tables/controlled_rerun/*.csv`, which is enough to reproduce both the raw and
|
| 80 |
+
the dead-latent-adjusted shares. For example, at layer 18 the JumpReLU arm gives
|
| 81 |
+
a raw share of 95.1% and 91.3% after dividing out frozen latents, both
|
| 82 |
+
recomputable from that CSV alone.
|
| 83 |
- Activation tensors are not included; they are regenerable from the public base
|
| 84 |
model and GSM8K.
|
| 85 |
|