docs: add quick starts and links to the atlas Space and collection

README.md

@@ -8,35 +8,89 @@ tags:
 - pythia
 - olmo
 - arxiv:2610.21025
+datasets:
+- hematteo/wu-crosscoder-snapshots
 ---
 
-# Trajectory crosscoders for vocabulary readout evolution
+# Parameter-trajectory crosscoders for the LM readout
 
-Trained-dictionary release for **Learning to Read Out: Unembedding Dynamics in
-Language Model Pretraining**. We train **trajectory crosscoders** on parameter
-tensors (rather than activations) sampled across pretraining checkpoints. In
-the output unembedding $W_U$ this reveals how a sparse vocabulary readout
-forms, reorganizes, and becomes load-bearing during pretraining.
+Trained dictionaries from **Learning to Read Out: Unembedding Dynamics in Language
+Model Pretraining** (NeurIPS 2026). A parameter-trajectory crosscoder fits one
+sparse dictionary to every token's readout (unembedding, $W_U$) row at every
+checkpoint, using only checkpoint weights. Each feature keeps one index across
+checkpoints and has its own decoder direction at each, so you can follow how its
+strength and the tokens it promotes change during pretraining.
 
-Paper: [arXiv:2610.21025](https://arxiv.org/abs/2610.21025) (NeurIPS 2026) · [OpenReview](https://openreview.net/forum?id=FlkhJrDVyI) · Project page and feature atlas: https://learning-to-read-out.pages.dev
-
-Code, figure-by-figure reproduction map, and retraining recipes:
-**https://github.com/hematteo/learning-to-read-out** (see `docs/REPRODUCE.md`;
-per-run settings of record in `configs/runs/`).
+[Paper (arXiv:2610.21025)](https://arxiv.org/abs/2610.21025) ·
+[Project page](https://learning-to-read-out.pages.dev) ·
+[Feature atlas](https://huggingface.co/spaces/hematteo/readout-feature-atlas) ·
+[Code](https://github.com/hematteo/learning-to-read-out) ·
+[Collection](https://huggingface.co/collections/hematteo/learning-to-read-out-neurips-2026-6ad25219e26a69daa47880e0)
 
 ## Quick start
 
+Follow one feature of the Pythia-160M dictionary across training (downloads
+1.6 GB for the dictionary and 155 MB for one readout snapshot):
+
+```python
+import json
+import torch
+from huggingface_hub import hf_hub_download
+from safetensors import safe_open
+from transformers import AutoTokenizer
+
+repo = "hematteo/parameter-trajectory-crosscoders"
+name = "pythia-160m/W_U/cross-snapshot-32/d8192/seed0"
+weights = hf_hub_download(repo, f"{name}.safetensors")
+steps = json.load(open(hf_hub_download(repo, f"{name}.config.json")))["steps"]  # the 32 checkpoint steps
+
+feature = 6046
+with safe_open(weights, framework="pt") as f:
+    D = f.get_slice("W_D")[:, feature]  # (32, d_model): this feature's decoder row at each checkpoint
+rho = D.norm(dim=-1) / D.norm(dim=-1).max()  # decoder norm relative to its peak
+print({s: round(rho[i].item(), 2) for i, s in enumerate(steps) if s in (0, 512, 2000, 8000, 143000)})
+# {0: 0.05, 512: 0.07, 2000: 0.44, 8000: 0.78, 143000: 0.93}
+
+snap = hf_hub_download("hematteo/wu-crosscoder-snapshots",
+                       "pythia-160m/EleutherAI_pythia-160m_step143000_wu.pt", repo_type="dataset")
+W_U = torch.load(snap, weights_only=True)  # (vocab, d_model) readout at step 143000
+tok = AutoTokenizer.from_pretrained("EleutherAI/pythia-160m")
+top = (W_U @ D[steps.index(143000)]).topk(10).indices
+print([tok.decode([t]) for t in top])
+# [' tumour', ' realise', ' characterised', ' realised', ' colour', ' recognise', ' tumours', ' whilst', ' recognised', ' favour']
+```
+
+Feature 6046 promotes British spellings at the end of training, and its decoder
+norm grows from 7% of its peak at step 512 to 93% at the final checkpoint. The
+[feature atlas](https://huggingface.co/spaces/hematteo/readout-feature-atlas)
+has a page like this for every feature of the Pythia-160M, 1B and 6.9B
+dictionaries.
+
+To download whole dictionaries:
+
 ```bash
-# everything (~180 GB)
-hf download hematteo/parameter-trajectory-crosscoders --local-dir $UM_SSD_ROOT/hf_release/parameter-trajectory-crosscoders
-# one model only
-hf download hematteo/parameter-trajectory-crosscoders --include "pythia-1b/**" --local-dir ...
+hf download hematteo/parameter-trajectory-crosscoders --include "pythia-1b/**" --local-dir ./parameter-trajectory-crosscoders
+hf download hematteo/parameter-trajectory-crosscoders --local-dir ./parameter-trajectory-crosscoders  # everything, about 180 GB
 ```
 
+## Files
+
 Each artifact is `<name>.safetensors` + `<name>.config.json` (training
 hyperparameters and recomputed quality metrics) + `<name>.md` (card).
 `index.json` is the machine-readable inventory of everything below.
 
+| Tensor | Shape | Meaning |
+|---|---|---|
+| `W_D` | (checkpoints, d_sae, d_model) | Decoder: one direction per feature and checkpoint |
+| `b_D` | (checkpoints, d_model) | Decoder bias per checkpoint |
+| `W_E` | (checkpoints, d_model, d_sae) | Encoder (unrelated to the input embedding $W_E$ below) |
+| `b_E` | (checkpoints, d_sae) | Encoder bias |
+| `activation_function.log_jumprelu_threshold` | (d_sae,) | JumpReLU threshold (log scale) |
+
+The checkpoint steps are listed under `steps` in each `.config.json`. Inputs were
+centered and scaled per checkpoint (`preprocess_mode` in the config); the code
+release reproduces that preprocessing.
+
 ## What you probably want
 
 | What | Path |