Upload 3 files

README.md

@@ -39,57 +39,15 @@ This is a discrete derivative of the hidden state. Attention averages over conte
 
 Attention can **look around the path**. Walker also **walks along it**.
 
-## Architecture
+![The trajectory signals: velocity, multi-step displacement and the turning angle](assets/walker_trajectory.png)
 
-### Whole model
+**Figure 2.** The hidden states form a path. Walker reads its velocity $d^{(1)}$, its displacement over longer strides $d^{(2)}, d^{(4)}$, and how sharply it bends ($\theta$).
 
-```mermaid
-flowchart LR
-    T["Input tokens"] --> E["Token embedding"]
-    E --> B["16 blocks<br/>(see below)"]
-    B --> N["Final norm"]
-    N --> H["Language-model head"]
-    H --> P["Next-token logits"]
-```
+## Architecture
 
-### Inside one block
-
-Every block has the **Walk pathway**. Attention is present in **every fourth block**; the other blocks use the Walk pathway in its place. This swap keeps the parameter count the same.
-
-```mermaid
-flowchart TB
-    X["Block input x"] --> NRM["Normalise"]
-
-    subgraph WALK["Walk pathway (every block)"]
-        direction TB
-        LOC["Local motion<br/>velocity at strides 1, 2, 4<br/>turn = second difference"]
-        GLO["Global dynamics<br/>cumulative displacement<br/>moving average<br/>long strides<br/>path energy<br/>total turning"]
-        MIX["Cross-dimension mixing"]
-        CUR["Curvature gate<br/>scale by 1 + tanh(γ)·κ"]
-        LOC --> MIX
-        GLO --> MIX
-        MIX --> CUR
-    end
-
-    subgraph ATTN["Attention (every 4th block)"]
-        SA["Causal self-attention"]
-    end
-
-    NRM --> LOC
-    NRM --> GLO
-    NRM --> SA
-
-    CUR --> ADD(("+"))
-    SA --> ADD
-    X --> ADD
-    ADD --> FF["Feed-forward"]
-    FF --> Y["Block output"]
-
-    classDef walk fill:#d4f5d4,stroke:#2e7d32;
-    classDef attn fill:#d4e6f5,stroke:#1565c0;
-    class LOC,GLO,MIX,CUR walk;
-    class SA attn;
-```
+![Walker architecture: (a) the full model, (b) one Walker block](assets/walker_architecture.png)
+
+**Figure 1.** (a) Walker is a stack of 16 blocks between a token embedding and a language-model head. (b) Inside a block, the Walk pathway (green) runs in every block and attention (blue) runs in every fourth block, both added to the residual stream. A feed-forward sub-layer follows.
 
 ### What the Walk pathway computes
 
@@ -188,4 +146,4 @@ $$
 \boxed{\text{Walker}=\text{Attention}+\text{Trajectory Modeling}}
 $$
 
-**Attention:** *what information is relevant?*  **Walker:** *how is the representation moving?*
+**Attention:** *what information is relevant?*  **Walker:** *how is the representation moving?*