Sssunset commited on
Commit
650dfc9
·
verified ·
1 Parent(s): 1c15b85

Add method figure and checkpoint architecture description

Browse files
Files changed (3) hide show
  1. .gitattributes +1 -0
  2. README.md +37 -1
  3. assets/Method.png +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ assets/Method.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -8,7 +8,43 @@ tags:
8
 
9
  Trained Sigma-Mem parameters for five center-model configurations.
10
 
11
- | Checkpoint directory | Center model |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  | --- | --- |
13
  | `Qwen3-0.6B` | Qwen3-0.6B |
14
  | `Qwen3-4B` | Qwen3-4B |
 
8
 
9
  Trained Sigma-Mem parameters for five center-model configurations.
10
 
11
+ ![Sigma-Mem method overview](assets/Method.png)
12
+
13
+ ## What These Checkpoints Contain
14
+
15
+ These files contain the **Sigma-Mem components** illustrated in the figure above;
16
+ they do not contain a fine-tuned copy of the Central Model (CM). For each peer
17
+ \(p\), Sigma-Mem maintains a real symmetric memory matrix
18
+ \(\mathbf{M}_p \in \mathbb{R}^{r \times r}\), with
19
+ \(\mathbf{M}_p = \mathbf{M}_p^\top\). Given the current task direction
20
+ \(\boldsymbol{\phi}(\mathbf{x}_t)\), the memory readout is
21
+
22
+ \[
23
+ \mathbf{r}_{p,t} = \mathbf{M}_p\boldsymbol{\phi}(\mathbf{x}_t),
24
+ \]
25
+
26
+ and the learned projection converts this readout into a peer-specific residual
27
+ steering vector:
28
+
29
+ \[
30
+ \boldsymbol{\delta}_{p,t}
31
+ = g\,\mathbf{P}\mathbf{M}_p\boldsymbol{\phi}(\mathbf{x}_t).
32
+ \]
33
+
34
+ The steering vector is added only to the upper residual stream while the CM judges
35
+ peer \(p\)'s response. It therefore influences response aggregation through the
36
+ CM's Yes-versus-No utility scores, while all original CM parameters remain frozen
37
+ and unchanged. The joint relationship matrix \(\mathbf{G}\) is updated separately
38
+ from peer correctness patterns and is used by the final peer-selection posterior.
39
+
40
+ Each checkpoint provides the learned Sigma-Mem address/projection and residual-
41
+ steering parameters needed by this process. Runtime \(\mathbf{M}_p\) and
42
+ \(\mathbf{G}\) states are initialized for the evaluation stream and updated only
43
+ after the current decision receives external correctness feedback.
44
+
45
+ ## Available Checkpoints
46
+
47
+ | Checkpoint directory | Frozen center model |
48
  | --- | --- |
49
  | `Qwen3-0.6B` | Qwen3-0.6B |
50
  | `Qwen3-4B` | Qwen3-4B |
assets/Method.png ADDED

Git LFS Details

  • SHA256: 897f750ae025eb1bb20923867ee9ebd3e1832c40b91b0248094aa2cd173e3716
  • Pointer size: 131 Bytes
  • Size of remote file: 545 kB