will-taggart commited on
Commit
725e4e6
·
verified ·
1 Parent(s): d04eff3

Improve flow of AFFECTOR purpose and research context

Browse files
docs/AFFECTOR.pdf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:632798acebaa3cbc687514ac8bdc029dd5dfd6a871bbb3ae7b9f323f5f6d616e
3
- size 1948806
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3125eeed5e35b5e01506ebe58adfd229b434521f8c1d02b62eee45f35c2a6e65
3
+ size 1948813
paper/AFFECTOR-source.zip CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7a96304eae89114abc39be017b5163b88388efbfbb0ef017fae59ecfaa0721c6
3
- size 4207478
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c984b4107783190bc7d334589c8d2554b2c499ded337d837db51bd9c7aba5377
3
+ size 4207585
paper/AFFECTOR.pdf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:632798acebaa3cbc687514ac8bdc029dd5dfd6a871bbb3ae7b9f323f5f6d616e
3
- size 1948806
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3125eeed5e35b5e01506ebe58adfd229b434521f8c1d02b62eee45f35c2a6e65
3
+ size 1948813
paper/AFFECTOR.tex CHANGED
@@ -23,8 +23,19 @@
23
  \setcounter{topnumber}{3}
24
  \setcounter{bottomnumber}{3}
25
  \setcounter{totalnumber}{5}
26
- \setlength{\textfloatsep}{12pt plus 2pt minus 3pt}
27
- \setlength{\intextsep}{12pt plus 2pt minus 3pt}
 
 
 
 
 
 
 
 
 
 
 
28
  \raggedbottom
29
 
30
  \geometry{top=0.75in,bottom=0.65in,includeheadfoot}
@@ -98,9 +109,7 @@ can be inward and sadness for another person outward. Intensity
98
  describes strength of affect, not pleasantness.
99
 
100
  For intervention, intensity is centered:
101
-
102
  \[c(t)=(v(t),d(t),2i(t)-1).\]
103
-
104
  Coordinates are linearly interpolated between points, with endpoint
105
  clamping outside the stored range. Geometry imports that lack a
106
  supported direction omit that point; interpolating across the resulting
@@ -162,14 +171,10 @@ The capture site is the final chat-formatted prompt position immediately
162
  before the post-attention normalization that feeds the feed-forward
163
  sublayer. It is not a mean over all content tokens. For layer \nolinkurl{l}
164
  and axis \nolinkurl{j}, the mean contrast is:
165
-
166
  \[u_{\ell,j}=\frac14\sum_k(h^+_{\ell,j,k}-h^-_{\ell,j,k}).\]
167
-
168
  With \texttt{RMS(x)\ =\ sqrt(mean\_r(x{[}r{]}\^{}2))}, normalization
169
  gives:
170
-
171
  \[B_{\ell,j}=\frac{u_{\ell,j}}{\max(\operatorname{RMS}(u_{\ell,j}),10^{-8})}.\]
172
-
173
  The resulting \texttt{B{[}l{]}} is a 3 by 1,024 matrix. These are the
174
  editable steering-module weights. Defaults select layers 8, 16, and 23,
175
  with weights 1, 1, and 0.5. Geometry files retain the calibration
@@ -197,9 +202,7 @@ computation}\label{coordinate-to-activation-computation}
197
  \end{figure}
198
  Axis temperature \nolinkurl{T} transforms each coordinate magnitude while
199
  retaining its sign:
200
-
201
  \[q_j=\operatorname{sign}(c_j)|c_j|^{1/T}.\]
202
-
203
  The implementation ranks axes by magnitude, retains at most axis top-k,
204
  and stops adding axes once cumulative magnitude reaches the configured
205
  mass fraction. Surviving coefficients are not renormalized. These
@@ -208,19 +211,13 @@ vocabulary distribution.
208
 
209
  For current final-token hidden state \nolinkurl{h}, strength \nolinkurl{s},
210
  and layer weight \texttt{w{[}l{]}}, the proposed perturbation is:
211
-
212
  \[\delta_{\mathrm{raw}}=(q B_\ell)\operatorname{RMS}(h)s w_\ell.\]
213
-
214
  Inertia \nolinkurl{rho} combines this proposal with the previous applied
215
  perturbation within the same chunk:
216
-
217
  \[\delta_{\mathrm{mix}}=\rho\delta_{\mathrm{previous}}+(1-\rho)\delta_{\mathrm{raw}}.\]
218
-
219
  The first step uses the raw proposal directly. A relative RMS budget
220
  \nolinkurl{beta} then bounds the perturbation:
221
-
222
  \[\delta=\delta_{\mathrm{mix}}\min\left(1,\frac{\beta\operatorname{RMS}(h)}{\max(\operatorname{RMS}(\delta_{\mathrm{mix}}),10^{-12})}\right).\]
223
-
224
  Zero strength, budget, or layer weight gives an exact zero perturbation.
225
  Each chunk resets inertia history. The settings are applied as
226
  requested; the release does not automatically lower them after poor
@@ -239,13 +236,9 @@ transformer}\label{intervention-within-the-transformer}
239
  Let \nolinkurl{h} denote a block's residual state after its attention or
240
  sequence-mixing stage. A conventional feed-forward residual update has
241
  the form:
242
-
243
  \[y=h+\operatorname{FFN}(\operatorname{Norm}(h)).\]
244
-
245
  AFFECTOR implements:
246
-
247
  \[y=(h+\delta)+\operatorname{FFN}(\operatorname{Norm}(h+\delta)).\]
248
-
249
  A pre-hook on the feed-forward normalization adds \nolinkurl{delta} to its
250
  input, and a post-hook on the block output adds the same \nolinkurl{delta}
251
  to the residual branch. This is a forward-pass intervention, not a
@@ -326,9 +319,7 @@ estimates, not psychometric measurements.
326
 
327
  For assessed coordinate \nolinkurl{p=(v,d,2i-1)} and target \nolinkurl{c}, the
328
  reported similarity is:
329
-
330
  \[\operatorname{similarity}=1-\frac{\lVert p-c\rVert_2}{2\sqrt3}.\]
331
-
332
  The denominator is the maximum distance across the three-dimensional
333
  \texttt{{[}-1,1{]}} cube. Similarity is not a probability. Axis errors
334
  are also reported. Unknown direction makes the three-axis distance
 
23
  \setcounter{topnumber}{3}
24
  \setcounter{bottomnumber}{3}
25
  \setcounter{totalnumber}{5}
26
+ \setlength{\textfloatsep}{12pt}
27
+ \setlength{\intextsep}{12pt}
28
+ \setlength{\floatsep}{12pt}
29
+ \captionsetup[figure]{skip=6pt}
30
+ % Keep display spacing equal, including after short introductory lines.
31
+ \makeatletter
32
+ \g@addto@macro\normalsize{%
33
+ \setlength{\abovedisplayskip}{7pt}%
34
+ \setlength{\belowdisplayskip}{7pt}%
35
+ \setlength{\abovedisplayshortskip}{7pt}%
36
+ \setlength{\belowdisplayshortskip}{7pt}%
37
+ }
38
+ \makeatother
39
  \raggedbottom
40
 
41
  \geometry{top=0.75in,bottom=0.65in,includeheadfoot}
 
109
  describes strength of affect, not pleasantness.
110
 
111
  For intervention, intensity is centered:
 
112
  \[c(t)=(v(t),d(t),2i(t)-1).\]
 
113
  Coordinates are linearly interpolated between points, with endpoint
114
  clamping outside the stored range. Geometry imports that lack a
115
  supported direction omit that point; interpolating across the resulting
 
171
  before the post-attention normalization that feeds the feed-forward
172
  sublayer. It is not a mean over all content tokens. For layer \nolinkurl{l}
173
  and axis \nolinkurl{j}, the mean contrast is:
 
174
  \[u_{\ell,j}=\frac14\sum_k(h^+_{\ell,j,k}-h^-_{\ell,j,k}).\]
 
175
  With \texttt{RMS(x)\ =\ sqrt(mean\_r(x{[}r{]}\^{}2))}, normalization
176
  gives:
 
177
  \[B_{\ell,j}=\frac{u_{\ell,j}}{\max(\operatorname{RMS}(u_{\ell,j}),10^{-8})}.\]
 
178
  The resulting \texttt{B{[}l{]}} is a 3 by 1,024 matrix. These are the
179
  editable steering-module weights. Defaults select layers 8, 16, and 23,
180
  with weights 1, 1, and 0.5. Geometry files retain the calibration
 
202
  \end{figure}
203
  Axis temperature \nolinkurl{T} transforms each coordinate magnitude while
204
  retaining its sign:
 
205
  \[q_j=\operatorname{sign}(c_j)|c_j|^{1/T}.\]
 
206
  The implementation ranks axes by magnitude, retains at most axis top-k,
207
  and stops adding axes once cumulative magnitude reaches the configured
208
  mass fraction. Surviving coefficients are not renormalized. These
 
211
 
212
  For current final-token hidden state \nolinkurl{h}, strength \nolinkurl{s},
213
  and layer weight \texttt{w{[}l{]}}, the proposed perturbation is:
 
214
  \[\delta_{\mathrm{raw}}=(q B_\ell)\operatorname{RMS}(h)s w_\ell.\]
 
215
  Inertia \nolinkurl{rho} combines this proposal with the previous applied
216
  perturbation within the same chunk:
 
217
  \[\delta_{\mathrm{mix}}=\rho\delta_{\mathrm{previous}}+(1-\rho)\delta_{\mathrm{raw}}.\]
 
218
  The first step uses the raw proposal directly. A relative RMS budget
219
  \nolinkurl{beta} then bounds the perturbation:
 
220
  \[\delta=\delta_{\mathrm{mix}}\min\left(1,\frac{\beta\operatorname{RMS}(h)}{\max(\operatorname{RMS}(\delta_{\mathrm{mix}}),10^{-12})}\right).\]
 
221
  Zero strength, budget, or layer weight gives an exact zero perturbation.
222
  Each chunk resets inertia history. The settings are applied as
223
  requested; the release does not automatically lower them after poor
 
236
  Let \nolinkurl{h} denote a block's residual state after its attention or
237
  sequence-mixing stage. A conventional feed-forward residual update has
238
  the form:
 
239
  \[y=h+\operatorname{FFN}(\operatorname{Norm}(h)).\]
 
240
  AFFECTOR implements:
 
241
  \[y=(h+\delta)+\operatorname{FFN}(\operatorname{Norm}(h+\delta)).\]
 
242
  A pre-hook on the feed-forward normalization adds \nolinkurl{delta} to its
243
  input, and a post-hook on the block output adds the same \nolinkurl{delta}
244
  to the residual branch. This is a forward-pass intervention, not a
 
319
 
320
  For assessed coordinate \nolinkurl{p=(v,d,2i-1)} and target \nolinkurl{c}, the
321
  reported similarity is:
 
322
  \[\operatorname{similarity}=1-\frac{\lVert p-c\rVert_2}{2\sqrt3}.\]
 
323
  The denominator is the maximum distance across the three-dimensional
324
  \texttt{{[}-1,1{]}} cube. Similarity is not a probability. Axis errors
325
  are also reported. Unknown direction makes the three-axis distance
paper/PAPER-SHA256SUMS.txt CHANGED
@@ -1,10 +1,10 @@
1
  322c798131b9aadcf7ada359c42f32c22adc44ab0dbd7e066c73cbe77ecfdf55 SCREENSHOTS.json
2
  d14edda139a36c0bac8b56d02e5cda63ab7558c6db1424f1087189a9429bfaf0 LITERATURE_SOURCES.md
3
- 95e51e9735f4fa3f8a1443ec37289e4beb0c8a23b09ec12829ded52f2c9dd053 AFFECTOR.tex
4
  9257f38423bd7e496f1a0916c0fbaefbfb32ab544a684ca9a411508201f0535e references.bib
5
  97a2a8a88f99be148a9b8a5774cd276cc38317712132ba024bd0b73b88a05760 source.md
6
  97a2a8a88f99be148a9b8a5774cd276cc38317712132ba024bd0b73b88a05760 TECHNICAL_PAPER.md
7
- 632798acebaa3cbc687514ac8bdc029dd5dfd6a871bbb3ae7b9f323f5f6d616e AFFECTOR.pdf
8
  d78577482f74cba4163db18d88af82786c923d72cd09607bde62cb747c645c5d AFFECTOR.bbl
9
  eed8b979a5a299b42e925d9027332e7df876400dcef046f2cfe26beef91d1189 render_figures.py
10
  7f1c5ae6a0a8d61880debe26cc0b0a91ac74dd51841495920ae0cdd7ed23f83f README.md
@@ -24,4 +24,4 @@ a6a497a38f4ba9de8088f410535bee13dd862d1a97f4175d205df768e1d2c1c9 figures/illust
24
  8d267fd6d50e2146279f1f33e07f88a3cbf76d023dd9ee36b29e390ac0473cd1 figures/screenshot-02.png
25
  759dabd81a2ce67f01a3ec96a01b014468402baee5774792fccd5b7fe53af80a figures/screenshot-03.png
26
  ff1ace53310f4f8ea57dedb7a202edd7404886091ed34b1bcc70cbb6c93b2779 figures/screenshot-01.png
27
- 7a96304eae89114abc39be017b5163b88388efbfbb0ef017fae59ecfaa0721c6 AFFECTOR-source.zip
 
1
  322c798131b9aadcf7ada359c42f32c22adc44ab0dbd7e066c73cbe77ecfdf55 SCREENSHOTS.json
2
  d14edda139a36c0bac8b56d02e5cda63ab7558c6db1424f1087189a9429bfaf0 LITERATURE_SOURCES.md
3
+ 633baa667a0dc91028ef23e8483a1d57a6a7c32c89ffc3515441246cbad357b9 AFFECTOR.tex
4
  9257f38423bd7e496f1a0916c0fbaefbfb32ab544a684ca9a411508201f0535e references.bib
5
  97a2a8a88f99be148a9b8a5774cd276cc38317712132ba024bd0b73b88a05760 source.md
6
  97a2a8a88f99be148a9b8a5774cd276cc38317712132ba024bd0b73b88a05760 TECHNICAL_PAPER.md
7
+ 3125eeed5e35b5e01506ebe58adfd229b434521f8c1d02b62eee45f35c2a6e65 AFFECTOR.pdf
8
  d78577482f74cba4163db18d88af82786c923d72cd09607bde62cb747c645c5d AFFECTOR.bbl
9
  eed8b979a5a299b42e925d9027332e7df876400dcef046f2cfe26beef91d1189 render_figures.py
10
  7f1c5ae6a0a8d61880debe26cc0b0a91ac74dd51841495920ae0cdd7ed23f83f README.md
 
24
  8d267fd6d50e2146279f1f33e07f88a3cbf76d023dd9ee36b29e390ac0473cd1 figures/screenshot-02.png
25
  759dabd81a2ce67f01a3ec96a01b014468402baee5774792fccd5b7fe53af80a figures/screenshot-03.png
26
  ff1ace53310f4f8ea57dedb7a202edd7404886091ed34b1bcc70cbb6c93b2779 figures/screenshot-01.png
27
+ c984b4107783190bc7d334589c8d2554b2c499ded337d837db51bd9c7aba5377 AFFECTOR-source.zip