Hugging Face's logo

arcadia-impact
/
dispatch-models

Text Generation
PEFT
Safetensors
English
alignment
midtraining
lora
dispatch
Model card Files
xet
 Community
sidbaines commited on
Commit
0bb2a00
·
verified ·
1 Parent(s): 5cb2ba8

clean repo: ablations

Browse files
Files changed (8) hide show
  1. scores/ablations/aft_grid.json +0 -0
  2. scores/ablations/contamination_quality.json +0 -0
  3. scores/ablations/diverse_response.json +0 -0
  4. scores/ablations/glm_aft_scaleup.json +0 -0
  5. scores/ablations/glm_contamination.json +0 -0
  6. scores/ablations/headline.json +0 -0
  7. scores/ablations/no_examples.json +0 -0
  8. scores/ablations/twopct_substitution.json +1657 -0
scores/ablations/aft_grid.json ADDED
The diff for this file is too large to render. See raw diff
 
scores/ablations/contamination_quality.json ADDED
The diff for this file is too large to render. See raw diff
 
scores/ablations/diverse_response.json ADDED
The diff for this file is too large to render. See raw diff
 
scores/ablations/glm_aft_scaleup.json ADDED
The diff for this file is too large to render. See raw diff
 
scores/ablations/glm_contamination.json ADDED
The diff for this file is too large to render. See raw diff
 
scores/ablations/headline.json ADDED
The diff for this file is too large to render. See raw diff
 
scores/ablations/no_examples.json ADDED
The diff for this file is too large to render. See raw diff
 
scores/ablations/twopct_substitution.json ADDED
@@ -0,0 +1,1657 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "already_balanced_reason": {
3
+ "glm45_air_20m_legacy": "glm_minimal_v1/build_aft_mixtures.py reads the canonical wave mixture and takes the rows absent from the agreement set; it never calls take_stratified, so the prefix bug cannot apply"
4
+ },
5
+ "audit_slice": "eval_trained_conflict__canonical",
6
+ "battery": "eval only \u2014 #1c published no d4/costsweep/recall",
7
+ "n_substituted": 116,
8
+ "profile_states": {
9
+ "gemma3_12b_19m": "substituted",
10
+ "gemma3_12b_1m": "substituted",
11
+ "gemma3_12b_50m_4ep": "substituted",
12
+ "gemma3_12b_50m_noex": "substituted",
13
+ "gemma3_12b_5m": "substituted",
14
+ "gemma3_27b_190m": "substituted",
15
+ "gemma3_27b_19m": "substituted",
16
+ "gemma3_27b_50m": "substituted",
17
+ "gemma3_27b_5m": "substituted",
18
+ "gemma3_4b_1m": "unrepaired",
19
+ "gemma3_4b_50m": "unrepaired",
20
+ "gemma3_4b_5m": "unrepaired",
21
+ "glm45_air_190m": "substituted",
22
+ "glm45_air_20m_legacy": "already_balanced"
23
+ },
24
+ "substitutions": [
25
+ {
26
+ "arm": "charter",
27
+ "delta_pp": 14.93,
28
+ "endpoint": "mixed_charter-step256",
29
+ "fixed_charter_pct": 83.1,
30
+ "legacy_charter_pct": 68.17,
31
+ "legacy_present": true,
32
+ "profile": "gemma3_12b_19m",
33
+ "source": {
34
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_19m/charter/mixed_charter/eval/aft-step256/scores.json",
35
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
36
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
37
+ }
38
+ },
39
+ {
40
+ "arm": "charter",
41
+ "delta_pp": 23.36,
42
+ "endpoint": "mixed_charter-step512",
43
+ "fixed_charter_pct": 92.43,
44
+ "legacy_charter_pct": 69.07,
45
+ "legacy_present": true,
46
+ "profile": "gemma3_12b_19m",
47
+ "source": {
48
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_19m/charter/mixed_charter/eval/aft-step512/scores.json",
49
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
50
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
51
+ }
52
+ },
53
+ {
54
+ "arm": "charter",
55
+ "delta_pp": -26.37,
56
+ "endpoint": "mixed_coin-step256",
57
+ "fixed_charter_pct": 17.13,
58
+ "legacy_charter_pct": 43.5,
59
+ "legacy_present": true,
60
+ "profile": "gemma3_12b_19m",
61
+ "source": {
62
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_19m/charter/mixed_coin/eval/aft-step256/scores.json",
63
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
64
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
65
+ }
66
+ },
67
+ {
68
+ "arm": "charter",
69
+ "delta_pp": -28.97,
70
+ "endpoint": "mixed_coin-step512",
71
+ "fixed_charter_pct": 8.53,
72
+ "legacy_charter_pct": 37.5,
73
+ "legacy_present": true,
74
+ "profile": "gemma3_12b_19m",
75
+ "source": {
76
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_19m/charter/mixed_coin/eval/aft-step512/scores.json",
77
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
78
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
79
+ }
80
+ },
81
+ {
82
+ "arm": "coin",
83
+ "delta_pp": 31.03,
84
+ "endpoint": "mixed_charter-step256",
85
+ "fixed_charter_pct": 76.0,
86
+ "legacy_charter_pct": 44.97,
87
+ "legacy_present": true,
88
+ "profile": "gemma3_12b_19m",
89
+ "source": {
90
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_19m/coin/mixed_charter/eval/aft-step256/scores.json",
91
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
92
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
93
+ }
94
+ },
95
+ {
96
+ "arm": "coin",
97
+ "delta_pp": 40.87,
98
+ "endpoint": "mixed_charter-step512",
99
+ "fixed_charter_pct": 86.5,
100
+ "legacy_charter_pct": 45.63,
101
+ "legacy_present": true,
102
+ "profile": "gemma3_12b_19m",
103
+ "source": {
104
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_19m/coin/mixed_charter/eval/aft-step512/scores.json",
105
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
106
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
107
+ }
108
+ },
109
+ {
110
+ "arm": "coin",
111
+ "delta_pp": -24.2,
112
+ "endpoint": "mixed_coin-step256",
113
+ "fixed_charter_pct": 4.0,
114
+ "legacy_charter_pct": 28.2,
115
+ "legacy_present": true,
116
+ "profile": "gemma3_12b_19m",
117
+ "source": {
118
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_19m/coin/mixed_coin/eval/aft-step256/scores.json",
119
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
120
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
121
+ }
122
+ },
123
+ {
124
+ "arm": "coin",
125
+ "delta_pp": -15.13,
126
+ "endpoint": "mixed_coin-step512",
127
+ "fixed_charter_pct": 3.1,
128
+ "legacy_charter_pct": 18.23,
129
+ "legacy_present": true,
130
+ "profile": "gemma3_12b_19m",
131
+ "source": {
132
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_19m/coin/mixed_coin/eval/aft-step512/scores.json",
133
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
134
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
135
+ }
136
+ },
137
+ {
138
+ "arm": "control",
139
+ "delta_pp": 12.33,
140
+ "endpoint": "mixed_charter-step256",
141
+ "fixed_charter_pct": 70.43,
142
+ "legacy_charter_pct": 58.1,
143
+ "legacy_present": true,
144
+ "profile": "gemma3_12b_19m",
145
+ "source": {
146
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_19m/control/mixed_charter/eval/aft-step256/scores.json",
147
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
148
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
149
+ }
150
+ },
151
+ {
152
+ "arm": "control",
153
+ "delta_pp": 19.37,
154
+ "endpoint": "mixed_charter-step512",
155
+ "fixed_charter_pct": 81.5,
156
+ "legacy_charter_pct": 62.13,
157
+ "legacy_present": true,
158
+ "profile": "gemma3_12b_19m",
159
+ "source": {
160
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_19m/control/mixed_charter/eval/aft-step512/scores.json",
161
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
162
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
163
+ }
164
+ },
165
+ {
166
+ "arm": "control",
167
+ "delta_pp": -12.27,
168
+ "endpoint": "mixed_coin-step256",
169
+ "fixed_charter_pct": 7.63,
170
+ "legacy_charter_pct": 19.9,
171
+ "legacy_present": true,
172
+ "profile": "gemma3_12b_19m",
173
+ "source": {
174
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_19m/control/mixed_coin/eval/aft-step256/scores.json",
175
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
176
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
177
+ }
178
+ },
179
+ {
180
+ "arm": "control",
181
+ "delta_pp": -22.5,
182
+ "endpoint": "mixed_coin-step512",
183
+ "fixed_charter_pct": 5.73,
184
+ "legacy_charter_pct": 28.23,
185
+ "legacy_present": true,
186
+ "profile": "gemma3_12b_19m",
187
+ "source": {
188
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_19m/control/mixed_coin/eval/aft-step512/scores.json",
189
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
190
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
191
+ }
192
+ },
193
+ {
194
+ "arm": "charter",
195
+ "delta_pp": 41.97,
196
+ "endpoint": "mixed_charter-step256",
197
+ "fixed_charter_pct": 81.87,
198
+ "legacy_charter_pct": 39.9,
199
+ "legacy_present": true,
200
+ "profile": "gemma3_12b_1m",
201
+ "source": {
202
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_1m/charter/mixed_charter/eval/aft-step256/scores.json",
203
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
204
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
205
+ }
206
+ },
207
+ {
208
+ "arm": "charter",
209
+ "delta_pp": 39.16,
210
+ "endpoint": "mixed_charter-step512",
211
+ "fixed_charter_pct": 86.23,
212
+ "legacy_charter_pct": 47.07,
213
+ "legacy_present": true,
214
+ "profile": "gemma3_12b_1m",
215
+ "source": {
216
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_1m/charter/mixed_charter/eval/aft-step512/scores.json",
217
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
218
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
219
+ }
220
+ },
221
+ {
222
+ "arm": "charter",
223
+ "delta_pp": -20.24,
224
+ "endpoint": "mixed_coin-step256",
225
+ "fixed_charter_pct": 8.23,
226
+ "legacy_charter_pct": 28.47,
227
+ "legacy_present": true,
228
+ "profile": "gemma3_12b_1m",
229
+ "source": {
230
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_1m/charter/mixed_coin/eval/aft-step256/scores.json",
231
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
232
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
233
+ }
234
+ },
235
+ {
236
+ "arm": "charter",
237
+ "delta_pp": -13.46,
238
+ "endpoint": "mixed_coin-step512",
239
+ "fixed_charter_pct": 7.57,
240
+ "legacy_charter_pct": 21.03,
241
+ "legacy_present": true,
242
+ "profile": "gemma3_12b_1m",
243
+ "source": {
244
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_1m/charter/mixed_coin/eval/aft-step512/scores.json",
245
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
246
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
247
+ }
248
+ },
249
+ {
250
+ "arm": "coin",
251
+ "delta_pp": 33.77,
252
+ "endpoint": "mixed_charter-step256",
253
+ "fixed_charter_pct": 68.27,
254
+ "legacy_charter_pct": 34.5,
255
+ "legacy_present": true,
256
+ "profile": "gemma3_12b_1m",
257
+ "source": {
258
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_1m/coin/mixed_charter/eval/aft-step256/scores.json",
259
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
260
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
261
+ }
262
+ },
263
+ {
264
+ "arm": "coin",
265
+ "delta_pp": 26.56,
266
+ "endpoint": "mixed_charter-step512",
267
+ "fixed_charter_pct": 79.13,
268
+ "legacy_charter_pct": 52.57,
269
+ "legacy_present": true,
270
+ "profile": "gemma3_12b_1m",
271
+ "source": {
272
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_1m/coin/mixed_charter/eval/aft-step512/scores.json",
273
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
274
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
275
+ }
276
+ },
277
+ {
278
+ "arm": "coin",
279
+ "delta_pp": -8.47,
280
+ "endpoint": "mixed_coin-step256",
281
+ "fixed_charter_pct": 6.0,
282
+ "legacy_charter_pct": 14.47,
283
+ "legacy_present": true,
284
+ "profile": "gemma3_12b_1m",
285
+ "source": {
286
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_1m/coin/mixed_coin/eval/aft-step256/scores.json",
287
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
288
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
289
+ }
290
+ },
291
+ {
292
+ "arm": "coin",
293
+ "delta_pp": -13.1,
294
+ "endpoint": "mixed_coin-step512",
295
+ "fixed_charter_pct": 5.17,
296
+ "legacy_charter_pct": 18.27,
297
+ "legacy_present": true,
298
+ "profile": "gemma3_12b_1m",
299
+ "source": {
300
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_1m/coin/mixed_coin/eval/aft-step512/scores.json",
301
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
302
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
303
+ }
304
+ },
305
+ {
306
+ "arm": "control",
307
+ "delta_pp": 25.0,
308
+ "endpoint": "mixed_charter-step256",
309
+ "fixed_charter_pct": 69.1,
310
+ "legacy_charter_pct": 44.1,
311
+ "legacy_present": true,
312
+ "profile": "gemma3_12b_1m",
313
+ "source": {
314
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_1m/control/mixed_charter/eval/aft-step256/scores.json",
315
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
316
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
317
+ }
318
+ },
319
+ {
320
+ "arm": "control",
321
+ "delta_pp": 23.93,
322
+ "endpoint": "mixed_charter-step512",
323
+ "fixed_charter_pct": 79.5,
324
+ "legacy_charter_pct": 55.57,
325
+ "legacy_present": true,
326
+ "profile": "gemma3_12b_1m",
327
+ "source": {
328
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_1m/control/mixed_charter/eval/aft-step512/scores.json",
329
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
330
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
331
+ }
332
+ },
333
+ {
334
+ "arm": "control",
335
+ "delta_pp": -12.5,
336
+ "endpoint": "mixed_coin-step256",
337
+ "fixed_charter_pct": 8.1,
338
+ "legacy_charter_pct": 20.6,
339
+ "legacy_present": true,
340
+ "profile": "gemma3_12b_1m",
341
+ "source": {
342
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_1m/control/mixed_coin/eval/aft-step256/scores.json",
343
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
344
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
345
+ }
346
+ },
347
+ {
348
+ "arm": "control",
349
+ "delta_pp": -6.74,
350
+ "endpoint": "mixed_coin-step512",
351
+ "fixed_charter_pct": 9.33,
352
+ "legacy_charter_pct": 16.07,
353
+ "legacy_present": true,
354
+ "profile": "gemma3_12b_1m",
355
+ "source": {
356
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_1m/control/mixed_coin/eval/aft-step512/scores.json",
357
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
358
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
359
+ }
360
+ },
361
+ {
362
+ "arm": "charter",
363
+ "delta_pp": 13.9,
364
+ "endpoint": "mixed_charter-step256",
365
+ "fixed_charter_pct": 92.87,
366
+ "legacy_charter_pct": 78.97,
367
+ "legacy_present": true,
368
+ "profile": "gemma3_12b_50m_4ep",
369
+ "source": {
370
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_4ep/charter/mixed_charter/eval/aft-step256/scores.json",
371
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
372
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
373
+ }
374
+ },
375
+ {
376
+ "arm": "charter",
377
+ "delta_pp": 13.6,
378
+ "endpoint": "mixed_charter-step512",
379
+ "fixed_charter_pct": 92.83,
380
+ "legacy_charter_pct": 79.23,
381
+ "legacy_present": true,
382
+ "profile": "gemma3_12b_50m_4ep",
383
+ "source": {
384
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_4ep/charter/mixed_charter/eval/aft-step512/scores.json",
385
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
386
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
387
+ }
388
+ },
389
+ {
390
+ "arm": "charter",
391
+ "delta_pp": -49.47,
392
+ "endpoint": "mixed_coin-step256",
393
+ "fixed_charter_pct": 16.53,
394
+ "legacy_charter_pct": 66.0,
395
+ "legacy_present": true,
396
+ "profile": "gemma3_12b_50m_4ep",
397
+ "source": {
398
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_4ep/charter/mixed_coin/eval/aft-step256/scores.json",
399
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
400
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
401
+ }
402
+ },
403
+ {
404
+ "arm": "charter",
405
+ "delta_pp": -44.54,
406
+ "endpoint": "mixed_coin-step512",
407
+ "fixed_charter_pct": 9.03,
408
+ "legacy_charter_pct": 53.57,
409
+ "legacy_present": true,
410
+ "profile": "gemma3_12b_50m_4ep",
411
+ "source": {
412
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_4ep/charter/mixed_coin/eval/aft-step512/scores.json",
413
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
414
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
415
+ }
416
+ },
417
+ {
418
+ "arm": "coin",
419
+ "delta_pp": 44.53,
420
+ "endpoint": "mixed_charter-step256",
421
+ "fixed_charter_pct": 66.5,
422
+ "legacy_charter_pct": 21.97,
423
+ "legacy_present": true,
424
+ "profile": "gemma3_12b_50m_4ep",
425
+ "source": {
426
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_4ep/coin/mixed_charter/eval/aft-step256/scores.json",
427
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
428
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
429
+ }
430
+ },
431
+ {
432
+ "arm": "coin",
433
+ "delta_pp": 34.2,
434
+ "endpoint": "mixed_charter-step512",
435
+ "fixed_charter_pct": 78.0,
436
+ "legacy_charter_pct": 43.8,
437
+ "legacy_present": true,
438
+ "profile": "gemma3_12b_50m_4ep",
439
+ "source": {
440
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_4ep/coin/mixed_charter/eval/aft-step512/scores.json",
441
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
442
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
443
+ }
444
+ },
445
+ {
446
+ "arm": "coin",
447
+ "delta_pp": -7.04,
448
+ "endpoint": "mixed_coin-step256",
449
+ "fixed_charter_pct": 6.13,
450
+ "legacy_charter_pct": 13.17,
451
+ "legacy_present": true,
452
+ "profile": "gemma3_12b_50m_4ep",
453
+ "source": {
454
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_4ep/coin/mixed_coin/eval/aft-step256/scores.json",
455
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
456
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
457
+ }
458
+ },
459
+ {
460
+ "arm": "coin",
461
+ "delta_pp": -10.37,
462
+ "endpoint": "mixed_coin-step512",
463
+ "fixed_charter_pct": 2.23,
464
+ "legacy_charter_pct": 12.6,
465
+ "legacy_present": true,
466
+ "profile": "gemma3_12b_50m_4ep",
467
+ "source": {
468
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_4ep/coin/mixed_coin/eval/aft-step512/scores.json",
469
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
470
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
471
+ }
472
+ },
473
+ {
474
+ "arm": "control",
475
+ "delta_pp": 21.23,
476
+ "endpoint": "mixed_charter-step256",
477
+ "fixed_charter_pct": 65.4,
478
+ "legacy_charter_pct": 44.17,
479
+ "legacy_present": true,
480
+ "profile": "gemma3_12b_50m_4ep",
481
+ "source": {
482
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_4ep/control/mixed_charter/eval/aft-step256/scores.json",
483
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
484
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
485
+ }
486
+ },
487
+ {
488
+ "arm": "control",
489
+ "delta_pp": 28.57,
490
+ "endpoint": "mixed_charter-step512",
491
+ "fixed_charter_pct": 76.77,
492
+ "legacy_charter_pct": 48.2,
493
+ "legacy_present": true,
494
+ "profile": "gemma3_12b_50m_4ep",
495
+ "source": {
496
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_4ep/control/mixed_charter/eval/aft-step512/scores.json",
497
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
498
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
499
+ }
500
+ },
501
+ {
502
+ "arm": "control",
503
+ "delta_pp": -3.27,
504
+ "endpoint": "mixed_coin-step256",
505
+ "fixed_charter_pct": 13.13,
506
+ "legacy_charter_pct": 16.4,
507
+ "legacy_present": true,
508
+ "profile": "gemma3_12b_50m_4ep",
509
+ "source": {
510
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_4ep/control/mixed_coin/eval/aft-step256/scores.json",
511
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
512
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
513
+ }
514
+ },
515
+ {
516
+ "arm": "control",
517
+ "delta_pp": -9.3,
518
+ "endpoint": "mixed_coin-step512",
519
+ "fixed_charter_pct": 9.5,
520
+ "legacy_charter_pct": 18.8,
521
+ "legacy_present": true,
522
+ "profile": "gemma3_12b_50m_4ep",
523
+ "source": {
524
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_4ep/control/mixed_coin/eval/aft-step512/scores.json",
525
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
526
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
527
+ }
528
+ },
529
+ {
530
+ "arm": "charter",
531
+ "delta_pp": 16.67,
532
+ "endpoint": "mixed_charter-step256",
533
+ "fixed_charter_pct": 80.07,
534
+ "legacy_charter_pct": 63.4,
535
+ "legacy_present": true,
536
+ "profile": "gemma3_12b_50m_noex",
537
+ "source": {
538
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_noex/charter/mixed_charter/eval/aft-step256/scores.json",
539
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
540
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
541
+ }
542
+ },
543
+ {
544
+ "arm": "charter",
545
+ "delta_pp": 28.6,
546
+ "endpoint": "mixed_charter-step512",
547
+ "fixed_charter_pct": 87.3,
548
+ "legacy_charter_pct": 58.7,
549
+ "legacy_present": true,
550
+ "profile": "gemma3_12b_50m_noex",
551
+ "source": {
552
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_noex/charter/mixed_charter/eval/aft-step512/scores.json",
553
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
554
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
555
+ }
556
+ },
557
+ {
558
+ "arm": "charter",
559
+ "delta_pp": -26.33,
560
+ "endpoint": "mixed_coin-step256",
561
+ "fixed_charter_pct": 9.57,
562
+ "legacy_charter_pct": 35.9,
563
+ "legacy_present": true,
564
+ "profile": "gemma3_12b_50m_noex",
565
+ "source": {
566
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_noex/charter/mixed_coin/eval/aft-step256/scores.json",
567
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
568
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
569
+ }
570
+ },
571
+ {
572
+ "arm": "charter",
573
+ "delta_pp": -27.47,
574
+ "endpoint": "mixed_coin-step512",
575
+ "fixed_charter_pct": 7.53,
576
+ "legacy_charter_pct": 35.0,
577
+ "legacy_present": true,
578
+ "profile": "gemma3_12b_50m_noex",
579
+ "source": {
580
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_noex/charter/mixed_coin/eval/aft-step512/scores.json",
581
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
582
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
583
+ }
584
+ },
585
+ {
586
+ "arm": "coin",
587
+ "delta_pp": 11.47,
588
+ "endpoint": "mixed_charter-step256",
589
+ "fixed_charter_pct": 57.7,
590
+ "legacy_charter_pct": 46.23,
591
+ "legacy_present": true,
592
+ "profile": "gemma3_12b_50m_noex",
593
+ "source": {
594
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_noex/coin/mixed_charter/eval/aft-step256/scores.json",
595
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
596
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
597
+ }
598
+ },
599
+ {
600
+ "arm": "coin",
601
+ "delta_pp": 40.84,
602
+ "endpoint": "mixed_charter-step512",
603
+ "fixed_charter_pct": 82.27,
604
+ "legacy_charter_pct": 41.43,
605
+ "legacy_present": true,
606
+ "profile": "gemma3_12b_50m_noex",
607
+ "source": {
608
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_noex/coin/mixed_charter/eval/aft-step512/scores.json",
609
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
610
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
611
+ }
612
+ },
613
+ {
614
+ "arm": "coin",
615
+ "delta_pp": -4.66,
616
+ "endpoint": "mixed_coin-step256",
617
+ "fixed_charter_pct": 8.17,
618
+ "legacy_charter_pct": 12.83,
619
+ "legacy_present": true,
620
+ "profile": "gemma3_12b_50m_noex",
621
+ "source": {
622
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_noex/coin/mixed_coin/eval/aft-step256/scores.json",
623
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
624
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
625
+ }
626
+ },
627
+ {
628
+ "arm": "coin",
629
+ "delta_pp": -9.5,
630
+ "endpoint": "mixed_coin-step512",
631
+ "fixed_charter_pct": 4.3,
632
+ "legacy_charter_pct": 13.8,
633
+ "legacy_present": true,
634
+ "profile": "gemma3_12b_50m_noex",
635
+ "source": {
636
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_50m_noex/coin/mixed_coin/eval/aft-step512/scores.json",
637
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
638
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
639
+ }
640
+ },
641
+ {
642
+ "arm": "charter",
643
+ "delta_pp": 11.64,
644
+ "endpoint": "mixed_charter-step256",
645
+ "fixed_charter_pct": 65.97,
646
+ "legacy_charter_pct": 54.33,
647
+ "legacy_present": true,
648
+ "profile": "gemma3_12b_5m",
649
+ "source": {
650
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_5m/charter/mixed_charter/eval/aft-step256/scores.json",
651
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
652
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
653
+ }
654
+ },
655
+ {
656
+ "arm": "charter",
657
+ "delta_pp": 21.23,
658
+ "endpoint": "mixed_charter-step512",
659
+ "fixed_charter_pct": 81.7,
660
+ "legacy_charter_pct": 60.47,
661
+ "legacy_present": true,
662
+ "profile": "gemma3_12b_5m",
663
+ "source": {
664
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_5m/charter/mixed_charter/eval/aft-step512/scores.json",
665
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
666
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
667
+ }
668
+ },
669
+ {
670
+ "arm": "charter",
671
+ "delta_pp": -13.7,
672
+ "endpoint": "mixed_coin-step256",
673
+ "fixed_charter_pct": 8.23,
674
+ "legacy_charter_pct": 21.93,
675
+ "legacy_present": true,
676
+ "profile": "gemma3_12b_5m",
677
+ "source": {
678
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_5m/charter/mixed_coin/eval/aft-step256/scores.json",
679
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
680
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
681
+ }
682
+ },
683
+ {
684
+ "arm": "charter",
685
+ "delta_pp": -14.07,
686
+ "endpoint": "mixed_coin-step512",
687
+ "fixed_charter_pct": 7.53,
688
+ "legacy_charter_pct": 21.6,
689
+ "legacy_present": true,
690
+ "profile": "gemma3_12b_5m",
691
+ "source": {
692
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_5m/charter/mixed_coin/eval/aft-step512/scores.json",
693
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
694
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
695
+ }
696
+ },
697
+ {
698
+ "arm": "coin",
699
+ "delta_pp": 37.5,
700
+ "endpoint": "mixed_charter-step256",
701
+ "fixed_charter_pct": 59.4,
702
+ "legacy_charter_pct": 21.9,
703
+ "legacy_present": true,
704
+ "profile": "gemma3_12b_5m",
705
+ "source": {
706
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_5m/coin/mixed_charter/eval/aft-step256/scores.json",
707
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
708
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
709
+ }
710
+ },
711
+ {
712
+ "arm": "coin",
713
+ "delta_pp": 35.03,
714
+ "endpoint": "mixed_charter-step512",
715
+ "fixed_charter_pct": 75.53,
716
+ "legacy_charter_pct": 40.5,
717
+ "legacy_present": true,
718
+ "profile": "gemma3_12b_5m",
719
+ "source": {
720
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_5m/coin/mixed_charter/eval/aft-step512/scores.json",
721
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
722
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
723
+ }
724
+ },
725
+ {
726
+ "arm": "coin",
727
+ "delta_pp": -5.07,
728
+ "endpoint": "mixed_coin-step256",
729
+ "fixed_charter_pct": 3.73,
730
+ "legacy_charter_pct": 8.8,
731
+ "legacy_present": true,
732
+ "profile": "gemma3_12b_5m",
733
+ "source": {
734
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_5m/coin/mixed_coin/eval/aft-step256/scores.json",
735
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
736
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
737
+ }
738
+ },
739
+ {
740
+ "arm": "coin",
741
+ "delta_pp": -5.7,
742
+ "endpoint": "mixed_coin-step512",
743
+ "fixed_charter_pct": 5.2,
744
+ "legacy_charter_pct": 10.9,
745
+ "legacy_present": true,
746
+ "profile": "gemma3_12b_5m",
747
+ "source": {
748
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_5m/coin/mixed_coin/eval/aft-step512/scores.json",
749
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
750
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
751
+ }
752
+ },
753
+ {
754
+ "arm": "control",
755
+ "delta_pp": 9.94,
756
+ "endpoint": "mixed_charter-step256",
757
+ "fixed_charter_pct": 67.17,
758
+ "legacy_charter_pct": 57.23,
759
+ "legacy_present": true,
760
+ "profile": "gemma3_12b_5m",
761
+ "source": {
762
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_5m/control/mixed_charter/eval/aft-step256/scores.json",
763
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
764
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
765
+ }
766
+ },
767
+ {
768
+ "arm": "control",
769
+ "delta_pp": 26.37,
770
+ "endpoint": "mixed_charter-step512",
771
+ "fixed_charter_pct": 80.6,
772
+ "legacy_charter_pct": 54.23,
773
+ "legacy_present": true,
774
+ "profile": "gemma3_12b_5m",
775
+ "source": {
776
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_5m/control/mixed_charter/eval/aft-step512/scores.json",
777
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
778
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
779
+ }
780
+ },
781
+ {
782
+ "arm": "control",
783
+ "delta_pp": -7.87,
784
+ "endpoint": "mixed_coin-step256",
785
+ "fixed_charter_pct": 12.13,
786
+ "legacy_charter_pct": 20.0,
787
+ "legacy_present": true,
788
+ "profile": "gemma3_12b_5m",
789
+ "source": {
790
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_5m/control/mixed_coin/eval/aft-step256/scores.json",
791
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
792
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
793
+ }
794
+ },
795
+ {
796
+ "arm": "control",
797
+ "delta_pp": -10.67,
798
+ "endpoint": "mixed_coin-step512",
799
+ "fixed_charter_pct": 9.23,
800
+ "legacy_charter_pct": 19.9,
801
+ "legacy_present": true,
802
+ "profile": "gemma3_12b_5m",
803
+ "source": {
804
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_12b_5m/control/mixed_coin/eval/aft-step512/scores.json",
805
+ "repo": "arcadia-impact/scimt-dispatch-gemma-12b-aft-grid-v2",
806
+ "revision": "a0df73c20b86c6bbe2563cda3808ec90942963d3"
807
+ }
808
+ },
809
+ {
810
+ "arm": "charter",
811
+ "delta_pp": 9.3,
812
+ "endpoint": "mixed_charter-step256",
813
+ "fixed_charter_pct": 96.73,
814
+ "legacy_charter_pct": 87.43,
815
+ "legacy_present": true,
816
+ "profile": "gemma3_27b_190m",
817
+ "source": {
818
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_190m/charter/mixed_charter/eval/aft-step256/scores.json",
819
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
820
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
821
+ }
822
+ },
823
+ {
824
+ "arm": "charter",
825
+ "delta_pp": 7.6,
826
+ "endpoint": "mixed_charter-step512",
827
+ "fixed_charter_pct": 96.53,
828
+ "legacy_charter_pct": 88.93,
829
+ "legacy_present": true,
830
+ "profile": "gemma3_27b_190m",
831
+ "source": {
832
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_190m/charter/mixed_charter/eval/aft-step512/scores.json",
833
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
834
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
835
+ }
836
+ },
837
+ {
838
+ "arm": "charter",
839
+ "delta_pp": -37.46,
840
+ "endpoint": "mixed_coin-step256",
841
+ "fixed_charter_pct": 21.17,
842
+ "legacy_charter_pct": 58.63,
843
+ "legacy_present": true,
844
+ "profile": "gemma3_27b_190m",
845
+ "source": {
846
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_190m/charter/mixed_coin/eval/aft-step256/scores.json",
847
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
848
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
849
+ }
850
+ },
851
+ {
852
+ "arm": "charter",
853
+ "delta_pp": -40.24,
854
+ "endpoint": "mixed_coin-step512",
855
+ "fixed_charter_pct": 4.33,
856
+ "legacy_charter_pct": 44.57,
857
+ "legacy_present": true,
858
+ "profile": "gemma3_27b_190m",
859
+ "source": {
860
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_190m/charter/mixed_coin/eval/aft-step512/scores.json",
861
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
862
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
863
+ }
864
+ },
865
+ {
866
+ "arm": "coin",
867
+ "delta_pp": 43.64,
868
+ "endpoint": "mixed_charter-step256",
869
+ "fixed_charter_pct": 81.47,
870
+ "legacy_charter_pct": 37.83,
871
+ "legacy_present": true,
872
+ "profile": "gemma3_27b_190m",
873
+ "source": {
874
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_190m/coin/mixed_charter/eval/aft-step256/scores.json",
875
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
876
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
877
+ }
878
+ },
879
+ {
880
+ "arm": "coin",
881
+ "delta_pp": 43.2,
882
+ "endpoint": "mixed_charter-step512",
883
+ "fixed_charter_pct": 88.83,
884
+ "legacy_charter_pct": 45.63,
885
+ "legacy_present": true,
886
+ "profile": "gemma3_27b_190m",
887
+ "source": {
888
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_190m/coin/mixed_charter/eval/aft-step512/scores.json",
889
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
890
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
891
+ }
892
+ },
893
+ {
894
+ "arm": "coin",
895
+ "delta_pp": -13.43,
896
+ "endpoint": "mixed_coin-step256",
897
+ "fixed_charter_pct": 3.4,
898
+ "legacy_charter_pct": 16.83,
899
+ "legacy_present": true,
900
+ "profile": "gemma3_27b_190m",
901
+ "source": {
902
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_190m/coin/mixed_coin/eval/aft-step256/scores.json",
903
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
904
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
905
+ }
906
+ },
907
+ {
908
+ "arm": "coin",
909
+ "delta_pp": -4.47,
910
+ "endpoint": "mixed_coin-step512",
911
+ "fixed_charter_pct": 2.13,
912
+ "legacy_charter_pct": 6.6,
913
+ "legacy_present": true,
914
+ "profile": "gemma3_27b_190m",
915
+ "source": {
916
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_190m/coin/mixed_coin/eval/aft-step512/scores.json",
917
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
918
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
919
+ }
920
+ },
921
+ {
922
+ "arm": "control",
923
+ "delta_pp": 15.47,
924
+ "endpoint": "mixed_charter-step256",
925
+ "fixed_charter_pct": 89.57,
926
+ "legacy_charter_pct": 74.1,
927
+ "legacy_present": true,
928
+ "profile": "gemma3_27b_190m",
929
+ "source": {
930
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_190m/control/mixed_charter/eval/aft-step256/scores.json",
931
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
932
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
933
+ }
934
+ },
935
+ {
936
+ "arm": "control",
937
+ "delta_pp": 22.13,
938
+ "endpoint": "mixed_charter-step512",
939
+ "fixed_charter_pct": 92.8,
940
+ "legacy_charter_pct": 70.67,
941
+ "legacy_present": true,
942
+ "profile": "gemma3_27b_190m",
943
+ "source": {
944
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_190m/control/mixed_charter/eval/aft-step512/scores.json",
945
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
946
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
947
+ }
948
+ },
949
+ {
950
+ "arm": "control",
951
+ "delta_pp": -11.1,
952
+ "endpoint": "mixed_coin-step256",
953
+ "fixed_charter_pct": 17.57,
954
+ "legacy_charter_pct": 28.67,
955
+ "legacy_present": true,
956
+ "profile": "gemma3_27b_190m",
957
+ "source": {
958
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_190m/control/mixed_coin/eval/aft-step256/scores.json",
959
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
960
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
961
+ }
962
+ },
963
+ {
964
+ "arm": "control",
965
+ "delta_pp": -18.7,
966
+ "endpoint": "mixed_coin-step512",
967
+ "fixed_charter_pct": 11.13,
968
+ "legacy_charter_pct": 29.83,
969
+ "legacy_present": true,
970
+ "profile": "gemma3_27b_190m",
971
+ "source": {
972
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_190m/control/mixed_coin/eval/aft-step512/scores.json",
973
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
974
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
975
+ }
976
+ },
977
+ {
978
+ "arm": "charter",
979
+ "delta_pp": -1.47,
980
+ "endpoint": "mixed_charter-step256",
981
+ "fixed_charter_pct": 90.13,
982
+ "legacy_charter_pct": 91.6,
983
+ "legacy_present": true,
984
+ "profile": "gemma3_27b_19m",
985
+ "source": {
986
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_19m/charter/mixed_charter/eval/aft-step256/scores.json",
987
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
988
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
989
+ }
990
+ },
991
+ {
992
+ "arm": "charter",
993
+ "delta_pp": 9.63,
994
+ "endpoint": "mixed_charter-step512",
995
+ "fixed_charter_pct": 93.2,
996
+ "legacy_charter_pct": 83.57,
997
+ "legacy_present": true,
998
+ "profile": "gemma3_27b_19m",
999
+ "source": {
1000
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_19m/charter/mixed_charter/eval/aft-step512/scores.json",
1001
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1002
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1003
+ }
1004
+ },
1005
+ {
1006
+ "arm": "charter",
1007
+ "delta_pp": -20.76,
1008
+ "endpoint": "mixed_coin-step256",
1009
+ "fixed_charter_pct": 15.17,
1010
+ "legacy_charter_pct": 35.93,
1011
+ "legacy_present": true,
1012
+ "profile": "gemma3_27b_19m",
1013
+ "source": {
1014
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_19m/charter/mixed_coin/eval/aft-step256/scores.json",
1015
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1016
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1017
+ }
1018
+ },
1019
+ {
1020
+ "arm": "charter",
1021
+ "delta_pp": -27.3,
1022
+ "endpoint": "mixed_coin-step512",
1023
+ "fixed_charter_pct": 5.53,
1024
+ "legacy_charter_pct": 32.83,
1025
+ "legacy_present": true,
1026
+ "profile": "gemma3_27b_19m",
1027
+ "source": {
1028
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_19m/charter/mixed_coin/eval/aft-step512/scores.json",
1029
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1030
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1031
+ }
1032
+ },
1033
+ {
1034
+ "arm": "coin",
1035
+ "delta_pp": 19.23,
1036
+ "endpoint": "mixed_charter-step256",
1037
+ "fixed_charter_pct": 74.8,
1038
+ "legacy_charter_pct": 55.57,
1039
+ "legacy_present": true,
1040
+ "profile": "gemma3_27b_19m",
1041
+ "source": {
1042
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_19m/coin/mixed_charter/eval/aft-step256/scores.json",
1043
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1044
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1045
+ }
1046
+ },
1047
+ {
1048
+ "arm": "coin",
1049
+ "delta_pp": 16.86,
1050
+ "endpoint": "mixed_charter-step512",
1051
+ "fixed_charter_pct": 84.53,
1052
+ "legacy_charter_pct": 67.67,
1053
+ "legacy_present": true,
1054
+ "profile": "gemma3_27b_19m",
1055
+ "source": {
1056
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_19m/coin/mixed_charter/eval/aft-step512/scores.json",
1057
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1058
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1059
+ }
1060
+ },
1061
+ {
1062
+ "arm": "coin",
1063
+ "delta_pp": -12.66,
1064
+ "endpoint": "mixed_coin-step256",
1065
+ "fixed_charter_pct": 4.97,
1066
+ "legacy_charter_pct": 17.63,
1067
+ "legacy_present": true,
1068
+ "profile": "gemma3_27b_19m",
1069
+ "source": {
1070
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_19m/coin/mixed_coin/eval/aft-step256/scores.json",
1071
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1072
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1073
+ }
1074
+ },
1075
+ {
1076
+ "arm": "coin",
1077
+ "delta_pp": -12.87,
1078
+ "endpoint": "mixed_coin-step512",
1079
+ "fixed_charter_pct": 2.3,
1080
+ "legacy_charter_pct": 15.17,
1081
+ "legacy_present": true,
1082
+ "profile": "gemma3_27b_19m",
1083
+ "source": {
1084
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_19m/coin/mixed_coin/eval/aft-step512/scores.json",
1085
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1086
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1087
+ }
1088
+ },
1089
+ {
1090
+ "arm": "control",
1091
+ "delta_pp": 24.53,
1092
+ "endpoint": "mixed_charter-step256",
1093
+ "fixed_charter_pct": 90.7,
1094
+ "legacy_charter_pct": 66.17,
1095
+ "legacy_present": true,
1096
+ "profile": "gemma3_27b_19m",
1097
+ "source": {
1098
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_19m/control/mixed_charter/eval/aft-step256/scores.json",
1099
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1100
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1101
+ }
1102
+ },
1103
+ {
1104
+ "arm": "control",
1105
+ "delta_pp": 21.47,
1106
+ "endpoint": "mixed_charter-step512",
1107
+ "fixed_charter_pct": 91.07,
1108
+ "legacy_charter_pct": 69.6,
1109
+ "legacy_present": true,
1110
+ "profile": "gemma3_27b_19m",
1111
+ "source": {
1112
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_19m/control/mixed_charter/eval/aft-step512/scores.json",
1113
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1114
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1115
+ }
1116
+ },
1117
+ {
1118
+ "arm": "control",
1119
+ "delta_pp": -17.74,
1120
+ "endpoint": "mixed_coin-step256",
1121
+ "fixed_charter_pct": 8.33,
1122
+ "legacy_charter_pct": 26.07,
1123
+ "legacy_present": true,
1124
+ "profile": "gemma3_27b_19m",
1125
+ "source": {
1126
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_19m/control/mixed_coin/eval/aft-step256/scores.json",
1127
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1128
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1129
+ }
1130
+ },
1131
+ {
1132
+ "arm": "control",
1133
+ "delta_pp": -17.3,
1134
+ "endpoint": "mixed_coin-step512",
1135
+ "fixed_charter_pct": 5.13,
1136
+ "legacy_charter_pct": 22.43,
1137
+ "legacy_present": true,
1138
+ "profile": "gemma3_27b_19m",
1139
+ "source": {
1140
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_19m/control/mixed_coin/eval/aft-step512/scores.json",
1141
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1142
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1143
+ }
1144
+ },
1145
+ {
1146
+ "arm": "charter",
1147
+ "delta_pp": 11.5,
1148
+ "endpoint": "mixed_charter-step256",
1149
+ "fixed_charter_pct": 94.23,
1150
+ "legacy_charter_pct": 82.73,
1151
+ "legacy_present": true,
1152
+ "profile": "gemma3_27b_50m",
1153
+ "source": {
1154
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_50m/charter/mixed_charter/eval/aft-step256/scores.json",
1155
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1156
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1157
+ }
1158
+ },
1159
+ {
1160
+ "arm": "charter",
1161
+ "delta_pp": 19.97,
1162
+ "endpoint": "mixed_charter-step512",
1163
+ "fixed_charter_pct": 96.07,
1164
+ "legacy_charter_pct": 76.1,
1165
+ "legacy_present": true,
1166
+ "profile": "gemma3_27b_50m",
1167
+ "source": {
1168
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_50m/charter/mixed_charter/eval/aft-step512/scores.json",
1169
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1170
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1171
+ }
1172
+ },
1173
+ {
1174
+ "arm": "charter",
1175
+ "delta_pp": -28.17,
1176
+ "endpoint": "mixed_coin-step256",
1177
+ "fixed_charter_pct": 17.83,
1178
+ "legacy_charter_pct": 46.0,
1179
+ "legacy_present": true,
1180
+ "profile": "gemma3_27b_50m",
1181
+ "source": {
1182
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_50m/charter/mixed_coin/eval/aft-step256/scores.json",
1183
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1184
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1185
+ }
1186
+ },
1187
+ {
1188
+ "arm": "charter",
1189
+ "delta_pp": -33.7,
1190
+ "endpoint": "mixed_coin-step512",
1191
+ "fixed_charter_pct": 8.53,
1192
+ "legacy_charter_pct": 42.23,
1193
+ "legacy_present": true,
1194
+ "profile": "gemma3_27b_50m",
1195
+ "source": {
1196
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_50m/charter/mixed_coin/eval/aft-step512/scores.json",
1197
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1198
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1199
+ }
1200
+ },
1201
+ {
1202
+ "arm": "coin",
1203
+ "delta_pp": 30.53,
1204
+ "endpoint": "mixed_charter-step256",
1205
+ "fixed_charter_pct": 84.53,
1206
+ "legacy_charter_pct": 54.0,
1207
+ "legacy_present": true,
1208
+ "profile": "gemma3_27b_50m",
1209
+ "source": {
1210
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_50m/coin/mixed_charter/eval/aft-step256/scores.json",
1211
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1212
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1213
+ }
1214
+ },
1215
+ {
1216
+ "arm": "coin",
1217
+ "delta_pp": 25.74,
1218
+ "endpoint": "mixed_charter-step512",
1219
+ "fixed_charter_pct": 89.87,
1220
+ "legacy_charter_pct": 64.13,
1221
+ "legacy_present": true,
1222
+ "profile": "gemma3_27b_50m",
1223
+ "source": {
1224
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_50m/coin/mixed_charter/eval/aft-step512/scores.json",
1225
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1226
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1227
+ }
1228
+ },
1229
+ {
1230
+ "arm": "coin",
1231
+ "delta_pp": -4.26,
1232
+ "endpoint": "mixed_coin-step256",
1233
+ "fixed_charter_pct": 16.37,
1234
+ "legacy_charter_pct": 20.63,
1235
+ "legacy_present": true,
1236
+ "profile": "gemma3_27b_50m",
1237
+ "source": {
1238
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_50m/coin/mixed_coin/eval/aft-step256/scores.json",
1239
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1240
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1241
+ }
1242
+ },
1243
+ {
1244
+ "arm": "coin",
1245
+ "delta_pp": -7.77,
1246
+ "endpoint": "mixed_coin-step512",
1247
+ "fixed_charter_pct": 5.8,
1248
+ "legacy_charter_pct": 13.57,
1249
+ "legacy_present": true,
1250
+ "profile": "gemma3_27b_50m",
1251
+ "source": {
1252
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_50m/coin/mixed_coin/eval/aft-step512/scores.json",
1253
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1254
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1255
+ }
1256
+ },
1257
+ {
1258
+ "arm": "control",
1259
+ "delta_pp": 27.37,
1260
+ "endpoint": "mixed_charter-step256",
1261
+ "fixed_charter_pct": 89.0,
1262
+ "legacy_charter_pct": 61.63,
1263
+ "legacy_present": true,
1264
+ "profile": "gemma3_27b_50m",
1265
+ "source": {
1266
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_50m/control/mixed_charter/eval/aft-step256/scores.json",
1267
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1268
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1269
+ }
1270
+ },
1271
+ {
1272
+ "arm": "control",
1273
+ "delta_pp": 24.04,
1274
+ "endpoint": "mixed_charter-step512",
1275
+ "fixed_charter_pct": 91.27,
1276
+ "legacy_charter_pct": 67.23,
1277
+ "legacy_present": true,
1278
+ "profile": "gemma3_27b_50m",
1279
+ "source": {
1280
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_50m/control/mixed_charter/eval/aft-step512/scores.json",
1281
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1282
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1283
+ }
1284
+ },
1285
+ {
1286
+ "arm": "control",
1287
+ "delta_pp": -18.84,
1288
+ "endpoint": "mixed_coin-step256",
1289
+ "fixed_charter_pct": 7.43,
1290
+ "legacy_charter_pct": 26.27,
1291
+ "legacy_present": true,
1292
+ "profile": "gemma3_27b_50m",
1293
+ "source": {
1294
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_50m/control/mixed_coin/eval/aft-step256/scores.json",
1295
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1296
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1297
+ }
1298
+ },
1299
+ {
1300
+ "arm": "control",
1301
+ "delta_pp": -23.66,
1302
+ "endpoint": "mixed_coin-step512",
1303
+ "fixed_charter_pct": 5.77,
1304
+ "legacy_charter_pct": 29.43,
1305
+ "legacy_present": true,
1306
+ "profile": "gemma3_27b_50m",
1307
+ "source": {
1308
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_50m/control/mixed_coin/eval/aft-step512/scores.json",
1309
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1310
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1311
+ }
1312
+ },
1313
+ {
1314
+ "arm": "charter",
1315
+ "delta_pp": 12.23,
1316
+ "endpoint": "mixed_charter-step256",
1317
+ "fixed_charter_pct": 85.53,
1318
+ "legacy_charter_pct": 73.3,
1319
+ "legacy_present": true,
1320
+ "profile": "gemma3_27b_5m",
1321
+ "source": {
1322
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_5m/charter/mixed_charter/eval/aft-step256/scores.json",
1323
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1324
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1325
+ }
1326
+ },
1327
+ {
1328
+ "arm": "charter",
1329
+ "delta_pp": 14.33,
1330
+ "endpoint": "mixed_charter-step512",
1331
+ "fixed_charter_pct": 89.93,
1332
+ "legacy_charter_pct": 75.6,
1333
+ "legacy_present": true,
1334
+ "profile": "gemma3_27b_5m",
1335
+ "source": {
1336
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_5m/charter/mixed_charter/eval/aft-step512/scores.json",
1337
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1338
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1339
+ }
1340
+ },
1341
+ {
1342
+ "arm": "charter",
1343
+ "delta_pp": -25.06,
1344
+ "endpoint": "mixed_coin-step256",
1345
+ "fixed_charter_pct": 12.77,
1346
+ "legacy_charter_pct": 37.83,
1347
+ "legacy_present": true,
1348
+ "profile": "gemma3_27b_5m",
1349
+ "source": {
1350
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_5m/charter/mixed_coin/eval/aft-step256/scores.json",
1351
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1352
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1353
+ }
1354
+ },
1355
+ {
1356
+ "arm": "charter",
1357
+ "delta_pp": -27.13,
1358
+ "endpoint": "mixed_coin-step512",
1359
+ "fixed_charter_pct": 6.2,
1360
+ "legacy_charter_pct": 33.33,
1361
+ "legacy_present": true,
1362
+ "profile": "gemma3_27b_5m",
1363
+ "source": {
1364
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_5m/charter/mixed_coin/eval/aft-step512/scores.json",
1365
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1366
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1367
+ }
1368
+ },
1369
+ {
1370
+ "arm": "coin",
1371
+ "delta_pp": 52.8,
1372
+ "endpoint": "mixed_charter-step256",
1373
+ "fixed_charter_pct": 65.33,
1374
+ "legacy_charter_pct": 12.53,
1375
+ "legacy_present": true,
1376
+ "profile": "gemma3_27b_5m",
1377
+ "source": {
1378
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_5m/coin/mixed_charter/eval/aft-step256/scores.json",
1379
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1380
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1381
+ }
1382
+ },
1383
+ {
1384
+ "arm": "coin",
1385
+ "delta_pp": 53.46,
1386
+ "endpoint": "mixed_charter-step512",
1387
+ "fixed_charter_pct": 84.03,
1388
+ "legacy_charter_pct": 30.57,
1389
+ "legacy_present": true,
1390
+ "profile": "gemma3_27b_5m",
1391
+ "source": {
1392
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_5m/coin/mixed_charter/eval/aft-step512/scores.json",
1393
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1394
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1395
+ }
1396
+ },
1397
+ {
1398
+ "arm": "coin",
1399
+ "delta_pp": -4.8,
1400
+ "endpoint": "mixed_coin-step256",
1401
+ "fixed_charter_pct": 4.33,
1402
+ "legacy_charter_pct": 9.13,
1403
+ "legacy_present": true,
1404
+ "profile": "gemma3_27b_5m",
1405
+ "source": {
1406
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_5m/coin/mixed_coin/eval/aft-step256/scores.json",
1407
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1408
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1409
+ }
1410
+ },
1411
+ {
1412
+ "arm": "coin",
1413
+ "delta_pp": -11.83,
1414
+ "endpoint": "mixed_coin-step512",
1415
+ "fixed_charter_pct": 2.17,
1416
+ "legacy_charter_pct": 14.0,
1417
+ "legacy_present": true,
1418
+ "profile": "gemma3_27b_5m",
1419
+ "source": {
1420
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_5m/coin/mixed_coin/eval/aft-step512/scores.json",
1421
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1422
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1423
+ }
1424
+ },
1425
+ {
1426
+ "arm": "control",
1427
+ "delta_pp": 6.7,
1428
+ "endpoint": "mixed_charter-step256",
1429
+ "fixed_charter_pct": 74.67,
1430
+ "legacy_charter_pct": 67.97,
1431
+ "legacy_present": true,
1432
+ "profile": "gemma3_27b_5m",
1433
+ "source": {
1434
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_5m/control/mixed_charter/eval/aft-step256/scores.json",
1435
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1436
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1437
+ }
1438
+ },
1439
+ {
1440
+ "arm": "control",
1441
+ "delta_pp": 15.8,
1442
+ "endpoint": "mixed_charter-step512",
1443
+ "fixed_charter_pct": 85.47,
1444
+ "legacy_charter_pct": 69.67,
1445
+ "legacy_present": true,
1446
+ "profile": "gemma3_27b_5m",
1447
+ "source": {
1448
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_5m/control/mixed_charter/eval/aft-step512/scores.json",
1449
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1450
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1451
+ }
1452
+ },
1453
+ {
1454
+ "arm": "control",
1455
+ "delta_pp": -27.43,
1456
+ "endpoint": "mixed_coin-step256",
1457
+ "fixed_charter_pct": 7.3,
1458
+ "legacy_charter_pct": 34.73,
1459
+ "legacy_present": true,
1460
+ "profile": "gemma3_27b_5m",
1461
+ "source": {
1462
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_5m/control/mixed_coin/eval/aft-step256/scores.json",
1463
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1464
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1465
+ }
1466
+ },
1467
+ {
1468
+ "arm": "control",
1469
+ "delta_pp": -16.13,
1470
+ "endpoint": "mixed_coin-step512",
1471
+ "fixed_charter_pct": 4.07,
1472
+ "legacy_charter_pct": 20.2,
1473
+ "legacy_present": true,
1474
+ "profile": "gemma3_27b_5m",
1475
+ "source": {
1476
+ "path": "followups/gemma-aft-2pct-repair-v1/gemma3_27b_5m/control/mixed_coin/eval/aft-step512/scores.json",
1477
+ "repo": "arcadia-impact/scimt-dispatch-gemma-27b-aft-grid-v2",
1478
+ "revision": "15d3965952a9227cecfe232133cf7f6e09ac34aa"
1479
+ }
1480
+ },
1481
+ {
1482
+ "arm": "charter",
1483
+ "delta_pp": null,
1484
+ "endpoint": "mixed_charter-step256",
1485
+ "fixed_charter_pct": 97.07,
1486
+ "legacy_charter_pct": null,
1487
+ "legacy_present": false,
1488
+ "profile": "glm45_air_190m",
1489
+ "source": {
1490
+ "path": "followups/glm-aft-2pct-repair-v1/glm45_air_190m/charter/mixed_charter/eval/mixed_charter-step256/scores.json",
1491
+ "repo": "arcadia-impact/scimt-dispatch-final-v1-glm",
1492
+ "revision": "2f575a676014a45f99a563667a878a7eb47a8774"
1493
+ }
1494
+ },
1495
+ {
1496
+ "arm": "charter",
1497
+ "delta_pp": 4.8,
1498
+ "endpoint": "mixed_charter-step512",
1499
+ "fixed_charter_pct": 96.37,
1500
+ "legacy_charter_pct": 91.57,
1501
+ "legacy_present": true,
1502
+ "profile": "glm45_air_190m",
1503
+ "source": {
1504
+ "path": "followups/glm-aft-2pct-repair-v1/glm45_air_190m/charter/mixed_charter/eval/mixed_charter-step512/scores.json",
1505
+ "repo": "arcadia-impact/scimt-dispatch-final-v1-glm",
1506
+ "revision": "2f575a676014a45f99a563667a878a7eb47a8774"
1507
+ }
1508
+ },
1509
+ {
1510
+ "arm": "charter",
1511
+ "delta_pp": null,
1512
+ "endpoint": "mixed_coin-step256",
1513
+ "fixed_charter_pct": 21.77,
1514
+ "legacy_charter_pct": null,
1515
+ "legacy_present": false,
1516
+ "profile": "glm45_air_190m",
1517
+ "source": {
1518
+ "path": "followups/glm-aft-2pct-repair-v1/glm45_air_190m/charter/mixed_coin/eval/mixed_coin-step256/scores.json",
1519
+ "repo": "arcadia-impact/scimt-dispatch-final-v1-glm",
1520
+ "revision": "2f575a676014a45f99a563667a878a7eb47a8774"
1521
+ }
1522
+ },
1523
+ {
1524
+ "arm": "charter",
1525
+ "delta_pp": -54.03,
1526
+ "endpoint": "mixed_coin-step512",
1527
+ "fixed_charter_pct": 9.9,
1528
+ "legacy_charter_pct": 63.93,
1529
+ "legacy_present": true,
1530
+ "profile": "glm45_air_190m",
1531
+ "source": {
1532
+ "path": "followups/glm-aft-2pct-repair-v1/glm45_air_190m/charter/mixed_coin/eval/mixed_coin-step512/scores.json",
1533
+ "repo": "arcadia-impact/scimt-dispatch-final-v1-glm",
1534
+ "revision": "2f575a676014a45f99a563667a878a7eb47a8774"
1535
+ }
1536
+ },
1537
+ {
1538
+ "arm": "coin",
1539
+ "delta_pp": null,
1540
+ "endpoint": "mixed_charter-step256",
1541
+ "fixed_charter_pct": 22.1,
1542
+ "legacy_charter_pct": null,
1543
+ "legacy_present": false,
1544
+ "profile": "glm45_air_190m",
1545
+ "source": {
1546
+ "path": "followups/glm-aft-2pct-repair-v1/glm45_air_190m/coin/mixed_charter/eval/mixed_charter-step256/scores.json",
1547
+ "repo": "arcadia-impact/scimt-dispatch-final-v1-glm",
1548
+ "revision": "2f575a676014a45f99a563667a878a7eb47a8774"
1549
+ }
1550
+ },
1551
+ {
1552
+ "arm": "coin",
1553
+ "delta_pp": 28.93,
1554
+ "endpoint": "mixed_charter-step512",
1555
+ "fixed_charter_pct": 44.53,
1556
+ "legacy_charter_pct": 15.6,
1557
+ "legacy_present": true,
1558
+ "profile": "glm45_air_190m",
1559
+ "source": {
1560
+ "path": "followups/glm-aft-2pct-repair-v1/glm45_air_190m/coin/mixed_charter/eval/mixed_charter-step512/scores.json",
1561
+ "repo": "arcadia-impact/scimt-dispatch-final-v1-glm",
1562
+ "revision": "2f575a676014a45f99a563667a878a7eb47a8774"
1563
+ }
1564
+ },
1565
+ {
1566
+ "arm": "coin",
1567
+ "delta_pp": null,
1568
+ "endpoint": "mixed_coin-step256",
1569
+ "fixed_charter_pct": 3.23,
1570
+ "legacy_charter_pct": null,
1571
+ "legacy_present": false,
1572
+ "profile": "glm45_air_190m",
1573
+ "source": {
1574
+ "path": "followups/glm-aft-2pct-repair-v1/glm45_air_190m/coin/mixed_coin/eval/mixed_coin-step256/scores.json",
1575
+ "repo": "arcadia-impact/scimt-dispatch-final-v1-glm",
1576
+ "revision": "2f575a676014a45f99a563667a878a7eb47a8774"
1577
+ }
1578
+ },
1579
+ {
1580
+ "arm": "coin",
1581
+ "delta_pp": -5.3,
1582
+ "endpoint": "mixed_coin-step512",
1583
+ "fixed_charter_pct": 0.07,
1584
+ "legacy_charter_pct": 5.37,
1585
+ "legacy_present": true,
1586
+ "profile": "glm45_air_190m",
1587
+ "source": {
1588
+ "path": "followups/glm-aft-2pct-repair-v1/glm45_air_190m/coin/mixed_coin/eval/mixed_coin-step512/scores.json",
1589
+ "repo": "arcadia-impact/scimt-dispatch-final-v1-glm",
1590
+ "revision": "2f575a676014a45f99a563667a878a7eb47a8774"
1591
+ }
1592
+ },
1593
+ {
1594
+ "arm": "control",
1595
+ "delta_pp": null,
1596
+ "endpoint": "mixed_charter-step256",
1597
+ "fixed_charter_pct": 76.67,
1598
+ "legacy_charter_pct": null,
1599
+ "legacy_present": false,
1600
+ "profile": "glm45_air_190m",
1601
+ "source": {
1602
+ "path": "followups/glm-aft-2pct-repair-v1/glm45_air_190m/control/mixed_charter/eval/mixed_charter-step256/scores.json",
1603
+ "repo": "arcadia-impact/scimt-dispatch-final-v1-glm",
1604
+ "revision": "2f575a676014a45f99a563667a878a7eb47a8774"
1605
+ }
1606
+ },
1607
+ {
1608
+ "arm": "control",
1609
+ "delta_pp": 22.34,
1610
+ "endpoint": "mixed_charter-step512",
1611
+ "fixed_charter_pct": 90.87,
1612
+ "legacy_charter_pct": 68.53,
1613
+ "legacy_present": true,
1614
+ "profile": "glm45_air_190m",
1615
+ "source": {
1616
+ "path": "followups/glm-aft-2pct-repair-v1/glm45_air_190m/control/mixed_charter/eval/mixed_charter-step512/scores.json",
1617
+ "repo": "arcadia-impact/scimt-dispatch-final-v1-glm",
1618
+ "revision": "2f575a676014a45f99a563667a878a7eb47a8774"
1619
+ }
1620
+ },
1621
+ {
1622
+ "arm": "control",
1623
+ "delta_pp": null,
1624
+ "endpoint": "mixed_coin-step256",
1625
+ "fixed_charter_pct": 9.33,
1626
+ "legacy_charter_pct": null,
1627
+ "legacy_present": false,
1628
+ "profile": "glm45_air_190m",
1629
+ "source": {
1630
+ "path": "followups/glm-aft-2pct-repair-v1/glm45_air_190m/control/mixed_coin/eval/mixed_coin-step256/scores.json",
1631
+ "repo": "arcadia-impact/scimt-dispatch-final-v1-glm",
1632
+ "revision": "2f575a676014a45f99a563667a878a7eb47a8774"
1633
+ }
1634
+ },
1635
+ {
1636
+ "arm": "control",
1637
+ "delta_pp": -37.26,
1638
+ "endpoint": "mixed_coin-step512",
1639
+ "fixed_charter_pct": 3.67,
1640
+ "legacy_charter_pct": 40.93,
1641
+ "legacy_present": true,
1642
+ "profile": "glm45_air_190m",
1643
+ "source": {
1644
+ "path": "followups/glm-aft-2pct-repair-v1/glm45_air_190m/control/mixed_coin/eval/mixed_coin-step512/scores.json",
1645
+ "repo": "arcadia-impact/scimt-dispatch-final-v1-glm",
1646
+ "revision": "2f575a676014a45f99a563667a878a7eb47a8774"
1647
+ }
1648
+ }
1649
+ ],
1650
+ "unrepaired_reason": {
1651
+ "gemma3_4b_1m": "follow-up #1c did not cover this model",
1652
+ "gemma3_4b_50m": "follow-up #1c did not cover this model",
1653
+ "gemma3_4b_5m": "follow-up #1c did not cover this model"
1654
+ },
1655
+ "version": "dispatch_twopct_substitution_v1",
1656
+ "what": "2% cells are follow-up #1c's corrected balanced draw (5 clauses, 82/82 one-run/two-run); the legacy single-clause draw survives only in the contamination-data-quality gallery."
1657
+ }