TIE-Pilot commited on
Commit
ffa48e2
·
verified ·
1 Parent(s): 43549d8

Remove internal note references from train_config.py

Browse files

Comments only: references to internal notes are removed or reworded. No configuration value, weight or config.json changes.

attnconv_b7_qwen3_4b/step_2616/train_config.py CHANGED
@@ -115,14 +115,14 @@ The consequence for reading the result: the step-1 paired gate licenses
115
  the four corrections bought ON TOP OF the attention head -- is still answerable, just not
116
  from the init: it comes from the held-out paired eval, which compares FINISHED checkpoints
117
  and never touches initialisation. That is the ruler the verdict rests on anyway; the
118
- shared init is worth exactly one step (see [[additive-module-init-trap]]).
119
 
120
  EPOCH COUNT IS THE OPEN DECISION. Written at 1 epoch so it reproduces the ladder's unit
121
  and is directly paired against every arm already on it. A longer run CONTAINS this one:
122
  checkpoint at 2616 and the 1-epoch verdict comes out of it for free, which is why there is
123
  no reason to run a separate 1-epoch arm first. If the epoch count changes, change
124
- `max_train_steps` AND `lr_schedule_steps` together -- see [[lr-schedule-confound]] for the
125
- arm that was confounded by exactly that field being left to a fallback.
126
  """
127
  import os
128
 
 
115
  the four corrections bought ON TOP OF the attention head -- is still answerable, just not
116
  from the init: it comes from the held-out paired eval, which compares FINISHED checkpoints
117
  and never touches initialisation. That is the ruler the verdict rests on anyway; the
118
+ shared init is worth exactly one step.
119
 
120
  EPOCH COUNT IS THE OPEN DECISION. Written at 1 epoch so it reproduces the ladder's unit
121
  and is directly paired against every arm already on it. A longer run CONTAINS this one:
122
  checkpoint at 2616 and the 1-epoch verdict comes out of it for free, which is why there is
123
  no reason to run a separate 1-epoch arm first. If the epoch count changes, change
124
+ `max_train_steps` AND `lr_schedule_steps` together -- an earlier
125
+ arm was confounded by exactly that field being left to a fallback.
126
  """
127
  import os
128
 
attnhead_b7_qwen3_4b/step_2616/train_config.py CHANGED
@@ -45,7 +45,7 @@ because o_proj=0 independently starves the attention branch (14 tensors) for any
45
  A null result could not then have been read as "the head does not help" rather than "the
46
  gate never opened". Nothing is zero-initialised now; instead out_norm's weight is set so
47
  the step-0 bias matches the magnitude of the vanilla markov bias it replaces (~0.064
48
- measured against 0.069), which is a scale choice and not a gate. Read a 1-epoch result with that in mind: [[oneepoch-external-anchor]] measured
49
  that a change which only pays off late reads as null at one epoch.
50
 
51
  SERVING COST (measured, refine_exp/bench_regrounding_*.py, H100, CUDA graph):
@@ -132,7 +132,7 @@ train = dict(
132
  # WRITTEN OUT, not left to a default. An arm on this ladder was once compared
133
  # against a baseline whose cosine schedule annealed to zero while its own stopped at
134
  # 99.6% of peak, because one config omitted this field and fell back to a different
135
- # value. See [[lr-schedule-confound]].
136
  lr_schedule_steps=2616,
137
  max_grad_norm=1.0,
138
  sharding_strategy="no_shard",
 
45
  A null result could not then have been read as "the head does not help" rather than "the
46
  gate never opened". Nothing is zero-initialised now; instead out_norm's weight is set so
47
  the step-0 bias matches the magnitude of the vanilla markov bias it replaces (~0.064
48
+ measured against 0.069), which is a scale choice and not a gate. Read a 1-epoch result with that in mind: an external 10-epoch reproduction showed
49
  that a change which only pays off late reads as null at one epoch.
50
 
51
  SERVING COST (measured, refine_exp/bench_regrounding_*.py, H100, CUDA graph):
 
132
  # WRITTEN OUT, not left to a default. An arm on this ladder was once compared
133
  # against a baseline whose cosine schedule annealed to zero while its own stopped at
134
  # 99.6% of peak, because one config omitted this field and fell back to a different
135
+ # value.
136
  lr_schedule_steps=2616,
137
  max_grad_norm=1.0,
138
  sharding_strategy="no_shard",