Maggio33 commited on
Commit
727b5f5
·
verified ·
1 Parent(s): eede7ab

Card: bands written before the run; end of second window specified after 7 of 10 values

Browse files
Files changed (1) hide show
  1. README.md +6 -4
README.md CHANGED
@@ -49,17 +49,19 @@ Data: [`analysis/scores-history.csv`](analysis/scores-history.csv) (step, tokens
49
  of three GPUs computed the row; each GPU was checked against a known point before use). 1.00 at 1.26 B and 1.42 B tokens
50
  is already in its learning-rate decay; 1.01 at the same token counts is not.
51
 
52
- ## Training loss vs. windows fixed before the run
53
 
54
- Before the run, windows for the mean training loss were written down from the training-loss scenarios on the 1.00 card
55
  (continuation of the 1.00 curve from step 1050):
56
 
57
- | steps | window fixed before the run | measured mean of logged values |
58
  |---|---|---:|
59
  | 1700–1800 (constant lr, pack-00) | 2.692 – 2.882 | 2.6977 |
60
  | 2770–2860 (end of the decay, pack-01) | 2.367 – 2.660 | 2.5083 |
61
 
62
- Both measurements fall inside their windows; the first one at the lower edge. This is one run, not a test of the
 
 
63
  scenarios.
64
 
65
  ## Usage
 
49
  of three GPUs computed the row; each GPU was checked against a known point before use). 1.00 at 1.26 B and 1.42 B tokens
50
  is already in its learning-rate decay; 1.01 at the same token counts is not.
51
 
52
+ ## Training loss vs. bands written before the run
53
 
54
+ Before the run, bands for the mean training loss were written down from the training-loss scenarios on the 1.00 card
55
  (continuation of the 1.00 curve from step 1050):
56
 
57
+ | steps | band written before the run | measured mean of logged values |
58
  |---|---|---:|
59
  | 1700–1800 (constant lr, pack-00) | 2.692 – 2.882 | 2.6977 |
60
  | 2770–2860 (end of the decay, pack-01) | 2.367 – 2.660 | 2.5083 |
61
 
62
+ Both bands were written before the run. The steps of the first window were also fixed before the run; the exact end of
63
+ the second window (the last 10 logged values, steps 2770–2860) was specified when 7 of those 10 values were already
64
+ logged. Both measurements fall inside their bands, the first one at the lower edge. This is one run, not a test of the
65
  scenarios.
66
 
67
  ## Usage