File size: 2,062 Bytes
6e0358e | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | [05:59:11] === CONTINUE ext32k through the rest of b4 ===
[05:59:11] resume run 2 at step 750 -> 2227 (1477 new steps, .774 B tokens)
[05:59:11] loader fast-forwards 12000 batches = .393 B; reads to 1.167 B of 1.171 B -- no wrap
[05:59:11] WARNING: .zero_g1 sidecar absent -- group 1 Adam moments start at zero (small transient)
[05:59:11] eval points: 1100 1650 2227
[05:59:11] training to step 1100
[06:52:12] === CONTINUE ext32k through the rest of b4 ===
[06:52:12] resume run 2 at step 750 -> 2227 (1477 new steps, .774 B tokens)
[06:52:12] loader fast-forwards 12000 batches = .393 B; reads to 1.167 B of 1.171 B -- no wrap
[06:52:12] WARNING: .zero_g1 sidecar absent -- group 1 Adam moments start at zero (small transient)
[06:52:12] eval points: 1100 1650 2227
[06:52:12] training to step 1100
[08:19:31] step 1100 reached, preserved b4c1100_blumodelcp_run2_step_1100.ckpt, loss loss: 2.558936
[08:19:31] EVAL @ step 1100
[08:21:34] step 1100 len 8192 -> niah_single,55.0
[08:25:40] step 1100 len 16360 -> niah_single,40.0
[08:34:54] step 1100 len 32744 -> niah_single,24.0
[08:34:54] training to step 1650
[10:51:34] step 1650 reached, preserved b4c1650_blumodelcp_run2_step_1650.ckpt, loss loss: 2.813203
[10:51:34] EVAL @ step 1650
[10:53:26] step 1650 len 8192 -> niah_single,64.0
[10:56:58] step 1650 len 16360 -> niah_single,55.0
[11:06:15] step 1650 len 32744 -> niah_single,33.0
[11:06:15] training to step 2227
[13:29:13] step 2227 reached, preserved b4c2227_blumodelcp_run2_step_2227.ckpt, loss loss: 2.757876
[13:29:13] EVAL @ step 2227
[13:30:57] step 2227 len 8192 -> niah_single,70.0
[13:34:20] step 2227 len 16360 -> niah_single,58.0
[13:43:00] step 2227 len 32744 -> niah_single,43.0
[13:43:00] === B4 CONTINUATION DONE ===
[13:43:00] progression vs ext32k@750 (niah_single):
[13:43:00] len 8192: 750=(see ext32k) 1100=55.0
1650=64.0
2227=70.0
[13:43:00] len 16360: 750=(see ext32k) 1100=40.0
1650=55.0
2227=58.0
[13:43:00] len 32744: 750=(see ext32k) 1100=24.0
1650=33.0
2227=43.0
|