| [14:30:41] === 32k -> 64k YaRN extension on b5 (4x H200, cp=4) === |
| [14:30:41] parent: b4 full epoch step 2227 (best 32k model) |
| [14:30:41] YaRN SCALE=2 ORIG=32768 -> 65536 == CP_T=65536 OK |
| [14:30:41] b5: 5 train shards = .481 B tokens available |
| [14:30:41] hard stop 18:00 UTC = 23:30 IST, reserving 50m |
| [14:30:41] PHASE 1: 3-step probe (64k at cp=4 projects to 108.8 GB of 143 GB) |
| [14:32:19] PROBE OK: peak mem: 108589MB |
| [14:32:19] measured step cost at real batch: 74.009 s/step |
| [14:32:19] PHASE 2: 210 steps = .110 B tokens = 12% of a b5 epoch |
| [14:32:19] WSD: warmup 100, stable 10, decay tail 100 -> FINAL checkpoint IS decayed |
| [14:41:08] === 32k -> 64k YaRN extension on b5 (4x H200, cp=4) === |
| [14:41:08] parent: b4 full epoch step 2227 (best 32k model) |
| [14:41:08] YaRN SCALE=2 ORIG=32768 -> 65536 == CP_T=65536 OK |
| [14:41:08] b5: 5 train shards = .481 B tokens available |
| [14:41:08] hard stop 18:00 UTC = 23:30 IST, reserving 50m |
| [14:41:08] measured step cost at real batch: 74.009 s/step |
| [14:41:08] STEPS_OVERRIDE=695 |
| [14:41:08] overstated the real step cost 5.8x (74.0 vs a measured 12.85 s/step). |
| [14:41:08] PHASE 2: 695 steps = .364 B tokens = 42% of a b5 epoch |
| [14:41:08] WSD: warmup 100, stable 593, decay tail 2 -> FINAL checkpoint IS decayed |
| [17:14:05] ext64k -> run 3, blumodelcp_run3_step_695.ckpt, loss loss: 2.934074 |
| [17:14:05] peak mem mem: 108493MB |
| [17:14:05] final lr lr 1.2000e-04 (should be ~6e-05 = decayed) |
| [17:14:05] PHASE 3: eval ext64k |
| [17:41:08] len 8192 -> niah_single,67.0
|
| [17:41:08] len 16360 -> niah_single,65.0
|
| [17:41:08] len 32744 -> niah_single,52.0
|
| [17:41:08] len 65512 -> niah_single,29.0
|
| [17:41:08] PHASE 4: upload |
| [17:41:10] ok docs/EXT64K_B5_RESULTS_2026-09-10.md |
| [17:41:12] ok logs/ext64k_len_8192/ |
| [17:41:14] ok logs/ext64k_len_16360/ |
| [17:41:17] ok logs/ext64k_len_32744/ |
| [17:41:21] ok logs/ext64k_len_65512/ |
|
|
