Upload logs/cont64k/master.log with huggingface_hub

logs/cont64k/master.log

@@ -0,0 +1,25 @@
+[14:27:33] === CONTINUE 64k b5 to a FULL EPOCH (8x H100, cp=8) ===
+[14:27:33] parent: ext64k run3 step 695 (64k model), weights-only via CP_INIT_FROM
+[14:27:33] YaRN SCALE=2 ORIG=32768 -> 65536 == CP_T=65536  OK
+[14:27:33] b5: 9 train shards = .865 B tokens on disk
+[14:27:33] disk: 123 GB free
+[14:27:33] PHASE 1: 3-step probe (8,192 tok/rank; extrapolates to ~67 GB of 81.5 GB)
+[14:28:49] PROBE OK: peak 
+[14:28:50] probe rate (UPPER BOUND, inflated by fixed costs x grad_accum): 0.0 s/step
+[14:28:50] PHASE 2: 1651 steps x 524288 = .865 B tokens = a full b5 epoch
+[14:28:50]   WSD: warmup 100, stable 1451, decay tail 100 -> final checkpoint IS decayed
+[14:30:03] FATAL: run 2 wrote NO checkpoint
+[14:37:39] === CONTINUE 64k b5 to a FULL EPOCH (8x H100, cp=8) ===
+[14:37:39] parent: ext64k run3 step 695 (64k model), weights-only via CP_INIT_FROM
+[14:37:39] YaRN SCALE=2 ORIG=32768 -> 65536 == CP_T=65536  OK
+[14:37:39] b5: 9 train shards = .865 B tokens on disk
+[14:37:39] disk: 123 GB free
+[14:37:39] PHASE 1: 3-step probe (8,192 tok/rank; extrapolates to ~67 GB of 81.5 GB)
+[14:40:04] PROBE OK: 3 steps, peak mem: 71843MB of 81559 MiB/card
+[14:40:04] probe rate (UPPER BOUND, inflated by fixed costs x grad_accum): 84.100 s/step
+[14:40:04] PHASE 2: 1651 steps x 524288 = .865 B tokens = a full b5 epoch
+[14:40:04]   WSD: warmup 100, stable 1451, decay tail 100 -> final checkpoint IS decayed
+[18:22:46] cont64k -> run 4, blumodelcp_run4_step_1651.ckpt, loss: 1.143309
+[18:22:46]   peak mem mem: 72099MB
+[18:22:46]   final lr lr 6.0030e-05  (want ~6e-05 = decayed)
+[18:22:46] PHASE 3: eval cont64k -- 4 lengths, one GPU each