Upload logs/mixcont/master.log with huggingface_hub
Browse files- logs/mixcont/master.log +14 -0
logs/mixcont/master.log
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[09:41:07] === mix -> FULL b5 EPOCH (8x H100, cp=8 dp=1) ===
|
| 2 |
+
[09:41:07] parent: mix64k step 1200 (83/70/50/31), weights-only via CP_INIT_FROM
|
| 3 |
+
[09:41:07] data: Data_Loader/Data/mixcont -- continuation from stream offset 629,342,677 (junction verified)
|
| 4 |
+
[09:41:07] b5 reaches 100% of its split in this run
|
| 5 |
+
[09:41:07] steps: 1200 -> 3300 (2100 new, 2102 usable)
|
| 6 |
+
[09:41:07] WSD: warmup 100, stable 1998, tail 2
|
| 7 |
+
[09:41:07] evals: global 2000 2600 3300 = local 800 1400 2100 (full resumable at these)
|
| 8 |
+
[09:41:07] disk: 140 GB free
|
| 9 |
+
[09:41:07] PHASE 1: 3-step probe (8,192 tok/rank; expect ~70 GB of 81.5)
|
| 10 |
+
[09:42:30] PROBE OK: 3 steps, peak mem: 71847MB
|
| 11 |
+
[09:42:30] PHASE 2: training 1200 -> 3300
|
| 12 |
+
[10:28:13] training done, run 2, loss: 1.440003
|
| 13 |
+
[10:28:13] peak mem mem: 72103MB
|
| 14 |
+
[10:28:13] === PHASE 2 COMPLETE -- finisher takes over for evals and upload ===
|