Upload logs/128k_master.log with huggingface_hub
Browse files- logs/128k_master.log +20 -0
logs/128k_master.log
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[05:38:41] FATAL: ./build/bluscriptCP_exec not built
|
| 2 |
+
[09:16:06] === 680M 131k extension, 8x H200, cp=8 dp=1 ===
|
| 3 |
+
[09:16:06] parent : t64 (98 @65,512 -- the 64k model, staged ladder)
|
| 4 |
+
[09:16:06] YaRN : SCALE=32 x ORIG=4096 = 131072 == CP_T INVARIANT HOLDS
|
| 5 |
+
[09:16:06] virtual range 0..4095 (full), 32 tokens/virtual unit
|
| 6 |
+
[09:16:06] LR : peak 1.8e-4 (6e-4 x 0.3), warmup 100 / stable 900 / DECAY 200 -> 6e-5
|
| 7 |
+
[09:16:06] data : Data_Loader/Data/mix128 b6=8.9 b5=39 b4=25 b3=15 b1=10, usable 1219 steps, running 1200
|
| 8 |
+
[09:16:06] PROBE cp=8 (16384 tok/rank, grad_accum 4)
|
| 9 |
+
[09:16:58] PROBE FAILED rc=1 steps=0
|
| 10 |
+
[09:17:28] === 680M 131k extension, 8x H200, cp=8 dp=1 ===
|
| 11 |
+
[09:17:28] parent : t64 (98 @65,512 -- the 64k model, staged ladder)
|
| 12 |
+
[09:17:28] YaRN : SCALE=32 x ORIG=4096 = 131072 == CP_T INVARIANT HOLDS
|
| 13 |
+
[09:17:28] virtual range 0..4095 (full), 32 tokens/virtual unit
|
| 14 |
+
[09:17:28] LR : peak 1.8e-4 (6e-4 x 0.3), warmup 100 / stable 900 / DECAY 200 -> 6e-5
|
| 15 |
+
[09:17:28] data : Data_Loader/Data/mix128 b6=8.9 b5=39 b4=25 b3=15 b1=10, usable 1219 steps, running 1200
|
| 16 |
+
[09:17:28] PROBE cp=8 (16384 tok/rank, grad_accum 4)
|
| 17 |
+
[09:19:08] PROBE OK cp=8: peak mem: 86705MB of 143771 MiB (H200), dt dt: 20144.88ms
|
| 18 |
+
[09:19:08] TRAINING 1200 steps
|
| 19 |
+
[11:17:55] done, run 3, loss: 1.024427
|
| 20 |
+
[11:17:55] === 131K TRAINING COMPLETE ===
|