Upload logs/t64_master.log with huggingface_hub
Browse files- logs/t64_master.log +11 -0
logs/t64_master.log
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[04:53:57] === 680M 64k extension, 8x H100, cp=8 dp=1 ===
|
| 2 |
+
[04:53:57] parent : p1final (95/90/82 at x64 -- best model we have)
|
| 3 |
+
[04:53:57] YaRN : SCALE=16 x ORIG=4096 = 65536 == CP_T INVARIANT HOLDS
|
| 4 |
+
[04:53:57] virtual range 0..4095 (full), 16 tokens/virtual unit
|
| 5 |
+
[04:53:57] LR : peak 1.8e-4 (6e-4 x 0.3), warmup 100 / stable 900 / DECAY 200 -> 6e-5
|
| 6 |
+
[04:53:57] data : Data_Loader/Data/mix64 b5=50 b4=25 b3=15 b1=10, usable 1220 steps, running 1200
|
| 7 |
+
[04:53:57] PROBE cp=8 (8192 tok/rank, grad_accum 8)
|
| 8 |
+
[04:55:15] PROBE OK cp=8: peak mem: 50855MB of 81559 MiB, dt dt: 9998.16ms
|
| 9 |
+
[04:55:15] TRAINING 1200 steps
|
| 10 |
+
[06:42:22] done, run 2, loss: 2.798417
|
| 11 |
+
[06:42:22] === 64K TRAINING COMPLETE ===
|