unparallelled commited on
Commit
29d844a
·
verified ·
1 Parent(s): cf86eb7

Upload logs/2026-09-22_t128c_ship.log with huggingface_hub

Browse files
Files changed (1) hide show
  1. logs/2026-09-22_t128c_ship.log +82 -0
logs/2026-09-22_t128c_ship.log ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [06:38:26] === t128c shipper: rank_0 every save, full set at 300 600 ===
2
+ [06:39:04] === t128c shipper: rank_0 every save, full set at 300 600 ===
3
+ [06:47:52] shipped t128c_step50_rank0.ckpt (2.0G)
4
+ [07:01:27] shipped t128c_step100_rank0.ckpt (2.0G)
5
+ [07:14:43] shipped t128c_step150_rank0.ckpt (2.0G)
6
+ [07:28:49] shipped t128c_step200_rank0.ckpt (2.0G)
7
+ [07:43:25] shipped t128c_step250_rank0.ckpt (2.0G)
8
+ [07:55:09] shipped t128c_step300_rank0.ckpt (2.0G)
9
+ [07:55:26] shipped t128c_step300_rank0.zero_g1 (590M)
10
+ [07:55:45] shipped t128c_step300_rank1.ckpt (2.0G)
11
+ [07:55:48] shipped t128c_step300_rank1.zero_g1 (140K)
12
+ [07:56:09] shipped t128c_step300_rank2.ckpt (2.0G)
13
+ [07:56:12] shipped t128c_step300_rank2.zero_g1 (140K)
14
+ [07:56:34] shipped t128c_step300_rank3.ckpt (2.0G)
15
+ [07:56:38] shipped t128c_step300_rank3.zero_g1 (140K)
16
+ [07:56:55] shipped t128c_step300_rank4.ckpt (2.0G)
17
+ [07:56:57] shipped t128c_step300_rank4.zero_g1 (140K)
18
+ [07:59:15] shipped t128c_step300_rank5.ckpt (2.0G)
19
+ [07:59:18] shipped t128c_step300_rank5.zero_g1 (140K)
20
+ [07:59:48] shipped t128c_step300_rank6.ckpt (2.0G)
21
+ [07:59:51] shipped t128c_step300_rank6.zero_g1 (140K)
22
+ [08:00:16] shipped t128c_step300_rank7.ckpt (2.0G)
23
+ [08:00:20] shipped t128c_step300_rank7.zero_g1 (136K)
24
+ [08:00:20] MILESTONE 300: full resumable set up
25
+ [08:01:05] MILESTONE 300: full resumable set up
26
+ [08:01:50] MILESTONE 300: full resumable set up
27
+ [08:02:35] MILESTONE 300: full resumable set up
28
+ [08:03:20] MILESTONE 300: full resumable set up
29
+ [08:04:06] MILESTONE 300: full resumable set up
30
+ [08:04:51] MILESTONE 300: full resumable set up
31
+ [08:05:36] MILESTONE 300: full resumable set up
32
+ [08:06:21] MILESTONE 300: full resumable set up
33
+ [08:07:06] MILESTONE 300: full resumable set up
34
+ [08:07:51] MILESTONE 300: full resumable set up
35
+ [08:08:36] MILESTONE 300: full resumable set up
36
+ [08:15:29] shipped t128c_step350_rank0.ckpt (2.0G)
37
+ [08:16:14] MILESTONE 300: full resumable set up
38
+ [08:16:59] MILESTONE 300: full resumable set up
39
+ [08:17:44] MILESTONE 300: full resumable set up
40
+ [08:18:29] MILESTONE 300: full resumable set up
41
+ [08:19:14] MILESTONE 300: full resumable set up
42
+ [08:19:59] MILESTONE 300: full resumable set up
43
+ [08:20:44] MILESTONE 300: full resumable set up
44
+ [08:21:29] MILESTONE 300: full resumable set up
45
+ [08:22:12] shipped t128c_step400_rank0.ckpt (2.0G)
46
+ [08:22:58] MILESTONE 300: full resumable set up
47
+ [08:23:43] MILESTONE 300: full resumable set up
48
+ [08:24:28] MILESTONE 300: full resumable set up
49
+ [08:25:13] MILESTONE 300: full resumable set up
50
+ [08:25:58] MILESTONE 300: full resumable set up
51
+ [08:26:43] MILESTONE 300: full resumable set up
52
+ [08:27:28] MILESTONE 300: full resumable set up
53
+ [08:28:13] MILESTONE 300: full resumable set up
54
+ [08:28:59] MILESTONE 300: full resumable set up
55
+ [08:29:44] MILESTONE 300: full resumable set up
56
+ [08:30:29] MILESTONE 300: full resumable set up
57
+ [08:31:14] MILESTONE 300: full resumable set up
58
+ [08:31:59] MILESTONE 300: full resumable set up
59
+ [08:32:44] MILESTONE 300: full resumable set up
60
+ [08:33:29] MILESTONE 300: full resumable set up
61
+ [08:34:14] MILESTONE 300: full resumable set up
62
+ [08:34:59] MILESTONE 300: full resumable set up
63
+ [08:35:24] shipped t128c_step450_rank0.ckpt (2.0G)
64
+ [08:36:10] MILESTONE 300: full resumable set up
65
+ [08:36:55] MILESTONE 300: full resumable set up
66
+ [08:37:40] MILESTONE 300: full resumable set up
67
+ [08:38:25] MILESTONE 300: full resumable set up
68
+ [08:39:10] MILESTONE 300: full resumable set up
69
+ [08:39:55] MILESTONE 300: full resumable set up
70
+ [08:40:40] MILESTONE 300: full resumable set up
71
+ [08:41:25] MILESTONE 300: full resumable set up
72
+ [08:42:11] MILESTONE 300: full resumable set up
73
+ [08:42:56] MILESTONE 300: full resumable set up
74
+ [08:43:41] MILESTONE 300: full resumable set up
75
+ [08:44:26] MILESTONE 300: full resumable set up
76
+ [08:45:11] MILESTONE 300: full resumable set up
77
+ [08:45:56] MILESTONE 300: full resumable set up
78
+ [08:46:41] MILESTONE 300: full resumable set up
79
+ [08:47:26] MILESTONE 300: full resumable set up
80
+ [08:48:12] MILESTONE 300: full resumable set up
81
+ [08:49:24] shipped t128c_step500_rank0.ckpt (2.0G)
82
+ [08:51:10] === trainer gone; shipped 25 files ===