Upload log_ts_e1.txt with huggingface_hub

log_ts_e1.txt

@@ -281,29 +281,59 @@ Consider using tensor.detach() first. (Triggered internally at /pytorch/torch/cs
 
  /tmp/ts_e1_m.pt             : 100%|██████████| 75.8MB / 75.8MB            
 
 
  /tmp/ts_e1_m.pt             : 100%|██████████| 75.8MB / 75.8MB            
-02:55:23 pushed ts_e1.pt
-02:55:24 push fail (Request ID: Root=1-6aa21c1c-6e9b1ab2694fbc80270fa240;e6cd0260-8178-49a4-ac6e-1d3799745dc3)
+
+
+
+
  /tmp/ts_e1_m.pt             :   3%|▎         | 2.60MB / 75.8MB            
+
+
  /tmp/ts_e1_m.pt             :   3%|▎         | 2.60MB / 75.8MB            
+
+
+
  /tmp/ts_e1_m.pt             :  30%|██▉       | 22.7MB / 75.8MB            
+
+
+
  /tmp/ts_e1_m.pt             :  55%|█████▌    | 41.7MB / 75.8MB            
+
+
+
  /tmp/ts_e1_m.pt             :  55%|█████▌    | 41.7MB / 75.8MB            
+
+
  /tmp/ts_e1_m.pt             :  62%|██████▏   | 46.9MB / 75.8MB            
+
+
+
  /tmp/ts_e1_m.pt             :  99%|█████████▉| 75.4MB / 75.8MB            
+
+
+
  /tmp/ts_e1_m.pt             :  99%|█████████▉| 75.4MB / 75.8MB            
+
+
  /tmp/ts_e1_m.pt             :  99%|█████████▉| 75.4MB / 75.8MB            
+
+
  /tmp/ts_e1_m.pt             :  99%|█████████▉| 75.4MB / 75.8MB            
+
+
  /tmp/ts_e1_m.pt             :  99%|█████████▉| 75.4MB / 75.8MB            
+
+
  /tmp/ts_e1_m.pt             :  99%|█████████▉| 75.4MB / 75.8MB            
+
+
  /tmp/ts_e1_m.pt             :  99%|█████████▉| 75.4MB / 75.8MB            
+
+
  /tmp/ts_e1_m.pt             : 100%|██████████| 75.8MB / 75.8MB            
+
+
+
  /tmp/ts_e1_m.pt             : 100%|██████████| 75.8MB / 75.8MB            
+
+
  /tmp/ts_e1_m.pt             : 100%|██████████| 75.8MB / 75.8MB            
+03:03:13 push fail (Request ID: Root=1-6aa21df1-42ca59670a11c29818b4dc37;fb5da22f-8c2f-4a7f-8801-b2adfa6c90e3)
+
+429 Too Many Requests: you 
+03:03:14 push fail (Request ID: Root=1-6aa21df2-6dd8635a5deda4436e6a353e;b019194f-9917-4c28-a24a-6ae3f8a9e565)
 
 429 Too Many Requests: you 
-02:55:38 step 99100 loss=3.3030 tok=1623.7M dt=3236
-02:55:54 step 99200 loss=3.1954 tok=1625.3M dt=3251
-02:56:10 step 99300 loss=3.2088 tok=1626.9M dt=3267
-02:56:26 step 99400 loss=3.2130 tok=1628.6M dt=3283
-02:56:41 step 99500 loss=3.0537 tok=1630.2M dt=3299
-02:56:57 step 99600 loss=2.9860 tok=1631.8M dt=3314
-02:57:12 step 99700 loss=3.0418 tok=1633.5M dt=3330
-02:57:28 step 99800 loss=3.2092 tok=1635.1M dt=3345
-02:57:43 step 99900 loss=3.1020 tok=1636.8M dt=3360
-02:57:58 step 100000 loss=3.0514 tok=1638.4M dt=3376
-02:57:59 pushed log_ts_e1.txt
-02:58:14 step 100100 loss=3.2910 tok=1640.0M dt=3392
-02:58:29 step 100200 loss=3.2070 tok=1641.7M dt=3407
-02:58:44 step 100300 loss=3.0791 tok=1643.3M dt=3422
-02:58:59 step 100400 loss=3.0260 tok=1645.0M dt=3436
-02:59:14 step 100500 loss=3.1751 tok=1646.6M dt=3451
-02:59:28 step 100600 loss=3.1796 tok=1648.2M dt=3466
-02:59:43 step 100700 loss=3.1122 tok=1649.9M dt=3481
-02:59:59 step 100800 loss=3.1172 tok=1651.5M dt=3496
-03:00:16 epoch 4 start
-03:00:25 step 100900 loss=3.1670 tok=1653.1M dt=3522
-03:00:40 step 101000 loss=3.1662 tok=1654.8M dt=3538
+03:03:29 step 102100 loss=3.2735 tok=1672.8M dt=3706
+03:03:44 step 102200 loss=3.0485 tok=1674.4M dt=3721
+03:03:59 step 102300 loss=3.2192 tok=1676.1M dt=3736
+03:04:13 step 102400 loss=3.0703 tok=1677.7M dt=3751
+03:04:29 step 102500 loss=2.9966 tok=1679.4M dt=3766
+03:04:43 step 102600 loss=3.2914 tok=1681.0M dt=3781
+03:04:58 step 102700 loss=3.3013 tok=1682.6M dt=3795
+03:05:13 step 102800 loss=3.2005 tok=1684.3M dt=3811
+03:05:28 step 102900 loss=3.1839 tok=1685.9M dt=3825
+03:05:43 step 103000 loss=3.0790 tok=1687.6M dt=3840