abl_C: train.log

abl_C/train.log

@@ -0,0 +1,29 @@
+09-29 15:25:31 pool new_text: 106090 -> use 16000
+09-29 15:25:31 pool old_text: 83086 -> use 4000
+09-29 15:25:31 pool mm: 40470 -> use 0
+09-29 15:25:45 model ready, cfg OmniCfg(n_latents=8, media_exit=8, sibling_from=16, image_tokens=280, frame_tokens=70, max_frames=8, option_context_max=8, catalog=True, catalog_max=128), LoRA 28.7M, train 19200 calib 800 eval 13922
+09-29 15:25:45 1000 steps (1624 batches, mean 11.8 records), start 0
+09-29 15:25:51 step 1/1000 loss 3.823 | 5.98s/step ETA 100m | mem 18.3G | oom 0
+09-29 15:27:08 step 50/1000 loss 2.269 | 1.65s/step ETA 26m | mem 24.1G | oom 0
+09-29 15:28:29 step 100/1000 loss 1.539 | 1.64s/step ETA 25m | mem 24.4G | oom 0
+09-29 15:29:36 step 150/1000 loss 1.219 | 1.54s/step ETA 22m | mem 24.4G | oom 0
+09-29 15:30:59 step 200/1000 loss 1.129 | 1.57s/step ETA 21m | mem 27.2G | oom 0
+09-29 15:32:09 step 250/1000 loss 1.011 | 1.53s/step ETA 19m | mem 27.2G | oom 0
+09-29 15:33:16 step 300/1000 loss 1.040 | 1.50s/step ETA 18m | mem 27.2G | oom 0
+09-29 15:34:13 step 350/1000 loss 0.950 | 1.45s/step ETA 16m | mem 27.2G | oom 0
+09-29 15:35:10 step 400/1000 loss 0.963 | 1.41s/step ETA 14m | mem 27.2G | oom 0
+09-29 15:36:04 step 450/1000 loss 0.892 | 1.38s/step ETA 13m | mem 27.2G | oom 0
+09-29 15:37:05 step 500/1000 loss 0.851 | 1.36s/step ETA 11m | mem 27.2G | oom 0
+09-29 15:38:14 step 550/1000 loss 0.900 | 1.36s/step ETA 10m | mem 27.2G | oom 0
+09-29 15:39:10 step 600/1000 loss 0.834 | 1.34s/step ETA 9m | mem 27.2G | oom 0
+09-29 15:39:58 step 650/1000 loss 0.766 | 1.31s/step ETA 8m | mem 27.2G | oom 0
+09-29 15:41:01 step 700/1000 loss 0.808 | 1.31s/step ETA 7m | mem 27.2G | oom 0
+09-29 15:42:01 step 750/1000 loss 0.789 | 1.30s/step ETA 5m | mem 27.2G | oom 0
+09-29 15:43:05 step 800/1000 loss 0.802 | 1.30s/step ETA 4m | mem 27.2G | oom 0
+09-29 15:44:03 step 850/1000 loss 0.826 | 1.29s/step ETA 3m | mem 27.2G | oom 0
+09-29 15:44:58 step 900/1000 loss 0.812 | 1.28s/step ETA 2m | mem 27.2G | oom 0
+09-29 15:45:51 step 950/1000 loss 0.764 | 1.27s/step ETA 1m | mem 27.2G | oom 0
+09-29 15:46:55 step 1000/1000 loss 0.806 | 1.27s/step ETA 0m | mem 27.2G | oom 0
+09-29 15:46:55 TRAIN DONE 1270s, oom 0
+09-29 15:47:29 temperatures {'choice:3-5': 1.15, 'choice:11-30': 1.0, 'noul:2': 1.3, 'choice:6-10': 1.3, 'score:3-5': 0.9, 'choice:2': 1.3, 'choice:31+': 1.0}; trust head acc 0.756 (base 0.714)
+09-29 15:47:33 [eval] jevbench_original      n=   72 acc 0.944 ece 0.132 (2s)