`rope_scaling`'s beta_fast field must be a float, got 32
`rope_scaling`'s beta_slow field must be a float, got 1
`rope_scaling`'s beta_fast field must be a float, got 32
`rope_scaling`'s beta_slow field must be a float, got 1
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
⚙️ Running in WANDB offline mode
🦥 Unsloth Zoo will now patch everything to make training faster!
==((====))== Unsloth 2026.9.2: Fast Olmo3 patching. Transformers: 4.57.6.
\\ /| NVIDIA H100 80GB HBM3. Num GPUs = 1. Max memory: 79.179 GB. Platform: Linux.
O^O/ \_/ \ Torch: 2.11.0+cu128. CUDA: 9.0. CUDA Toolkit: 12.8. Triton: 3.6.0
\ / Bfloat16 = TRUE. FA [Xformers = 0.0.35. FA2 = False]
"-____-" Free license: http://github.com/unslothai/unsloth
Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
Unsloth: QLoRA and full finetuning all not selected. Switching to 16bit LoRA.
Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s]Loading checkpoint shards: 33%|███▎ | 1/3 [00:00<00:01, 1.39it/s]Loading checkpoint shards: 67%|██████▋ | 2/3 [00:01<00:00, 1.38it/s]Loading checkpoint shards: 100%|██████████| 3/3 [00:02<00:00, 1.42it/s]Loading checkpoint shards: 100%|██████████| 3/3 [00:02<00:00, 1.41it/s]
ROPE_GUARD OK train_unsloth load CompassioninMachineLearning/olmo-compassion-web-seed2-CPT-best: olmo3 theta=500000.0 scaling={'rope_type': 'yarn', 'factor': 8.0, 'original_max_position_embeddings': 8192.0, 'beta_fast': 32.0, 'beta_slow': 1.0, 'attention_factor': 1.2079441541679836}
Unsloth: Dropout = 0 is supported for fast patching. You are using dropout = 0.05.
Unsloth will patch all other layers, except LoRA matrices, causing a performance hit.
Unsloth: Explicit target_modules are constrained by the finetune_(vision|language|attention|mlp) filters; adapters attach only where both select.
Unsloth: Allowing gradients for `base_model.model.model.embed_tokens` since it's in `modules_to_save`.
Unsloth: Allowing gradients for `base_model.model.lm_head` since it's in `modules_to_save`.
Generating train split: 0 examples [00:00, ? examples/s]Generating train split: 890 examples [00:00, 5319.50 examples/s]Generating train split: 1781 examples [00:00, 2540.67 examples/s]Generating train split: 2643 examples [00:00, 3055.20 examples/s]Generating train split: 3587 examples [00:01, 3872.16 examples/s]Generating train split: 4465 examples [00:01, 3583.66 examples/s]Generating train split: 5410 examples [00:01, 2897.26 examples/s]Generating train split: 6313 examples [00:01, 3351.21 examples/s]Generating train split: 7214 examples [00:02, 3583.27 examples/s]Generating train split: 8108 examples [00:02, 3777.57 examples/s]Generating train split: 9017 examples [00:02, 2956.99 examples/s]Generating train split: 9904 examples [00:02, 3294.76 examples/s]Generating train split: 10797 examples [00:03, 3645.41 examples/s]Generating train split: 11743 examples [00:03, 2839.51 examples/s]Generating train split: 12646 examples [00:03, 3194.23 examples/s]Generating train split: 13586 examples [00:04, 3599.93 examples/s]Generating train split: 14510 examples [00:04, 3829.93 examples/s]Generating train split: 15424 examples [00:04, 3010.59 examples/s]Generating train split: 16339 examples [00:04, 3291.50 examples/s]Generating train split: 17261 examples [00:05, 3684.79 examples/s]Generating train split: 18165 examples [00:05, 2880.21 examples/s]Generating train split: 18500 examples [00:05, 3263.30 examples/s]
Generating train split: 0 examples [00:00, ? examples/s]Generating train split: 5000 examples [00:00, 8873.76 examples/s]Generating train split: 5000 examples [00:00, 8645.79 examples/s]
tokenizing (num_proc=8): 0%| | 0/23500 [00:00<?, ? examples/s]tokenizing (num_proc=8): 0%| | 10/23500 [00:00<24:15, 16.14 examples/s]tokenizing (num_proc=8): 0%| | 58/23500 [00:00<03:50, 101.91 examples/s]tokenizing (num_proc=8): 1%| | 136/23500 [00:00<01:37, 239.66 examples/s]tokenizing (num_proc=8): 1%| | 241/23500 [00:00<00:55, 419.15 examples/s]tokenizing (num_proc=8): 2%|▏ | 404/23500 [00:01<00:32, 706.85 examples/s]tokenizing (num_proc=8): 2%|▏ | 577/23500 [00:01<00:23, 959.96 examples/s]tokenizing (num_proc=8): 3%|▎ | 792/23500 [00:01<00:17, 1265.49 examples/s]tokenizing (num_proc=8): 4%|▍ | 996/23500 [00:01<00:15, 1447.71 examples/s]tokenizing (num_proc=8): 5%|▌ | 1193/23500 [00:01<00:14, 1536.99 examples/s]tokenizing (num_proc=8): 6%|▌ | 1377/23500 [00:01<00:14, 1559.21 examples/s]tokenizing (num_proc=8): 7%|▋ | 1579/23500 [00:01<00:13, 1676.85 examples/s]tokenizing (num_proc=8): 8%|▊ | 1781/23500 [00:01<00:12, 1745.30 examples/s]tokenizing (num_proc=8): 8%|▊ | 1967/23500 [00:01<00:12, 1746.78 examples/s]tokenizing (num_proc=8): 9%|▉ | 2149/23500 [00:02<00:12, 1733.47 examples/s]tokenizing (num_proc=8): 10%|▉ | 2330/23500 [00:02<00:12, 1737.12 examples/s]tokenizing (num_proc=8): 11%|█ | 2525/23500 [00:02<00:12, 1744.37 examples/s]tokenizing (num_proc=8): 12%|█▏ | 2703/23500 [00:02<00:12, 1673.21 examples/s]tokenizing (num_proc=8): 12%|█▏ | 2896/23500 [00:02<00:11, 1731.24 examples/s]tokenizing (num_proc=8): 13%|█▎ | 3113/23500 [00:02<00:11, 1788.93 examples/s]tokenizing (num_proc=8): 14%|█▍ | 3325/23500 [00:02<00:10, 1856.23 examples/s]tokenizing (num_proc=8): 15%|█▍ | 3518/23500 [00:02<00:10, 1829.49 examples/s]tokenizing (num_proc=8): 16%|█▌ | 3702/23500 [00:02<00:10, 1814.77 examples/s]tokenizing (num_proc=8): 17%|█▋ | 3900/23500 [00:02<00:10, 1843.77 examples/s]tokenizing (num_proc=8): 17%|█▋ | 4090/23500 [00:03<00:11, 1756.22 examples/s]tokenizing (num_proc=8): 18%|█▊ | 4268/23500 [00:03<00:10, 1751.36 examples/s]tokenizing (num_proc=8): 19%|█▉ | 4451/23500 [00:03<00:10, 1762.23 examples/s]tokenizing (num_proc=8): 20%|█▉ | 4650/23500 [00:03<00:10, 1804.58 examples/s]tokenizing (num_proc=8): 21%|██ | 4838/23500 [00:03<00:10, 1804.20 examples/s]tokenizing (num_proc=8): 21%|██▏ | 5050/23500 [00:03<00:09, 1880.47 examples/s]tokenizing (num_proc=8): 22%|██▏ | 5240/23500 [00:03<00:10, 1802.90 examples/s]tokenizing (num_proc=8): 23%|██▎ | 5426/23500 [00:03<00:10, 1779.91 examples/s]tokenizing (num_proc=8): 24%|██▍ | 5626/23500 [00:03<00:09, 1814.99 examples/s]tokenizing (num_proc=8): 25%|██▍ | 5815/23500 [00:04<00:09, 1799.52 examples/s]tokenizing (num_proc=8): 26%|██▌ | 6002/23500 [00:04<00:09, 1775.45 examples/s]tokenizing (num_proc=8): 26%|██▋ | 6186/23500 [00:04<00:09, 1746.92 examples/s]tokenizing (num_proc=8): 27%|██▋ | 6362/23500 [00:04<00:09, 1721.67 examples/s]tokenizing (num_proc=8): 28%|██▊ | 6536/23500 [00:04<00:10, 1681.44 examples/s]tokenizing (num_proc=8): 29%|██▊ | 6707/23500 [00:04<00:10, 1670.17 examples/s]tokenizing (num_proc=8): 29%|██▉ | 6878/23500 [00:04<00:10, 1656.82 examples/s]tokenizing (num_proc=8): 30%|██▉ | 7045/23500 [00:04<00:10, 1640.12 examples/s]tokenizing (num_proc=8): 31%|███ | 7220/23500 [00:04<00:11, 1479.40 examples/s]tokenizing (num_proc=8): 31%|███▏ | 7383/23500 [00:05<00:11, 1442.23 examples/s]tokenizing (num_proc=8): 32%|███▏ | 7534/23500 [00:05<00:12, 1327.70 examples/s]tokenizing (num_proc=8): 33%|███▎ | 7673/23500 [00:05<00:13, 1193.50 examples/s]tokenizing (num_proc=8): 33%|███▎ | 7803/23500 [00:05<00:15, 1034.42 examples/s]tokenizing (num_proc=8): 34%|███▎ | 7924/23500 [00:05<00:15, 1030.30 examples/s]tokenizing (num_proc=8): 34%|███▍ | 8039/23500 [00:05<00:16, 920.21 examples/s] tokenizing (num_proc=8): 35%|███▍ | 8148/23500 [00:05<00:16, 925.50 examples/s]tokenizing (num_proc=8): 35%|███▌ | 8248/23500 [00:06<00:16, 941.37 examples/s]tokenizing (num_proc=8): 36%|███▌ | 8375/23500 [00:06<00:14, 1016.54 examples/s]tokenizing (num_proc=8): 36%|███▌ | 8514/23500 [00:06<00:13, 1092.91 examples/s]tokenizing (num_proc=8): 37%|███▋ | 8674/23500 [00:06<00:12, 1200.73 examples/s]tokenizing (num_proc=8): 38%|███▊ | 8836/23500 [00:06<00:11, 1289.74 examples/s]tokenizing (num_proc=8): 38%|███▊ | 9019/23500 [00:06<00:10, 1390.90 examples/s]tokenizing (num_proc=8): 39%|███▉ | 9211/23500 [00:06<00:09, 1509.22 examples/s]tokenizing (num_proc=8): 40%|███▉ | 9391/23500 [00:06<00:09, 1561.49 examples/s]tokenizing (num_proc=8): 41%|████ | 9583/23500 [00:06<00:08, 1629.02 examples/s]tokenizing (num_proc=8): 42%|████▏ | 9775/23500 [00:06<00:08, 1679.28 examples/s]tokenizing (num_proc=8): 42%|████▏ | 9980/23500 [00:07<00:07, 1730.40 examples/s]tokenizing (num_proc=8): 43%|████▎ | 10164/23500 [00:07<00:07, 1721.04 examples/s]tokenizing (num_proc=8): 44%|████▍ | 10387/23500 [00:07<00:07, 1849.20 examples/s]tokenizing (num_proc=8): 45%|████▌ | 10591/23500 [00:07<00:06, 1896.27 examples/s]tokenizing (num_proc=8): 46%|████▌ | 10795/23500 [00:07<00:06, 1839.97 examples/s]tokenizing (num_proc=8): 47%|████▋ | 10998/23500 [00:07<00:06, 1876.24 examples/s]tokenizing (num_proc=8): 48%|████▊ | 11187/23500 [00:07<00:06, 1862.64 examples/s]tokenizing (num_proc=8): 48%|████▊ | 11377/23500 [00:07<00:06, 1847.88 examples/s]tokenizing (num_proc=8): 49%|████▉ | 11573/23500 [00:07<00:06, 1758.23 examples/s]tokenizing (num_proc=8): 50%|█████ | 11774/23500 [00:08<00:06, 1813.85 examples/s]tokenizing (num_proc=8): 51%|█████ | 11964/23500 [00:08<00:06, 1781.53 examples/s]tokenizing (num_proc=8): 52%|█████▏ | 12149/23500 [00:08<00:06, 1787.83 examples/s]tokenizing (num_proc=8): 53%|█████▎ | 12349/23500 [00:08<00:06, 1791.87 examples/s]tokenizing (num_proc=8): 53%|█████▎ | 12539/23500 [00:08<00:06, 1804.79 examples/s]tokenizing (num_proc=8): 54%|█████▍ | 12720/23500 [00:08<00:06, 1741.90 examples/s]tokenizing (num_proc=8): 55%|█████▌ | 12945/23500 [00:08<00:05, 1843.07 examples/s]tokenizing (num_proc=8): 56%|█████▌ | 13149/23500 [00:08<00:05, 1862.13 examples/s]tokenizing (num_proc=8): 57%|█████▋ | 13350/23500 [00:08<00:05, 1882.77 examples/s]tokenizing (num_proc=8): 58%|█████▊ | 13550/23500 [00:09<00:05, 1869.50 examples/s]tokenizing (num_proc=8): 59%|█████▊ | 13750/23500 [00:09<00:05, 1831.79 examples/s]tokenizing (num_proc=8): 59%|█████▉ | 13947/23500 [00:09<00:05, 1817.30 examples/s]tokenizing (num_proc=8): 60%|██████ | 14148/23500 [00:09<00:05, 1862.38 examples/s]tokenizing (num_proc=8): 61%|██████ | 14343/23500 [00:09<00:05, 1744.44 examples/s]tokenizing (num_proc=8): 62%|██████▏ | 14529/23500 [00:09<00:05, 1740.88 examples/s]tokenizing (num_proc=8): 63%|██████▎ | 14718/23500 [00:09<00:05, 1746.84 examples/s]tokenizing (num_proc=8): 63%|██████▎ | 14904/23500 [00:09<00:05, 1701.03 examples/s]tokenizing (num_proc=8): 64%|██████▍ | 15076/23500 [00:09<00:05, 1621.07 examples/s]tokenizing (num_proc=8): 65%|██████▍ | 15247/23500 [00:10<00:05, 1608.35 examples/s]tokenizing (num_proc=8): 66%|██████▌ | 15416/23500 [00:10<00:05, 1547.45 examples/s]tokenizing (num_proc=8): 66%|██████▋ | 15572/23500 [00:10<00:05, 1499.76 examples/s]tokenizing (num_proc=8): 67%|██████▋ | 15746/23500 [00:10<00:04, 1552.41 examples/s]tokenizing (num_proc=8): 68%|██████▊ | 15906/23500 [00:10<00:05, 1489.24 examples/s]tokenizing (num_proc=8): 68%|██████▊ | 16063/23500 [00:10<00:05, 1374.94 examples/s]tokenizing (num_proc=8): 69%|██████▉ | 16206/23500 [00:10<00:05, 1329.77 examples/s]tokenizing (num_proc=8): 70%|██████▉ | 16353/23500 [00:10<00:05, 1314.97 examples/s]tokenizing (num_proc=8): 70%|███████ | 16563/23500 [00:10<00:04, 1496.21 examples/s]tokenizing (num_proc=8): 71%|███████▏ | 16790/23500 [00:11<00:04, 1664.04 examples/s]tokenizing (num_proc=8): 72%|███████▏ | 16972/23500 [00:11<00:03, 1703.02 examples/s]tokenizing (num_proc=8): 73%|███████▎ | 17155/23500 [00:11<00:03, 1684.34 examples/s]tokenizing (num_proc=8): 74%|███████▎ | 17325/23500 [00:11<00:03, 1660.20 examples/s]tokenizing (num_proc=8): 74%|███████▍ | 17494/23500 [00:11<00:03, 1646.05 examples/s]tokenizing (num_proc=8): 75%|███████▌ | 17661/23500 [00:11<00:03, 1640.22 examples/s]tokenizing (num_proc=8): 76%|███████▌ | 17857/23500 [00:11<00:03, 1719.45 examples/s]tokenizing (num_proc=8): 77%|███████▋ | 18093/23500 [00:11<00:02, 1887.91 examples/s]tokenizing (num_proc=8): 78%|███████▊ | 18284/23500 [00:11<00:02, 1869.45 examples/s]tokenizing (num_proc=8): 79%|███████▊ | 18481/23500 [00:12<00:02, 1873.78 examples/s]tokenizing (num_proc=8): 79%|███████▉ | 18672/23500 [00:12<00:02, 1864.89 examples/s]tokenizing (num_proc=8): 80%|████████ | 18864/23500 [00:12<00:02, 1799.66 examples/s]tokenizing (num_proc=8): 81%|████████ | 19061/23500 [00:12<00:02, 1839.68 examples/s]tokenizing (num_proc=8): 82%|████████▏ | 19256/23500 [00:12<00:02, 1774.05 examples/s]tokenizing (num_proc=8): 83%|████████▎ | 19441/23500 [00:12<00:02, 1794.71 examples/s]tokenizing (num_proc=8): 84%|████████▎ | 19643/23500 [00:12<00:02, 1854.12 examples/s]tokenizing (num_proc=8): 85%|████████▍ | 19866/23500 [00:12<00:01, 1950.55 examples/s]tokenizing (num_proc=8): 85%|████████▌ | 20064/23500 [00:12<00:01, 1915.38 examples/s]tokenizing (num_proc=8): 86%|████████▌ | 20265/23500 [00:12<00:01, 1921.94 examples/s]tokenizing (num_proc=8): 87%|████████▋ | 20461/23500 [00:13<00:01, 1883.27 examples/s]tokenizing (num_proc=8): 88%|████████▊ | 20664/23500 [00:13<00:01, 1905.08 examples/s]tokenizing (num_proc=8): 89%|████████▉ | 20863/23500 [00:13<00:01, 1885.52 examples/s]tokenizing (num_proc=8): 90%|████████▉ | 21058/23500 [00:13<00:01, 1838.83 examples/s]tokenizing (num_proc=8): 90%|█████████ | 21256/23500 [00:13<00:01, 1860.24 examples/s]tokenizing (num_proc=8): 91%|█████████▏| 21460/23500 [00:13<00:01, 1902.17 examples/s]tokenizing (num_proc=8): 92%|█████████▏| 21651/23500 [00:13<00:01, 1820.95 examples/s]tokenizing (num_proc=8): 93%|█████████▎| 21843/23500 [00:13<00:00, 1694.84 examples/s]tokenizing (num_proc=8): 94%|█████████▍| 22032/23500 [00:13<00:00, 1691.06 examples/s]tokenizing (num_proc=8): 95%|█████████▍| 22223/23500 [00:14<00:00, 1744.39 examples/s]tokenizing (num_proc=8): 95%|█████████▌| 22400/23500 [00:14<00:00, 1686.33 examples/s]tokenizing (num_proc=8): 96%|█████████▌| 22576/23500 [00:14<00:00, 1686.92 examples/s]tokenizing (num_proc=8): 97%|█████████▋| 22751/23500 [00:14<00:00, 1545.13 examples/s]tokenizing (num_proc=8): 98%|█████████▊| 22925/23500 [00:14<00:00, 1476.08 examples/s]tokenizing (num_proc=8): 98%|█████████▊| 23076/23500 [00:14<00:00, 1458.47 examples/s]tokenizing (num_proc=8): 99%|█████████▉| 23229/23500 [00:14<00:00, 1227.95 examples/s]tokenizing (num_proc=8): 99%|█████████▉| 23361/23500 [00:15<00:00, 1092.76 examples/s]tokenizing (num_proc=8): 100%|█████████▉| 23483/23500 [00:15<00:00, 814.35 examples/s] tokenizing (num_proc=8): 100%|██████████| 23500/23500 [00:15<00:00, 1491.48 examples/s]
Filter: 0%| | 0/23500 [00:00<?, ? examples/s]Filter: 4%|▍ | 1000/23500 [00:00<00:04, 4784.37 examples/s]Filter: 9%|▊ | 2000/23500 [00:00<00:03, 5445.31 examples/s]Filter: 13%|█▎ | 3000/23500 [00:00<00:03, 5588.24 examples/s]Filter: 17%|█▋ | 4000/23500 [00:00<00:03, 5908.47 examples/s]Filter: 21%|██▏ | 5000/23500 [00:00<00:03, 6121.82 examples/s]Filter: 26%|██▌ | 6000/23500 [00:01<00:02, 6368.29 examples/s]Filter: 30%|██▉ | 7000/23500 [00:01<00:02, 6473.33 examples/s]Filter: 34%|███▍ | 8000/23500 [00:01<00:02, 6452.18 examples/s]Filter: 38%|███▊ | 9000/23500 [00:01<00:02, 6569.25 examples/s]Filter: 43%|████▎ | 10000/23500 [00:01<00:02, 6589.54 examples/s]Filter: 47%|████▋ | 11000/23500 [00:01<00:01, 6655.93 examples/s]Filter: 51%|█████ | 12000/23500 [00:01<00:01, 6528.26 examples/s]Filter: 55%|█████▌ | 13000/23500 [00:02<00:01, 6549.41 examples/s]Filter: 60%|█████▉ | 14000/23500 [00:02<00:01, 6373.99 examples/s]Filter: 64%|██████▍ | 15000/23500 [00:02<00:01, 6471.53 examples/s]Filter: 68%|██████▊ | 16000/23500 [00:02<00:01, 6454.40 examples/s]Filter: 72%|███████▏ | 17000/23500 [00:02<00:01, 6447.25 examples/s]Filter: 77%|███████▋ | 18000/23500 [00:02<00:00, 6594.78 examples/s]Filter: 81%|████████ | 19000/23500 [00:02<00:00, 6550.96 examples/s]Filter: 85%|████████▌ | 20000/23500 [00:03<00:00, 6608.87 examples/s]Filter: 89%|████████▉ | 21000/23500 [00:03<00:00, 6603.36 examples/s]Filter: 94%|█████████▎| 22000/23500 [00:03<00:00, 6578.00 examples/s]Filter: 98%|█████████▊| 23000/23500 [00:03<00:00, 6622.20 examples/s]Filter: 100%|██████████| 23500/23500 [00:03<00:00, 6408.46 examples/s]
Filter: 0%| | 0/23500 [00:00<?, ? examples/s]Filter: 4%|▍ | 1000/23500 [00:00<00:03, 6425.12 examples/s]Filter: 9%|▊ | 2000/23500 [00:00<00:03, 6171.01 examples/s]Filter: 13%|█▎ | 3000/23500 [00:00<00:03, 5854.18 examples/s]Filter: 17%|█▋ | 4000/23500 [00:00<00:03, 5976.34 examples/s]Filter: 21%|██▏ | 5000/23500 [00:00<00:03, 5936.87 examples/s]Filter: 26%|██▌ | 6000/23500 [00:00<00:02, 6106.53 examples/s]Filter: 30%|██▉ | 7000/23500 [00:01<00:02, 6146.79 examples/s]Filter: 34%|███▍ | 8000/23500 [00:01<00:02, 5968.91 examples/s]Filter: 38%|███▊ | 9000/23500 [00:01<00:02, 6090.12 examples/s]Filter: 43%|████▎ | 10000/23500 [00:01<00:02, 5865.71 examples/s]Filter: 47%|████▋ | 11000/23500 [00:01<00:02, 6009.88 examples/s]Filter: 51%|█████ | 12000/23500 [00:01<00:01, 5981.25 examples/s]Filter: 55%|█████▌ | 13000/23500 [00:02<00:01, 6001.61 examples/s]Filter: 60%|█████▉ | 14000/23500 [00:02<00:01, 5983.39 examples/s]Filter: 64%|██████▍ | 15000/23500 [00:02<00:01, 5792.35 examples/s]Filter: 68%|██████▊ | 16000/23500 [00:02<00:01, 5833.27 examples/s]Filter: 72%|███████▏ | 17000/23500 [00:02<00:01, 5834.18 examples/s]Filter: 77%|███████▋ | 18000/23500 [00:03<00:00, 6012.67 examples/s]Filter: 81%|████████ | 19000/23500 [00:03<00:00, 5998.50 examples/s]Filter: 85%|████████▌ | 20000/23500 [00:03<00:00, 5932.09 examples/s]Filter: 89%|████████▉ | 21000/23500 [00:03<00:00, 6032.11 examples/s]Filter: 94%|█████████▎| 22000/23500 [00:03<00:00, 5791.11 examples/s]Filter: 98%|█████████▊| 23000/23500 [00:03<00:00, 5951.17 examples/s]Filter: 100%|██████████| 23500/23500 [00:03<00:00, 5960.52 examples/s]
supervised tokens: tool 4221981, replay 10741108 (71.8% replay)
replay capped to 25% of supervised tokens: kept 1406715 replay tokens, 1209 replay rows
rows: 23500 raw -> 15347 kept; by source: {'recovery': 3750, 'xlam': 5000, 'replay:compassion_12185_cleaned': 1209, 'apigen': 4888, 'efficiency': 500}
[manifest] git=534143b028a9765e904b4d6d1b22205027804f19 data_sha256={'train_olmo-compassion-web-s2_replay50.jsonl': 'b43d87068505', 'xlam5000_max1call.jsonl': 'a8aae6cf5c36'}
wandb: WARNING git root /workspace/anchor/olmo-compassion-web-s2/wandb does not exist
wandb: Tracking run with wandb version 0.30.0
wandb: W&B syncing is set to `offline` in this directory. Run `wandb online` or set WANDB_MODE=online to enable cloud syncing.
wandb: Run data is saved locally in /workspace/anchor/olmo-compassion-web-s2/wandb/wandb/offline-run-20261016_031236-b77g3dzm
wandb: View this run in the terminal with `wandb leet`
wandb: Detected [huggingface_hub.inference] in use.
wandb: Use W&B Weave for improved LLM call tracing. Install Weave with `pip install weave` then add `import weave` to the top of your script.
wandb: For more information, check out the docs at: https://weave-docs.wandb.ai
==((====))== Unsloth - 2x faster free finetuning | Num GPUs used = 1
\\ /| Num examples = 14,963 | Num Epochs = 1 | Total steps = 936
O^O/ \_/ \ Batch size per device = 2 | Gradient accumulation steps = 8
\ / Data Parallel GPUs = 1 | Total batch size (2 x 8 x 1) = 16
"-____-" Trainable parameters = 981,385,216 of 8,279,396,352 (11.85% trained)
Unsloth: Setting lr = 2.00e-05 instead of 2.00e-04 for embed_tokens.
Unsloth: Setting lr = 2.00e-05 instead of 2.00e-04 for lm_head.
0%| | 0/936 [00:00<?, ?it/s] 0%| | 1/936 [00:46<12:02:54, 46.39s/it] 0%| | 2/936 [01:08<8:23:04, 32.32s/it] 0%| | 3/936 [01:32<7:19:53, 28.29s/it] 0%| | 4/936 [01:59<7:11:13, 27.76s/it] 1%| | 5/936 [02:18<6:23:36, 24.72s/it] 1%| | 5/936 [02:18<6:23:36, 24.72s/it] 1%| | 6/936 [02:42<6:20:15, 24.53s/it] 1%| | 7/936 [03:06<6:17:27, 24.38s/it] 1%| | 8/936 [03:29<6:06:23, 23.69s/it] 1%| | 9/936 [03:54<6:12:09, 24.09s/it] 1%| | 10/936 [04:20<6:23:27, 24.85s/it] 1%| | 10/936 [04:20<6:23:27, 24.85s/it] 1%| | 11/936 [04:47<6:33:18, 25.51s/it] 1%|▏ | 12/936 [05:08<6:10:41, 24.07s/it] 1%|▏ | 13/936 [05:28<5:53:17, 22.97s/it] 1%|▏ | 14/936 [05:50<5:45:10, 22.46s/it] 2%|▏ | 15/936 [06:10<5:36:18, 21.91s/it] 2%|▏ | 15/936 [06:10<5:36:18, 21.91s/it] 2%|▏ | 16/936 [06:29<5:22:24, 21.03s/it] 2%|▏ | 17/936 [06:54<5:39:32, 22.17s/it] 2%|▏ | 18/936 [07:15<5:32:15, 21.72s/it] 2%|▏ | 19/936 [07:35<5:26:45, 21.38s/it] 2%|▏ | 20/936 [07:59<5:37:14, 22.09s/it] 2%|▏ | 20/936 [07:59<5:37:14, 22.09s/it] 2%|▏ | 21/936 [08:19<5:28:37, 21.55s/it] 2%|▏ | 22/936 [08:43<5:38:20, 22.21s/it] 2%|▏ | 23/936 [09:08<5:48:53, 22.93s/it] 3%|▎ | 24/936 [09:26<5:27:09, 21.52s/it] 3%|▎ | 25/936 [09:52<5:47:40, 22.90s/it] 3%|▎ | 25/936 [09:52<5:47:40, 22.90s/it] 3%|▎ | 26/936 [10:16<5:52:29, 23.24s/it] 3%|▎ | 27/936 [10:40<5:54:19, 23.39s/it] 3%|▎ | 28/936 [11:01<5:45:12, 22.81s/it] 3%|▎ | 29/936 [11:25<5:46:56, 22.95s/it] 3%|▎ | 30/936 [11:48<5:49:09, 23.12s/it] 3%|▎ | 30/936 [11:48<5:49:09, 23.12s/it] 3%|▎ | 31/936 [12:09<5:39:52, 22.53s/it] 3%|▎ | 32/936 [12:36<5:57:27, 23.72s/it] 4%|▎ | 33/936 [12:55<5:35:50, 22.31s/it] 4%|▎ | 34/936 [13:20<5:46:39, 23.06s/it] 4%|▎ | 35/936 [13:44<5:50:35, 23.35s/it] 4%|▎ | 35/936 [13:44<5:50:35, 23.35s/it] 4%|▍ | 36/936 [14:08<5:54:31, 23.64s/it] 4%|▍ | 37/936 [14:30<5:46:50, 23.15s/it] 4%|▍ | 38/936 [14:49<5:27:24, 21.88s/it] 4%|▍ | 39/936 [15:11<5:29:39, 22.05s/it] 4%|▍ | 40/936 [15:36<5:41:10, 22.85s/it] 4%|▍ | 40/936 [15:36<5:41:10, 22.85s/it] 4%|▍ | 41/936 [15:56<5:27:25, 21.95s/it] 4%|▍ | 42/936 [16:19<5:31:12, 22.23s/it] 5%|▍ | 43/936 [16:37<5:15:30, 21.20s/it] 5%|▍ | 44/936 [17:01<5:23:53, 21.79s/it] 5%|▍ | 45/936 [17:21<5:16:05, 21.29s/it] 5%|▍ | 45/936 [17:21<5:16:05, 21.29s/it] 5%|▍ | 46/936 [17:45<5:30:27, 22.28s/it] 5%|▌ | 47/936 [18:04<5:13:14, 21.14s/it] 5%|▌ | 48/936 [18:25<5:12:50, 21.14s/it] 5%|▌ | 49/936 [18:43<4:58:09, 20.17s/it] 5%|▌ | 50/936 [19:02<4:54:46, 19.96s/it] 5%|▌ | 50/936 [19:02<4:54:46, 19.96s/it]Unsloth: Will smartly offload gradients to save VRAM!
Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
{'loss': 1.1207, 'grad_norm': 1.163063645362854, 'learning_rate': 1.7021276595744682e-05, 'epoch': 0.01}
{'loss': 0.9711, 'grad_norm': 1.4067025184631348, 'learning_rate': 3.829787234042553e-05, 'epoch': 0.01}
{'loss': 0.8716, 'grad_norm': 0.8022639155387878, 'learning_rate': 5.9574468085106384e-05, 'epoch': 0.02}
{'loss': 0.6679, 'grad_norm': 1.227128267288208, 'learning_rate': 8.085106382978723e-05, 'epoch': 0.02}
{'loss': 0.6026, 'grad_norm': 1.3577823638916016, 'learning_rate': 0.00010212765957446809, 'epoch': 0.03}
{'loss': 0.7036, 'grad_norm': 1.6052050590515137, 'learning_rate': 0.00012340425531914893, 'epoch': 0.03}
{'loss': 0.5792, 'grad_norm': 1.4057849645614624, 'learning_rate': 0.0001446808510638298, 'epoch': 0.04}
{'loss': 0.5134, 'grad_norm': 1.3657870292663574, 'learning_rate': 0.00016595744680851065, 'epoch': 0.04}
{'loss': 0.5381, 'grad_norm': 1.0391675233840942, 'learning_rate': 0.0001872340425531915, 'epoch': 0.05}
{'loss': 0.5001, 'grad_norm': 0.9683766961097717, 'learning_rate': 0.00019999750239122662, 'epoch': 0.05}
Filter: 0%| | 0/384 [00:00<?, ? examples/s][A
Filter: 100%|██████████| 384/384 [00:00<00:00, 1437.93 examples/s][AFilter: 100%|██████████| 384/384 [00:00<00:00, 1425.00 examples/s]
0%| | 0/192 [00:00<?, ?it/s][A
1%| | 2/192 [00:00<01:32, 2.04it/s][A
2%|▏ | 3/192 [00:02<02:27, 1.28it/s][A
2%|▏ | 4/192 [00:03<02:33, 1.23it/s][A
3%|▎ | 5/192 [00:03<02:23, 1.30it/s][A
3%|▎ | 6/192 [00:03<01:51, 1.67it/s][A
4%|▎ | 7/192 [00:04<02:02, 1.51it/s][A
4%|▍ | 8/192 [00:05<02:05, 1.47it/s][A
5%|▍ | 9/192 [00:05<01:39, 1.84it/s][A
5%|▌ | 10/192 [00:06<02:09, 1.41it/s][A
6%|▌ | 11/192 [00:07<02:25, 1.24it/s][A
6%|▋ | 12/192 [00:08<02:34, 1.17it/s][A
7%|▋ | 13/192 [00:09<02:32, 1.17it/s][A
7%|▋ | 14/192 [00:09<02:01, 1.47it/s][A
8%|▊ | 15/192 [00:10<02:13, 1.33it/s][A
8%|▊ | 16/192 [00:11<02:05, 1.40it/s][A
9%|▉ | 17/192 [00:11<01:45, 1.66it/s][A
9%|▉ | 18/192 [00:13<02:15, 1.29it/s][A
10%|▉ | 19/192 [00:14<02:27, 1.17it/s][A
10%|█ | 20/192 [00:15<02:51, 1.00it/s][A
11%|█ | 21/192 [00:16<02:59, 1.05s/it][A
12%|█▏ | 23/192 [00:17<02:23, 1.18it/s][A
12%|█▎ | 24/192 [00:18<02:32, 1.10it/s][A
13%|█▎ | 25/192 [00:19<02:19, 1.20it/s][A
14%|█▎ | 26/192 [00:20<02:17, 1.21it/s][A
14%|█▍ | 27/192 [00:20<02:08, 1.29it/s][A
15%|█▍ | 28/192 [00:21<02:16, 1.20it/s][A
15%|█▌ | 29/192 [00:22<01:48, 1.50it/s][A
16%|█▌ | 30/192 [00:22<01:32, 1.76it/s][A
16%|█▌ | 31/192 [00:23<01:56, 1.38it/s][A
17%|█▋ | 32/192 [00:24<01:59, 1.34it/s][A
18%|█▊ | 34/192 [00:25<01:43, 1.52it/s][A
18%|█▊ | 35/192 [00:26<01:54, 1.38it/s][A
19%|█▉ | 36/192 [00:27<02:15, 1.15it/s][A
19%|█▉ | 37/192 [00:28<02:30, 1.03it/s][A
20%|█▉ | 38/192 [00:29<02:29, 1.03it/s][A
20%|██ | 39/192 [00:30<02:21, 1.08it/s][A
21%|██ | 40/192 [00:31<02:13, 1.13it/s][A
22%|██▏ | 42/192 [00:31<01:26, 1.74it/s][A
22%|██▏ | 43/192 [00:32<01:43, 1.45it/s][A
23%|██▎ | 44/192 [00:33<01:26, 1.70it/s][A
23%|██▎ | 45/192 [00:33<01:08, 2.15it/s][A
24%|██▍ | 46/192 [00:34<01:13, 1.99it/s][A
24%|██▍ | 47/192 [00:34<01:25, 1.69it/s][A
25%|██▌ | 48/192 [00:35<01:42, 1.41it/s][A
26%|██▌ | 49/192 [00:36<01:53, 1.26it/s][A
26%|██▌ | 50/192 [00:37<01:30, 1.57it/s][A
27%|██▋ | 51/192 [00:37<01:34, 1.50it/s][A
27%|██▋ | 52/192 [00:38<01:53, 1.23it/s][A
28%|██▊ | 53/192 [00:39<01:53, 1.22it/s][A
28%|██▊ | 54/192 [00:40<01:41, 1.35it/s][A
29%|██▉ | 56/192 [00:41<01:22, 1.65it/s][A
30%|███ | 58/192 [00:42<01:12, 1.84it/s][A
31%|███ | 59/192 [00:43<01:35, 1.39it/s][A
31%|███▏ | 60/192 [00:44<01:45, 1.25it/s][A
32%|███▏ | 61/192 [00:45<01:38, 1.32it/s][A
32%|███▏ | 62/192 [00:46<01:40, 1.29it/s][A
33%|███▎ | 63/192 [00:46<01:30, 1.42it/s][A
33%|███▎ | 64/192 [00:47<01:39, 1.28it/s][A
34%|███▍ | 65/192 [00:48<01:41, 1.26it/s][A
34%|███▍ | 66/192 [00:49<01:39, 1.27it/s][A
35%|███▍ | 67/192 [00:49<01:39, 1.25it/s][A
36%|███▌ | 69/192 [00:51<01:24, 1.45it/s][A
36%|███▋ | 70/192 [00:51<01:20, 1.51it/s][A
37%|███▋ | 71/192 [00:52<01:27, 1.38it/s][A
38%|███▊ | 73/192 [00:53<01:15, 1.58it/s][A
39%|███▊ | 74/192 [00:54<01:18, 1.51it/s][A
39%|███▉ | 75/192 [00:54<01:16, 1.53it/s][A
40%|███▉ | 76/192 [00:55<01:14, 1.56it/s][A
40%|████ | 77/192 [00:56<01:26, 1.33it/s][A
41%|████ | 78/192 [00:57<01:34, 1.20it/s][A
41%|████ | 79/192 [00:58<01:39, 1.13it/s][A
42%|████▏ | 80/192 [00:59<01:50, 1.02it/s][A
42%|████▏ | 81/192 [01:00<01:53, 1.02s/it][A
43%|████▎ | 82/192 [01:01<01:45, 1.04it/s][A
43%|████▎ | 83/192 [01:02<01:51, 1.02s/it][A
44%|████▍ | 84/192 [01:04<01:59, 1.10s/it][A
44%|████▍ | 85/192 [01:05<01:51, 1.04s/it][A
45%|████▍ | 86/192 [01:06<01:50, 1.04s/it][A
45%|████▌ | 87/192 [01:07<01:58, 1.12s/it][A
46%|████▌ | 88/192 [01:07<01:31, 1.14it/s][A
46%|████▋ | 89/192 [01:08<01:29, 1.15it/s][A
47%|████▋ | 90/192 [01:09<01:33, 1.09it/s][A
47%|████▋ | 91/192 [01:10<01:36, 1.05it/s][A
48%|████▊ | 92/192 [01:11<01:33, 1.07it/s][A
48%|████▊ | 93/192 [01:12<01:34, 1.04it/s][A
49%|████▉ | 94/192 [01:14<01:46, 1.08s/it][A
49%|████▉ | 95/192 [01:15<01:47, 1.11s/it][A
50%|█████ | 96/192 [01:15<01:24, 1.14it/s][A
51%|█████ | 97/192 [01:16<01:29, 1.06it/s][A
51%|█████ | 98/192 [01:16<01:10, 1.33it/s][A
52%|█████▏ | 99/192 [01:17<01:13, 1.27it/s][A
52%|█████▏ | 100/192 [01:18<01:19, 1.16it/s][A
53%|█████▎ | 101/192 [01:19<01:26, 1.06it/s][A
53%|█████▎ | 102/192 [01:21<01:32, 1.03s/it][A
54%|█████▎ | 103/192 [01:22<01:30, 1.02s/it][A
54%|█████▍ | 104/192 [01:22<01:08, 1.28it/s][A
55%|█████▍ | 105/192 [01:23<01:04, 1.36it/s][A
55%|█████▌ | 106/192 [01:24<01:12, 1.19it/s][A
56%|█████▌ | 107/192 [01:24<00:59, 1.43it/s][A
56%|█████▋ | 108/192 [01:25<01:01, 1.36it/s][A
57%|█████▋ | 109/192 [01:26<01:16, 1.08it/s][A
57%|█████▋ | 110/192 [01:27<01:21, 1.01it/s][A
58%|█████▊ | 111/192 [01:29<01:35, 1.18s/it][A
58%|█████▊ | 112/192 [01:29<01:14, 1.07it/s][A
59%|█████▉ | 113/192 [01:30<01:14, 1.06it/s][A
59%|█████▉ | 114/192 [01:32<01:22, 1.06s/it][A
60%|█████▉ | 115/192 [01:33<01:23, 1.08s/it][A
60%|██████ | 116/192 [01:33<01:03, 1.20it/s][A
61%|██████ | 117/192 [01:34<00:58, 1.28it/s][A
61%|██████▏ | 118/192 [01:35<01:02, 1.19it/s][A
62%|██████▏ | 119/192 [01:36<01:04, 1.13it/s][A
62%|██████▎ | 120/192 [01:37<01:06, 1.09it/s][A
63%|██████▎ | 121/192 [01:38<01:10, 1.01it/s][A
64%|██████▎ | 122/192 [01:39<01:10, 1.01s/it][A
64%|██████▍ | 123/192 [01:40<01:12, 1.05s/it][A
65%|██████▍ | 124/192 [01:41<01:14, 1.10s/it][A
65%|██████▌ | 125/192 [01:42<00:57, 1.16it/s][A
66%|██████▌ | 126/192 [01:42<00:55, 1.18it/s][A
66%|██████▌ | 127/192 [01:43<00:44, 1.45it/s][A
67%|██████▋ | 128/192 [01:44<00:50, 1.26it/s][A
67%|██████▋ | 129/192 [01:45<00:51, 1.22it/s][A
68%|██████▊ | 130/192 [01:46<00:54, 1.13it/s][A
68%|██████▊ | 131/192 [01:47<00:54, 1.12it/s][A
69%|██████▉ | 132/192 [01:47<00:51, 1.17it/s][A
69%|██████▉ | 133/192 [01:48<00:49, 1.20it/s][A
70%|██████▉ | 134/192 [01:49<00:50, 1.14it/s][A
70%|███████ | 135/192 [01:50<00:54, 1.04it/s][A
71%|███████ | 136/192 [01:51<00:53, 1.04it/s][A
71%|███████▏ | 137/192 [01:53<00:59, 1.09s/it][A
72%|███████▏ | 138/192 [01:54<00:57, 1.07s/it][A
72%|███████▏ | 139/192 [01:55<01:01, 1.16s/it][A
73%|███████▎ | 140/192 [01:55<00:46, 1.12it/s][A
73%|███████▎ | 141/192 [01:56<00:45, 1.13it/s][A
74%|███████▍ | 142/192 [01:57<00:41, 1.19it/s][A
74%|███████▍ | 143/192 [01:57<00:32, 1.51it/s][A
75%|███████▌ | 144/192 [01:58<00:33, 1.41it/s][A
76%|███████▌ | 145/192 [01:59<00:37, 1.26it/s][A
76%|███████▌ | 146/192 [02:00<00:42, 1.08it/s][A
77%|███████▋ | 147/192 [02:01<00:44, 1.02it/s][A
77%|███████▋ | 148/192 [02:02<00:41, 1.07it/s][A
78%|███████▊ | 149/192 [02:03<00:39, 1.09it/s][A
78%|███████▊ | 150/192 [02:04<00:37, 1.13it/s][A
79%|███████▊ | 151/192 [02:05<00:38, 1.06it/s][A
79%|███████▉ | 152/192 [02:05<00:29, 1.34it/s][A
80%|███████▉ | 153/192 [02:06<00:28, 1.39it/s][A
80%|████████ | 154/192 [02:07<00:28, 1.33it/s][A
81%|████████ | 155/192 [02:08<00:34, 1.07it/s][A
81%|████████▏ | 156/192 [02:09<00:37, 1.03s/it][A
82%|████████▏ | 157/192 [02:10<00:34, 1.02it/s][A
83%|████████▎ | 159/192 [02:10<00:18, 1.76it/s][A
83%|████████▎ | 160/192 [02:11<00:20, 1.58it/s][A
84%|████████▍ | 161/192 [02:12<00:22, 1.38it/s][A
84%|████████▍ | 162/192 [02:13<00:21, 1.39it/s][A
85%|████████▍ | 163/192 [02:13<00:20, 1.44it/s][A
85%|████████▌ | 164/192 [02:14<00:18, 1.51it/s][A
86%|████████▌ | 165/192 [02:15<00:19, 1.39it/s][A
86%|████████▋ | 166/192 [02:16<00:22, 1.17it/s][A
87%|████████▋ | 167/192 [02:17<00:22, 1.10it/s][A
88%|████████▊ | 168/192 [02:18<00:22, 1.07it/s][A
88%|████████▊ | 169/192 [02:19<00:24, 1.06s/it][A
89%|████████▊ | 170/192 [02:21<00:24, 1.12s/it][A
89%|████████▉ | 171/192 [02:22<00:22, 1.05s/it][A
90%|████████▉ | 172/192 [02:23<00:21, 1.07s/it][A
90%|█████████ | 173/192 [02:24<00:19, 1.02s/it][A
91%|█████████ | 174/192 [02:24<00:16, 1.07it/s][A
91%|█████████ | 175/192 [02:25<00:16, 1.05it/s][A
92%|█████████▏| 176/192 [02:26<00:14, 1.09it/s][A
92%|█████████▏| 177/192 [02:27<00:13, 1.11it/s][A
93%|█████████▎| 178/192 [02:28<00:13, 1.06it/s][A
93%|█████████▎| 179/192 [02:28<00:10, 1.26it/s][A
94%|█████████▍| 180/192 [02:29<00:09, 1.28it/s][A
94%|█████████▍| 181/192 [02:30<00:08, 1.25it/s][A
95%|█████████▍| 182/192 [02:31<00:07, 1.36it/s][A
96%|█████████▌| 184/192 [02:31<00:03, 2.30it/s][A
96%|█████████▋| 185/192 [02:32<00:04, 1.70it/s][A
97%|█████████▋| 186/192 [02:33<00:04, 1.43it/s][A
97%|█████████▋| 187/192 [02:33<00:03, 1.65it/s][A
98%|█████████▊| 188/192 [02:34<00:02, 1.66it/s][A
98%|█████████▊| 189/192 [02:35<00:02, 1.49it/s][A
99%|█████████▉| 190/192 [02:36<00:01, 1.27it/s][A
99%|█████████▉| 191/192 [02:36<00:00, 1.54it/s][A
100%|██████████| 192/192 [02:37<00:00, 1.26it/s][A
[A 5%|▌ | 50/936 [21:49<4:54:46, 19.96s/it]
100%|██████████| 192/192 [02:38<00:00, 1.26it/s][A
[A 5%|▌ | 51/936 [22:10<17:15:42, 70.22s/it] 6%|▌ | 52/936 [22:29<13:29:19, 54.93s/it] 6%|▌ | 53/936 [22:46<10:40:51, 43.55s/it] 6%|▌ | 54/936 [23:05<8:50:53, 36.11s/it] 6%|▌ | 55/936 [23:22<7:26:33, 30.41s/it] 6%|▌ | 55/936 [23:22<7:26:33, 30.41s/it] 6%|▌ | 56/936 [23:46<6:57:30, 28.47s/it] 6%|▌ | 57/936 [24:02<6:01:13, 24.66s/it] 6%|▌ | 58/936 [24:25<5:54:12, 24.21s/it] 6%|▋ | 59/936 [24:46<5:38:51, 23.18s/it] 6%|▋ | 60/936 [25:05<5:21:52, 22.05s/it] 6%|▋ | 60/936 [25:05<5:21:52, 22.05s/it] 7%|▋ | 61/936 [25:26<5:14:45, 21.58s/it] 7%|▋ | 62/936 [25:45<5:06:31, 21.04s/it] 7%|▋ | 63/936 [26:06<5:03:12, 20.84s/it] 7%|▋ | 64/936 [26:29<5:13:52, 21.60s/it] 7%|▋ | 65/936 [26:49<5:07:25, 21.18s/it] 7%|▋ | 65/936 [26:49<5:07:25, 21.18s/it] 7%|▋ | 66/936 [27:17<5:35:09, 23.11s/it] 7%|▋ | 67/936 [27:39<5:31:43, 22.90s/it] 7%|▋ | 68/936 [28:02<5:29:26, 22.77s/it] 7%|▋ | 69/936 [28:26<5:34:11, 23.13s/it] 7%|▋ | 70/936 [28:49<5:34:49, 23.20s/it] 7%|▋ | 70/936 [28:49<5:34:49, 23.20s/it] 8%|▊ | 71/936 [29:09<5:20:17, 22.22s/it] 8%|▊ | 72/936 [29:32<5:22:25, 22.39s/it] 8%|▊ | 73/936 [29:49<4:59:39, 20.83s/it] 8%|▊ | 74/936 [30:13<5:14:05, 21.86s/it] 8%|▊ | 75/936 [30:34<5:09:32, 21.57s/it] 8%|▊ | 75/936 [30:34<5:09:32, 21.57s/it] 8%|▊ | 76/936 [30:54<5:02:33, 21.11s/it] 8%|▊ | 77/936 [31:21<5:26:46, 22.82s/it] 8%|▊ | 78/936 [31:41<5:12:51, 21.88s/it] 8%|▊ | 79/936 [32:01<5:03:49, 21.27s/it] 9%|▊ | 80/936 [32:20<4:54:14, 20.62s/it] 9%|▊ | 80/936 [32:20<4:54:14, 20.62s/it] 9%|▊ | 81/936 [32:42<4:59:49, 21.04s/it] 9%|▉ | 82/936 [33:03<5:00:33, 21.12s/it] 9%|▉ | 83/936 [33:26<5:08:15, 21.68s/it] 9%|▉ | 84/936 [33:49<5:12:06, 21.98s/it] 9%|▉ | 85/936 [34:08<4:58:41, 21.06s/it] 9%|▉ | 85/936 [34:08<4:58:41, 21.06s/it] 9%|▉ | 86/936 [34:28<4:56:51, 20.95s/it] 9%|▉ | 87/936 [34:47<4:45:54, 20.21s/it] 9%|▉ | 88/936 [35:04<4:32:27, 19.28s/it] 10%|▉ | 89/936 [35:28<4:53:49, 20.81s/it] 10%|▉ | 90/936 [35:48<4:48:26, 20.46s/it] 10%|▉ | 90/936 [35:48<4:48:26, 20.46s/it] 10%|▉ | 91/936 [36:14<5:11:28, 22.12s/it] 10%|▉ | 92/936 [36:37<5:15:38, 22.44s/it] 10%|▉ | 93/936 [37:06<5:43:31, 24.45s/it] 10%|█ | 94/936 [37:27<5:28:57, 23.44s/it] 10%|█ | 95/936 [37:49<5:22:42, 23.02s/it] 10%|█ | 95/936 [37:49<5:22:42, 23.02s/it] 10%|█ | 96/936 [38:09<5:07:35, 21.97s/it] 10%|█ | 97/936 [38:23<4:36:31, 19.78s/it] 10%|█ | 98/936 [38:46<4:47:12, 20.56s/it] 11%|█ | 99/936 [39:01<4:22:05, 18.79s/it] 11%|█ | 100/936 [39:20<4:22:38, 18.85s/it] 11%|█ | 100/936 [39:20<4:22:38, 18.85s/it]{'eval_loss': 0.4231555461883545, 'eval_runtime': 166.2485, 'eval_samples_per_second': 2.31, 'eval_steps_per_second': 1.155, 'epoch': 0.05}
{'loss': 0.6182, 'grad_norm': 1.1648342609405518, 'learning_rate': 0.0001999694057253083, 'epoch': 0.06}
{'loss': 0.4698, 'grad_norm': 1.2007864713668823, 'learning_rate': 0.00019991009918336413, 'epoch': 0.06}
{'loss': 0.3948, 'grad_norm': 0.8219057321548462, 'learning_rate': 0.00019981960128055687, 'epoch': 0.07}
{'loss': 0.5086, 'grad_norm': 0.9363082647323608, 'learning_rate': 0.0001996979402698136, 'epoch': 0.07}
{'loss': 0.4872, 'grad_norm': 0.6071483492851257, 'learning_rate': 0.00019954515413300516, 'epoch': 0.08}
{'loss': 0.6249, 'grad_norm': 0.6483224034309387, 'learning_rate': 0.00019936129056908865, 'epoch': 0.09}
{'loss': 0.4704, 'grad_norm': 0.6583413481712341, 'learning_rate': 0.00019914640697921585, 'epoch': 0.09}
{'loss': 0.5516, 'grad_norm': 0.5416878461837769, 'learning_rate': 0.00019890057044881306, 'epoch': 0.1}
{'loss': 0.4088, 'grad_norm': 0.656437873840332, 'learning_rate': 0.0001986238577266374, 'epoch': 0.1}
{'loss': 0.4826, 'grad_norm': 0.6249133944511414, 'learning_rate': 0.00019831635520081625, 'epoch': 0.11}
0%| | 0/192 [00:00<?, ?it/s][A
1%| | 2/192 [00:00<00:47, 4.03it/s][A
2%|▏ | 3/192 [00:01<01:21, 2.32it/s][A
2%|▏ | 4/192 [00:01<01:15, 2.48it/s][A
3%|▎ | 5/192 [00:01<00:58, 3.19it/s][A
3%|▎ | 6/192 [00:01<00:55, 3.36it/s][A
4%|▎ | 7/192 [00:02<00:53, 3.43it/s][A
4%|▍ | 8/192 [00:02<00:48, 3.82it/s][A
5%|▍ | 9/192 [00:02<00:46, 3.90it/s][A
5%|▌ | 10/192 [00:03<00:52, 3.49it/s][A
6%|▌ | 11/192 [00:03<01:03, 2.87it/s][A
6%|▋ | 12/192 [00:03<00:55, 3.23it/s][A
7%|▋ | 13/192 [00:04<00:57, 3.13it/s][A
7%|▋ | 14/192 [00:04<00:54, 3.24it/s][A
8%|▊ | 15/192 [00:04<00:45, 3.90it/s][A
9%|▉ | 17/192 [00:04<00:42, 4.10it/s][A
9%|▉ | 18/192 [00:05<00:50, 3.45it/s][A
10%|▉ | 19/192 [00:05<01:01, 2.83it/s][A
10%|█ | 20/192 [00:06<01:11, 2.40it/s][A
11%|█ | 21/192 [00:06<01:11, 2.41it/s][A
12%|█▏ | 23/192 [00:07<00:57, 2.93it/s][A
12%|█▎ | 24/192 [00:07<00:57, 2.93it/s][A
13%|█▎ | 25/192 [00:07<00:47, 3.52it/s][A
14%|█▍ | 27/192 [00:08<00:33, 4.97it/s][A
15%|█▍ | 28/192 [00:08<00:34, 4.74it/s][A
15%|█▌ | 29/192 [00:08<00:37, 4.29it/s][A
16%|█▌ | 30/192 [00:08<00:42, 3.80it/s][A
16%|█▌ | 31/192 [00:09<00:45, 3.54it/s][A
17%|█▋ | 33/192 [00:09<00:30, 5.16it/s][A
18%|█▊ | 34/192 [00:09<00:33, 4.69it/s][A
18%|█▊ | 35/192 [00:10<00:42, 3.69it/s][A
19%|█▉ | 36/192 [00:10<00:53, 2.92it/s][A
19%|█▉ | 37/192 [00:11<01:03, 2.45it/s][A
20%|█▉ | 38/192 [00:11<00:55, 2.79it/s][A
20%|██ | 39/192 [00:11<00:52, 2.92it/s][A
21%|██ | 40/192 [00:12<00:48, 3.12it/s][A
21%|██▏ | 41/192 [00:12<00:38, 3.89it/s][A
22%|██▏ | 42/192 [00:12<00:40, 3.67it/s][A
22%|██▏ | 43/192 [00:13<00:53, 2.81it/s][A
23%|██▎ | 44/192 [00:13<00:49, 2.97it/s][A
23%|██▎ | 45/192 [00:13<00:40, 3.61it/s][A
24%|██▍ | 47/192 [00:13<00:27, 5.35it/s][A
25%|██▌ | 48/192 [00:13<00:30, 4.78it/s][A
26%|██▌ | 49/192 [00:14<00:40, 3.56it/s][A
26%|██▌ | 50/192 [00:14<00:39, 3.61it/s][A
27%|██▋ | 51/192 [00:14<00:37, 3.77it/s][A
27%|██▋ | 52/192 [00:15<00:43, 3.23it/s][A
28%|██▊ | 53/192 [00:15<00:43, 3.17it/s][A
29%|██▊ | 55/192 [00:15<00:28, 4.88it/s][A
30%|██▉ | 57/192 [00:15<00:21, 6.37it/s][A
30%|███ | 58/192 [00:16<00:26, 5.14it/s][A
31%|███ | 59/192 [00:16<00:38, 3.42it/s][A
31%|███▏ | 60/192 [00:17<00:39, 3.31it/s][A
32%|███▏ | 61/192 [00:17<00:32, 3.98it/s][A
32%|███▏ | 62/192 [00:17<00:35, 3.67it/s][A
33%|███▎ | 63/192 [00:18<00:44, 2.93it/s][A
33%|███▎ | 64/192 [00:18<00:39, 3.23it/s][A
34%|███▍ | 66/192 [00:18<00:32, 3.93it/s][A
35%|███▍ | 67/192 [00:19<00:33, 3.73it/s][A
36%|███▌ | 69/192 [00:19<00:29, 4.19it/s][A
36%|███▋ | 70/192 [00:20<00:38, 3.19it/s][A
37%|███▋ | 71/192 [00:20<00:41, 2.95it/s][A
38%|███▊ | 73/192 [00:20<00:34, 3.42it/s][A
39%|███▊ | 74/192 [00:21<00:34, 3.39it/s][A
39%|███▉ | 75/192 [00:21<00:29, 4.01it/s][A
40%|████ | 77/192 [00:21<00:26, 4.33it/s][A
41%|████ | 78/192 [00:22<00:34, 3.34it/s][A
41%|████ | 79/192 [00:22<00:39, 2.86it/s][A
42%|████▏ | 80/192 [00:23<00:43, 2.58it/s][A
42%|████▏ | 81/192 [00:23<00:42, 2.59it/s][A
43%|████▎ | 83/192 [00:24<00:35, 3.08it/s][A
44%|████▍ | 84/192 [00:24<00:41, 2.61it/s][A
44%|████▍ | 85/192 [00:25<00:40, 2.62it/s][A
45%|████▍ | 86/192 [00:25<00:38, 2.75it/s][A
45%|████▌ | 87/192 [00:25<00:44, 2.37it/s][A
46%|████▌ | 88/192 [00:26<00:40, 2.57it/s][A
46%|████▋ | 89/192 [00:26<00:38, 2.65it/s][A
47%|████▋ | 90/192 [00:26<00:35, 2.85it/s][A
47%|████▋ | 91/192 [00:27<00:33, 2.98it/s][A
48%|████▊ | 92/192 [00:27<00:34, 2.88it/s][A
48%|████▊ | 93/192 [00:28<00:39, 2.53it/s][A
49%|████▉ | 94/192 [00:28<00:45, 2.14it/s][A
49%|████▉ | 95/192 [00:29<00:44, 2.16it/s][A
50%|█████ | 96/192 [00:29<00:40, 2.35it/s][A
51%|█████ | 97/192 [00:30<00:44, 2.12it/s][A
51%|█████ | 98/192 [00:30<00:39, 2.37it/s][A
52%|█████▏ | 99/192 [00:30<00:37, 2.45it/s][A
52%|█████▏ | 100/192 [00:31<00:40, 2.26it/s][A
53%|█████▎ | 101/192 [00:31<00:45, 1.99it/s][A
53%|█████▎ | 102/192 [00:32<00:44, 2.00it/s][A
54%|█████▎ | 103/192 [00:32<00:44, 2.00it/s][A
54%|█████▍ | 104/192 [00:33<00:36, 2.40it/s][A
55%|█████▍ | 105/192 [00:33<00:28, 3.06it/s][A
55%|█████▌ | 106/192 [00:33<00:28, 2.99it/s][A
56%|█████▌ | 107/192 [00:34<00:29, 2.92it/s][A
56%|█████▋ | 108/192 [00:34<00:27, 3.02it/s][A
57%|█████▋ | 109/192 [00:34<00:35, 2.37it/s][A
57%|█████▋ | 110/192 [00:35<00:40, 2.03it/s][A
58%|█████▊ | 111/192 [00:36<00:48, 1.66it/s][A
58%|█████▊ | 112/192 [00:36<00:43, 1.84it/s][A
59%|█████▉ | 113/192 [00:37<00:40, 1.95it/s][A
59%|█████▉ | 114/192 [00:37<00:42, 1.84it/s][A
60%|█████▉ | 115/192 [00:38<00:43, 1.77it/s][A
60%|██████ | 116/192 [00:38<00:36, 2.11it/s][A
61%|██████ | 117/192 [00:38<00:27, 2.68it/s][A
61%|██████▏ | 118/192 [00:39<00:24, 2.97it/s][A
62%|██████▏ | 119/192 [00:39<00:27, 2.65it/s][A
62%|██████▎ | 120/192 [00:39<00:24, 2.88it/s][A
63%|██████▎ | 121/192 [00:40<00:26, 2.70it/s][A
64%|██████▎ | 122/192 [00:40<00:25, 2.79it/s][A
64%|██████▍ | 123/192 [00:41<00:27, 2.48it/s][A
65%|██████▍ | 124/192 [00:41<00:27, 2.49it/s][A
65%|██████▌ | 125/192 [00:41<00:24, 2.68it/s][A
66%|██████▌ | 127/192 [00:42<00:19, 3.39it/s][A
67%|██████▋ | 128/192 [00:42<00:18, 3.40it/s][A
67%|██████▋ | 129/192 [00:42<00:19, 3.19it/s][A
68%|██████▊ | 130/192 [00:43<00:23, 2.67it/s][A
68%|██████▊ | 131/192 [00:43<00:23, 2.61it/s][A
69%|██████▉ | 132/192 [00:44<00:20, 2.90it/s][A
69%|██████▉ | 133/192 [00:44<00:19, 3.07it/s][A
70%|██████▉ | 134/192 [00:44<00:17, 3.35it/s][A
70%|███████ | 135/192 [00:45<00:19, 2.95it/s][A
71%|███████ | 136/192 [00:45<00:21, 2.66it/s][A
71%|███████▏ | 137/192 [00:46<00:24, 2.21it/s][A
72%|███████▏ | 138/192 [00:46<00:25, 2.08it/s][A
72%|███████▏ | 139/192 [00:47<00:27, 1.92it/s][A
73%|███████▎ | 140/192 [00:47<00:23, 2.21it/s][A
73%|███████▎ | 141/192 [00:47<00:18, 2.81it/s][A
74%|███████▍ | 142/192 [00:48<00:15, 3.19it/s][A
74%|███████▍ | 143/192 [00:48<00:14, 3.35it/s][A
75%|███████▌ | 144/192 [00:48<00:14, 3.35it/s][A
76%|███████▌ | 145/192 [00:49<00:16, 2.82it/s][A
76%|███████▌ | 146/192 [00:49<00:18, 2.51it/s][A
77%|███████▋ | 147/192 [00:49<00:17, 2.55it/s][A
77%|███████▋ | 148/192 [00:50<00:16, 2.67it/s][A
78%|███████▊ | 149/192 [00:50<00:15, 2.72it/s][A
78%|███████▊ | 150/192 [00:50<00:14, 2.86it/s][A
79%|███████▊ | 151/192 [00:51<00:16, 2.41it/s][A
79%|███████▉ | 152/192 [00:51<00:15, 2.65it/s][A
80%|███████▉ | 153/192 [00:51<00:11, 3.26it/s][A
80%|████████ | 154/192 [00:52<00:11, 3.22it/s][A
81%|████████ | 155/192 [00:52<00:14, 2.47it/s][A
81%|████████▏ | 156/192 [00:53<00:15, 2.26it/s][A
82%|████████▏ | 157/192 [00:53<00:14, 2.42it/s][A
83%|████████▎ | 159/192 [00:53<00:08, 3.79it/s][A
83%|████████▎ | 160/192 [00:54<00:08, 3.57it/s][A
84%|████████▍ | 161/192 [00:54<00:10, 3.02it/s][A
84%|████████▍ | 162/192 [00:54<00:09, 3.32it/s][A
85%|████████▍ | 163/192 [00:55<00:07, 4.05it/s][A
86%|████████▌ | 165/192 [00:55<00:06, 4.33it/s][A
86%|████████▋ | 166/192 [00:55<00:07, 3.53it/s][A
87%|████████▋ | 167/192 [00:56<00:07, 3.37it/s][A
88%|████████▊ | 168/192 [00:56<00:06, 3.54it/s][A
88%|████████▊ | 169/192 [00:57<00:08, 2.63it/s][A
89%|████████▊ | 170/192 [00:57<00:09, 2.35it/s][A
89%|████████▉ | 171/192 [00:57<00:08, 2.53it/s][A
90%|████████▉ | 172/192 [00:58<00:07, 2.55it/s][A
90%|█████████ | 173/192 [00:58<00:07, 2.57it/s][A
91%|█████████ | 174/192 [00:58<00:06, 2.94it/s][A
91%|█████████ | 175/192 [00:59<00:06, 2.57it/s][A
92%|█████████▏| 176/192 [00:59<00:05, 2.72it/s][A
92%|█████████▏| 177/192 [00:59<00:04, 3.39it/s][A
93%|█████████▎| 178/192 [01:00<00:04, 3.40it/s][A
93%|█████████▎| 179/192 [01:00<00:04, 2.89it/s][A
94%|█████████▍| 180/192 [01:00<00:03, 3.20it/s][A
95%|█████████▍| 182/192 [01:01<00:02, 4.81it/s][A
96%|█████████▌| 184/192 [01:01<00:01, 6.29it/s][A
96%|█████████▋| 185/192 [01:01<00:01, 5.14it/s][A
97%|█████████▋| 186/192 [01:02<00:01, 3.68it/s][A
97%|█████████▋| 187/192 [01:02<00:01, 3.40it/s][A
98%|█████████▊| 189/192 [01:02<00:00, 4.80it/s][A
99%|█████████▉| 190/192 [01:02<00:00, 4.23it/s][A
99%|█████████▉| 191/192 [01:03<00:00, 3.87it/s][A
100%|██████████| 192/192 [01:03<00:00, 3.38it/s][A
[A 11%|█ | 100/936 [40:24<4:22:38, 18.85s/it]
100%|██████████| 192/192 [01:04<00:00, 3.38it/s][A
[Awandb: WARNING URL not available in offline run
`rope_scaling`'s beta_fast field must be a float, got 32
`rope_scaling`'s beta_slow field must be a float, got 1
11%|█ | 101/936 [40:59<9:57:54, 42.96s/it] 11%|█ | 102/936 [41:20<8:27:12, 36.49s/it] 11%|█ | 103/936 [41:44<7:32:35, 32.60s/it] 11%|█ | 104/936 [42:53<10:03:00, 43.49s/it] 11%|█ | 105/936 [43:02<7:39:55, 33.21s/it] 11%|█ | 105/936 [43:02<7:39:55, 33.21s/it] 11%|█▏ | 106/936 [43:20<6:39:21, 28.87s/it] 11%|█▏ | 107/936 [43:38<5:50:38, 25.38s/it] 12%|█▏ | 108/936 [43:55<5:17:57, 23.04s/it] 12%|█▏ | 109/936 [44:14<4:59:14, 21.71s/it] 12%|█▏ | 110/936 [44:38<5:08:26, 22.41s/it] 12%|█▏ | 110/936 [44:38<5:08:26, 22.41s/it] 12%|█▏ | 111/936 [45:05<5:27:32, 23.82s/it] 12%|█▏ | 112/936 [45:45<6:33:11, 28.63s/it] 12%|█▏ | 113/936 [46:09<6:15:13, 27.36s/it] 12%|█▏ | 114/936 [46:42<6:37:24, 29.01s/it] 12%|█▏ | 115/936 [47:13<6:43:24, 29.48s/it] 12%|█▏ | 115/936 [47:13<6:43:24, 29.48s/it] 12%|█▏ | 116/936 [47:46<6:57:32, 30.55s/it] 12%|█▎ | 117/936 [48:13<6:42:02, 29.45s/it] 13%|█▎ | 118/936 [48:39<6:29:33, 28.57s/it] 13%|█▎ | 119/936 [48:54<5:33:30, 24.49s/it] 13%|█▎ | 120/936 [49:13<5:11:02, 22.87s/it] 13%|█▎ | 120/936 [49:13<5:11:02, 22.87s/it] 13%|█▎ | 121/936 [49:33<4:55:51, 21.78s/it] 13%|█▎ | 122/936 [49:53<4:48:47, 21.29s/it] 13%|█▎ | 123/936 [50:13<4:45:39, 21.08s/it] 13%|█▎ | 124/936 [50:34<4:45:09, 21.07s/it] 13%|█▎ | 125/936 [50:54<4:38:40, 20.62s/it] 13%|█▎ | 125/936 [50:54<4:38:40, 20.62s/it] 13%|█▎ | 126/936 [51:10<4:19:17, 19.21s/it] 14%|█▎ | 127/936 [51:25<4:03:03, 18.03s/it] 14%|█▎ | 128/936 [51:38<3:43:29, 16.60s/it] 14%|█▍ | 129/936 [51:58<3:57:09, 17.63s/it] 14%|█▍ | 130/936 [52:15<3:52:21, 17.30s/it] 14%|█▍ | 130/936 [52:15<3:52:21, 17.30s/it] 14%|█▍ | 131/936 [52:37<4:10:38, 18.68s/it] 14%|█▍ | 132/936 [52:52<3:55:58, 17.61s/it] 14%|█▍ | 133/936 [53:12<4:04:11, 18.25s/it] 14%|█▍ | 134/936 [53:28<3:56:48, 17.72s/it] 14%|█▍ | 135/936 [53:41<3:37:18, 16.28s/it] 14%|█▍ | 135/936 [53:41<3:37:18, 16.28s/it] 15%|█▍ | 136/936 [53:55<3:29:31, 15.71s/it] 15%|█▍ | 137/936 [54:12<3:31:28, 15.88s/it] 15%|█▍ | 138/936 [54:36<4:03:20, 18.30s/it] 15%|█▍ | 139/936 [54:50<3:46:00, 17.01s/it] 15%|█▍ | 140/936 [55:06<3:43:01, 16.81s/it] 15%|█▍ | 140/936 [55:06<3:43:01, 16.81s/it] 15%|█▌ | 141/936 [55:25<3:52:45, 17.57s/it] 15%|█▌ | 142/936 [55:40<3:42:15, 16.80s/it] 15%|█▌ | 143/936 [56:02<4:02:42, 18.36s/it] 15%|█▌ | 144/936 [56:22<4:07:53, 18.78s/it] 15%|█▌ | 145/936 [56:37<3:53:56, 17.75s/it] 15%|█▌ | 145/936 [56:37<3:53:56, 17.75s/it] 16%|█▌ | 146/936 [56:48<3:25:32, 15.61s/it] 16%|█▌ | 147/936 [56:59<3:08:20, 14.32s/it] 16%|█▌ | 148/936 [57:14<3:10:59, 14.54s/it] 16%|█▌ | 149/936 [57:35<3:35:38, 16.44s/it] 16%|█▌ | 150/936 [57:57<3:57:33, 18.13s/it] 16%|█▌ | 150/936 [57:57<3:57:33, 18.13s/it]{'eval_loss': 0.3424660265445709, 'eval_runtime': 64.3443, 'eval_samples_per_second': 5.968, 'eval_steps_per_second': 2.984, 'epoch': 0.11}
{'loss': 0.4567, 'grad_norm': 0.601626455783844, 'learning_rate': 0.00019797815887187726, 'epoch': 0.11}
{'loss': 0.4607, 'grad_norm': 0.4807310104370117, 'learning_rate': 0.00019760937432277786, 'epoch': 0.12}
{'loss': 0.5798, 'grad_norm': 0.5414025783538818, 'learning_rate': 0.0001972101166859425, 'epoch': 0.12}
{'loss': 0.3468, 'grad_norm': 0.46784576773643494, 'learning_rate': 0.0001967805106073192, 'epoch': 0.13}
{'loss': 0.4652, 'grad_norm': 0.5162056088447571, 'learning_rate': 0.00019632069020746572, 'epoch': 0.13}
{'loss': 0.5919, 'grad_norm': 0.6472817063331604, 'learning_rate': 0.00019583079903967785, 'epoch': 0.14}
{'loss': 0.5115, 'grad_norm': 0.692052960395813, 'learning_rate': 0.000195310990045173, 'epoch': 0.14}
{'loss': 0.3182, 'grad_norm': 0.44978657364845276, 'learning_rate': 0.0001947614255053428, 'epoch': 0.15}
{'loss': 0.4794, 'grad_norm': 0.5151785016059875, 'learning_rate': 0.00019418227699108956, 'epoch': 0.16}
{'loss': 0.5511, 'grad_norm': 0.5038560628890991, 'learning_rate': 0.00019357372530926322, 'epoch': 0.16}
0%| | 0/192 [00:00<?, ?it/s][A
1%| | 2/192 [00:00<00:47, 4.04it/s][A
2%|▏ | 3/192 [00:01<01:20, 2.34it/s][A
2%|▏ | 4/192 [00:01<01:15, 2.50it/s][A
3%|▎ | 5/192 [00:01<00:57, 3.24it/s][A
3%|▎ | 6/192 [00:01<00:55, 3.37it/s][A
4%|▎ | 7/192 [00:02<00:53, 3.46it/s][A
4%|▍ | 8/192 [00:02<00:47, 3.84it/s][A
5%|▍ | 9/192 [00:02<00:46, 3.93it/s][A
5%|▌ | 10/192 [00:03<00:52, 3.50it/s][A
6%|▌ | 11/192 [00:03<01:03, 2.87it/s][A
6%|▋ | 12/192 [00:03<00:55, 3.24it/s][A
7%|▋ | 13/192 [00:04<00:57, 3.14it/s][A
7%|▋ | 14/192 [00:04<00:55, 3.23it/s][A
8%|▊ | 15/192 [00:04<00:45, 3.92it/s][A
9%|▉ | 17/192 [00:04<00:42, 4.15it/s][A
9%|▉ | 18/192 [00:05<00:50, 3.45it/s][A
10%|▉ | 19/192 [00:05<01:00, 2.84it/s][A
10%|█ | 20/192 [00:06<01:11, 2.40it/s][A
11%|█ | 21/192 [00:06<01:11, 2.40it/s][A
12%|█▏ | 23/192 [00:07<00:57, 2.94it/s][A
12%|█▎ | 24/192 [00:07<00:57, 2.95it/s][A
13%|█▎ | 25/192 [00:07<00:47, 3.55it/s][A
14%|█▍ | 27/192 [00:08<00:32, 5.01it/s][A
15%|█▍ | 28/192 [00:08<00:34, 4.74it/s][A
15%|█▌ | 29/192 [00:08<00:36, 4.48it/s][A
16%|█▌ | 30/192 [00:08<00:40, 3.96it/s][A
16%|█▌ | 31/192 [00:09<00:45, 3.58it/s][A
17%|█▋ | 33/192 [00:09<00:30, 5.23it/s][A
18%|█▊ | 34/192 [00:09<00:33, 4.71it/s][A
18%|█▊ | 35/192 [00:10<00:42, 3.70it/s][A
19%|█▉ | 36/192 [00:10<00:52, 2.95it/s][A
19%|█▉ | 37/192 [00:11<01:03, 2.46it/s][A
20%|█▉ | 38/192 [00:11<00:55, 2.80it/s][A
20%|██ | 39/192 [00:11<00:52, 2.90it/s][A
21%|██ | 40/192 [00:11<00:48, 3.13it/s][A
21%|██▏ | 41/192 [00:12<00:39, 3.85it/s][A
22%|██▏ | 42/192 [00:12<00:41, 3.65it/s][A
22%|██▏ | 43/192 [00:13<00:54, 2.74it/s][A
23%|██▎ | 44/192 [00:13<00:49, 3.00it/s][A
23%|██▎ | 45/192 [00:13<00:40, 3.66it/s][A
24%|██▍ | 47/192 [00:13<00:26, 5.44it/s][A
25%|██▌ | 48/192 [00:13<00:30, 4.79it/s][A
26%|██▌ | 49/192 [00:14<00:40, 3.57it/s][A
26%|██▌ | 50/192 [00:14<00:39, 3.62it/s][A
27%|██▋ | 51/192 [00:14<00:37, 3.73it/s][A
27%|██▋ | 52/192 [00:15<00:43, 3.24it/s][A
28%|██▊ | 53/192 [00:15<00:43, 3.17it/s][A
29%|██▊ | 55/192 [00:15<00:28, 4.88it/s][A
30%|██▉ | 57/192 [00:15<00:21, 6.36it/s][A
30%|███ | 58/192 [00:16<00:25, 5.17it/s][A
31%|███ | 59/192 [00:16<00:38, 3.42it/s][A
31%|███▏ | 60/192 [00:17<00:39, 3.31it/s][A
32%|███▏ | 61/192 [00:17<00:32, 3.98it/s][A
32%|███▏ | 62/192 [00:17<00:35, 3.67it/s][A
33%|███▎ | 63/192 [00:18<00:44, 2.93it/s][A
33%|███▎ | 64/192 [00:18<00:39, 3.24it/s][A
34%|███▍ | 66/192 [00:18<00:32, 3.93it/s][A
35%|███▍ | 67/192 [00:19<00:33, 3.72it/s][A
36%|███▌ | 69/192 [00:19<00:29, 4.18it/s][A
36%|███▋ | 70/192 [00:19<00:37, 3.21it/s][A
37%|███▋ | 71/192 [00:20<00:40, 2.97it/s][A
38%|███▊ | 73/192 [00:20<00:34, 3.45it/s][A
39%|███▊ | 74/192 [00:21<00:34, 3.41it/s][A
39%|███▉ | 75/192 [00:21<00:28, 4.04it/s][A
40%|████ | 77/192 [00:21<00:26, 4.37it/s][A
41%|████ | 78/192 [00:22<00:33, 3.40it/s][A
41%|████ | 79/192 [00:22<00:39, 2.89it/s][A
42%|████▏ | 80/192 [00:23<00:42, 2.62it/s][A
42%|████▏ | 81/192 [00:23<00:42, 2.61it/s][A
43%|████▎ | 83/192 [00:24<00:34, 3.12it/s][A
44%|████▍ | 84/192 [00:24<00:41, 2.62it/s][A
44%|████▍ | 85/192 [00:24<00:40, 2.62it/s][A
45%|████▍ | 86/192 [00:25<00:38, 2.75it/s][A
45%|████▌ | 87/192 [00:25<00:44, 2.37it/s][A
46%|████▌ | 88/192 [00:26<00:40, 2.57it/s][A
46%|████▋ | 89/192 [00:26<00:39, 2.64it/s][A
47%|████▋ | 90/192 [00:26<00:35, 2.85it/s][A
47%|████▋ | 91/192 [00:27<00:34, 2.96it/s][A
48%|████▊ | 92/192 [00:27<00:34, 2.90it/s][A
48%|████▊ | 93/192 [00:27<00:39, 2.53it/s][A
49%|████▉ | 94/192 [00:28<00:45, 2.15it/s][A
49%|████▉ | 95/192 [00:29<00:44, 2.17it/s][A
50%|█████ | 96/192 [00:29<00:40, 2.36it/s][A
51%|█████ | 97/192 [00:29<00:44, 2.13it/s][A
51%|█████ | 98/192 [00:30<00:39, 2.37it/s][A
52%|█████▏ | 99/192 [00:30<00:37, 2.47it/s][A
52%|█████▏ | 100/192 [00:31<00:40, 2.27it/s][A
53%|█████▎ | 101/192 [00:31<00:45, 2.00it/s][A
53%|█████▎ | 102/192 [00:32<00:44, 2.01it/s][A
54%|█████▎ | 103/192 [00:32<00:43, 2.02it/s][A
54%|█████▍ | 104/192 [00:33<00:36, 2.43it/s][A
55%|█████▍ | 105/192 [00:33<00:35, 2.46it/s][A
55%|█████▌ | 106/192 [00:33<00:33, 2.59it/s][A
56%|█████▌ | 107/192 [00:34<00:32, 2.65it/s][A
56%|█████▋ | 108/192 [00:34<00:29, 2.81it/s][A
57%|█████▋ | 109/192 [00:35<00:36, 2.27it/s][A
57%|█████▋ | 110/192 [00:35<00:41, 1.98it/s][A
58%|█████▊ | 111/192 [00:36<00:49, 1.65it/s][A
58%|█████▊ | 112/192 [00:36<00:43, 1.82it/s][A
59%|█████▉ | 113/192 [00:37<00:40, 1.95it/s][A
59%|█████▉ | 114/192 [00:38<00:42, 1.83it/s][A
60%|█████▉ | 115/192 [00:38<00:43, 1.76it/s][A
60%|██████ | 116/192 [00:38<00:36, 2.10it/s][A
61%|██████ | 117/192 [00:39<00:27, 2.68it/s][A
61%|██████▏ | 118/192 [00:39<00:25, 2.96it/s][A
62%|██████▏ | 119/192 [00:39<00:27, 2.65it/s][A
62%|██████▎ | 120/192 [00:40<00:24, 2.89it/s][A
63%|██████▎ | 121/192 [00:40<00:26, 2.70it/s][A
64%|██████▎ | 122/192 [00:40<00:25, 2.79it/s][A
64%|██████▍ | 123/192 [00:41<00:25, 2.68it/s][A
65%|██████▍ | 124/192 [00:41<00:27, 2.44it/s][A
65%|██████▌ | 125/192 [00:42<00:25, 2.60it/s][A
66%|██████▌ | 127/192 [00:42<00:19, 3.30it/s][A
67%|██████▋ | 128/192 [00:42<00:19, 3.34it/s][A
67%|██████▋ | 129/192 [00:43<00:20, 3.14it/s][A
68%|██████▊ | 130/192 [00:43<00:23, 2.67it/s][A
68%|██████▊ | 131/192 [00:44<00:23, 2.61it/s][A
69%|██████▉ | 132/192 [00:44<00:20, 2.90it/s][A
69%|██████▉ | 133/192 [00:44<00:19, 3.07it/s][A
70%|██████▉ | 134/192 [00:44<00:17, 3.34it/s][A
70%|███████ | 135/192 [00:45<00:19, 2.95it/s][A
71%|███████ | 136/192 [00:45<00:20, 2.67it/s][A
71%|███████▏ | 137/192 [00:46<00:24, 2.21it/s][A
72%|███████▏ | 138/192 [00:46<00:25, 2.09it/s][A
72%|███████▏ | 139/192 [00:47<00:27, 1.92it/s][A
73%|███████▎ | 140/192 [00:47<00:23, 2.22it/s][A
73%|███████▎ | 141/192 [00:47<00:18, 2.82it/s][A
74%|███████▍ | 142/192 [00:48<00:15, 3.13it/s][A
74%|███████▍ | 143/192 [00:48<00:14, 3.37it/s][A
75%|███████▌ | 144/192 [00:48<00:14, 3.35it/s][A
76%|███████▌ | 145/192 [00:49<00:16, 2.79it/s][A
76%|███████▌ | 146/192 [00:49<00:18, 2.55it/s][A
77%|███████▋ | 147/192 [00:50<00:17, 2.57it/s][A
77%|███████▋ | 148/192 [00:50<00:16, 2.69it/s][A
78%|███████▊ | 149/192 [00:50<00:15, 2.75it/s][A
78%|███████▊ | 150/192 [00:51<00:14, 2.87it/s][A
79%|███████▊ | 151/192 [00:51<00:16, 2.43it/s][A
79%|███████▉ | 152/192 [00:51<00:14, 2.68it/s][A
80%|███████▉ | 153/192 [00:52<00:11, 3.29it/s][A
80%|████████ | 154/192 [00:52<00:11, 3.21it/s][A
81%|████████ | 155/192 [00:52<00:14, 2.49it/s][A
81%|████████▏ | 156/192 [00:53<00:15, 2.27it/s][A
82%|████████▏ | 157/192 [00:53<00:14, 2.43it/s][A
83%|████████▎ | 159/192 [00:53<00:08, 3.81it/s][A
83%|████████▎ | 160/192 [00:54<00:09, 3.52it/s][A
84%|████████▍ | 161/192 [00:54<00:10, 3.03it/s][A
84%|████████▍ | 162/192 [00:55<00:08, 3.34it/s][A
85%|████████▍ | 163/192 [00:55<00:07, 4.07it/s][A
86%|████████▌ | 165/192 [00:55<00:06, 4.38it/s][A
86%|████████▋ | 166/192 [00:55<00:07, 3.55it/s][A
87%|████████▋ | 167/192 [00:56<00:07, 3.44it/s][A
88%|████████▊ | 168/192 [00:56<00:06, 3.57it/s][A
88%|████████▊ | 169/192 [00:57<00:08, 2.65it/s][A
89%|████████▊ | 170/192 [00:57<00:09, 2.35it/s][A
89%|████████▉ | 171/192 [00:58<00:08, 2.54it/s][A
90%|████████▉ | 172/192 [00:58<00:07, 2.54it/s][A
90%|█████████ | 173/192 [00:58<00:07, 2.58it/s][A
91%|█████████ | 174/192 [00:59<00:06, 2.95it/s][A
91%|█████████ | 175/192 [00:59<00:06, 2.58it/s][A
92%|█████████▏| 176/192 [00:59<00:05, 2.72it/s][A
92%|█████████▏| 177/192 [00:59<00:04, 3.41it/s][A
93%|█████████▎| 178/192 [01:00<00:04, 3.40it/s][A
93%|█████████▎| 179/192 [01:00<00:04, 2.85it/s][A
94%|█████████▍| 180/192 [01:00<00:03, 3.18it/s][A
95%|█████████▍| 182/192 [01:01<00:02, 4.80it/s][A
96%|█████████▌| 184/192 [01:01<00:01, 6.28it/s][A
96%|█████████▋| 185/192 [01:01<00:01, 5.13it/s][A
97%|█████████▋| 186/192 [01:02<00:01, 3.72it/s][A
97%|█████████▋| 187/192 [01:02<00:01, 3.41it/s][A
98%|█████████▊| 189/192 [01:02<00:00, 4.84it/s][A
99%|█████████▉| 190/192 [01:03<00:00, 4.22it/s][A
99%|█████████▉| 191/192 [01:03<00:00, 3.87it/s][A
100%|██████████| 192/192 [01:03<00:00, 3.38it/s][A
[A 16%|█▌ | 150/936 [59:02<3:57:33, 18.13s/it]
100%|██████████| 192/192 [01:04<00:00, 3.38it/s][A
[A 16%|█▌ | 151/936 [59:27<8:36:28, 39.48s/it] 16%|█▌ | 152/936 [59:42<7:02:22, 32.32s/it] 16%|█▋ | 153/936 [59:59<6:02:31, 27.78s/it] 16%|█▋ | 154/936 [1:00:12<5:01:27, 23.13s/it] 17%|█▋ | 155/936 [1:00:25<4:21:00, 20.05s/it] 17%|█▋ | 155/936 [1:00:25<4:21:00, 20.05s/it] 17%|█▋ | 156/936 [1:00:40<4:02:40, 18.67s/it] 17%|█▋ | 157/936 [1:00:57<3:53:52, 18.01s/it] 17%|█▋ | 158/936 [1:01:06<3:19:31, 15.39s/it] 17%|█▋ | 159/936 [1:01:22<3:20:57, 15.52s/it] 17%|█▋ | 160/936 [1:01:37<3:21:10, 15.56s/it] 17%|█▋ | 160/936 [1:01:37<3:21:10, 15.56s/it] 17%|█▋ | 161/936 [1:02:02<3:54:49, 18.18s/it] 17%|█▋ | 162/936 [1:02:22<4:04:29, 18.95s/it] 17%|█▋ | 163/936 [1:02:42<4:06:57, 19.17s/it] 18%|█▊ | 164/936 [1:02:58<3:54:39, 18.24s/it] 18%|█▊ | 165/936 [1:03:15<3:47:48, 17.73s/it] 18%|█▊ | 165/936 [1:03:15<3:47:48, 17.73s/it] 18%|█▊ | 166/936 [1:03:30<3:38:56, 17.06s/it] 18%|█▊ | 167/936 [1:03:50<3:47:46, 17.77s/it] 18%|█▊ | 168/936 [1:04:04<3:34:38, 16.77s/it] 18%|█▊ | 169/936 [1:04:17<3:18:24, 15.52s/it] 18%|█▊ | 170/936 [1:04:22<2:40:27, 12.57s/it] 18%|█▊ | 170/936 [1:04:22<2:40:27, 12.57s/it] 18%|█▊ | 171/936 [1:04:38<2:53:56, 13.64s/it] 18%|█▊ | 172/936 [1:04:53<2:57:16, 13.92s/it] 18%|█▊ | 173/936 [1:05:07<2:58:06, 14.01s/it] 19%|█▊ | 174/936 [1:05:21<2:57:59, 14.02s/it] 19%|█▊ | 175/936 [1:05:32<2:45:02, 13.01s/it] 19%|█▊ | 175/936 [1:05:32<2:45:02, 13.01s/it] 19%|█▉ | 176/936 [1:05:54<3:18:29, 15.67s/it] 19%|█▉ | 177/936 [1:06:09<3:15:22, 15.44s/it] 19%|█▉ | 178/936 [1:06:26<3:21:15, 15.93s/it] 19%|█▉ | 179/936 [1:06:40<3:15:55, 15.53s/it] 19%|█▉ | 180/936 [1:06:56<3:16:25, 15.59s/it] 19%|█▉ | 180/936 [1:06:56<3:16:25, 15.59s/it] 19%|█▉ | 181/936 [1:07:18<3:38:20, 17.35s/it] 19%|█▉ | 182/936 [1:07:31<3:24:38, 16.28s/it] 20%|█▉ | 183/936 [1:07:41<2:58:11, 14.20s/it] 20%|█▉ | 184/936 [1:08:03<3:27:35, 16.56s/it] 20%|█▉ | 185/936 [1:08:21<3:34:51, 17.17s/it] 20%|█▉ | 185/936 [1:08:21<3:34:51, 17.17s/it] 20%|█▉ | 186/936 [1:08:41<3:42:44, 17.82s/it] 20%|█▉ | 187/936 [1:08:53<3:21:46, 16.16s/it] 20%|██ | 188/936 [1:09:08<3:16:56, 15.80s/it] 20%|██ | 189/936 [1:09:27<3:27:15, 16.65s/it] 20%|██ | 190/936 [1:09:42<3:23:40, 16.38s/it] 20%|██ | 190/936 [1:09:42<3:23:40, 16.38s/it] 20%|██ | 191/936 [1:10:00<3:28:00, 16.75s/it] 21%|██ | 192/936 [1:10:21<3:42:39, 17.96s/it] 21%|██ | 193/936 [1:10:45<4:07:55, 20.02s/it] 21%|██ | 194/936 [1:11:04<4:00:37, 19.46s/it] 21%|██ | 195/936 [1:11:20<3:48:07, 18.47s/it] 21%|██ | 195/936 [1:11:20<3:48:07, 18.47s/it] 21%|██ | 196/936 [1:11:36<3:38:23, 17.71s/it] 21%|██ | 197/936 [1:11:58<3:53:28, 18.96s/it] 21%|██ | 198/936 [1:12:18<3:57:04, 19.27s/it] 21%|██▏ | 199/936 [1:12:33<3:41:58, 18.07s/it] 21%|██▏ | 200/936 [1:12:50<3:36:22, 17.64s/it] 21%|██▏ | 200/936 [1:12:50<3:36:22, 17.64s/it]{'eval_loss': 0.33385396003723145, 'eval_runtime': 64.3938, 'eval_samples_per_second': 5.963, 'eval_steps_per_second': 2.982, 'epoch': 0.16}
{'loss': 0.4903, 'grad_norm': 0.5619863867759705, 'learning_rate': 0.00019293596044621416, 'epoch': 0.17}
{'loss': 0.4651, 'grad_norm': 0.6460179090499878, 'learning_rate': 0.00019226918150848068, 'epoch': 0.17}
{'loss': 0.466, 'grad_norm': 0.6186167597770691, 'learning_rate': 0.00019157359666062903, 'epoch': 0.18}
{'loss': 0.4862, 'grad_norm': 0.5170834064483643, 'learning_rate': 0.00019084942306026568, 'epoch': 0.18}
{'loss': 0.56, 'grad_norm': 0.5558421611785889, 'learning_rate': 0.0001900968867902419, 'epoch': 0.19}
{'loss': 0.5789, 'grad_norm': 0.5665396451950073, 'learning_rate': 0.00018931622278807212, 'epoch': 0.19}
{'loss': 0.2663, 'grad_norm': 0.6116076707839966, 'learning_rate': 0.00018850767477258753, 'epoch': 0.2}
{'loss': 0.4403, 'grad_norm': 0.5950028300285339, 'learning_rate': 0.00018767149516784874, 'epoch': 0.2}
{'loss': 0.3972, 'grad_norm': 0.5122601985931396, 'learning_rate': 0.00018680794502434018, 'epoch': 0.21}
{'loss': 0.4764, 'grad_norm': 0.5357354283332825, 'learning_rate': 0.00018591729393747187, 'epoch': 0.21}
0%| | 0/192 [00:00<?, ?it/s][A
1%| | 2/192 [00:00<00:47, 4.04it/s][A
2%|▏ | 3/192 [00:01<01:20, 2.34it/s][A
2%|▏ | 4/192 [00:01<01:15, 2.50it/s][A
3%|▎ | 5/192 [00:01<00:58, 3.22it/s][A
3%|▎ | 6/192 [00:01<00:55, 3.36it/s][A
4%|▎ | 7/192 [00:02<00:53, 3.43it/s][A
4%|▍ | 8/192 [00:02<00:48, 3.82it/s][A
5%|▍ | 9/192 [00:02<00:46, 3.91it/s][A
5%|▌ | 10/192 [00:03<00:52, 3.48it/s][A
6%|▌ | 11/192 [00:03<01:03, 2.86it/s][A
6%|▋ | 12/192 [00:03<00:55, 3.23it/s][A
7%|▋ | 13/192 [00:04<00:57, 3.13it/s][A
7%|▋ | 14/192 [00:04<00:55, 3.23it/s][A
8%|▊ | 15/192 [00:04<00:45, 3.91it/s][A
9%|▉ | 17/192 [00:04<00:42, 4.14it/s][A
9%|▉ | 18/192 [00:05<00:50, 3.44it/s][A
10%|▉ | 19/192 [00:05<01:01, 2.83it/s][A
10%|█ | 20/192 [00:06<01:11, 2.39it/s][A
11%|█ | 21/192 [00:06<01:11, 2.40it/s][A
12%|█▏ | 23/192 [00:07<00:57, 2.92it/s][A
12%|█▎ | 24/192 [00:07<00:57, 2.92it/s][A
13%|█▎ | 25/192 [00:07<00:47, 3.52it/s][A
14%|█▍ | 27/192 [00:08<00:33, 4.97it/s][A
15%|█▍ | 28/192 [00:08<00:34, 4.71it/s][A
15%|█▌ | 29/192 [00:08<00:36, 4.46it/s][A
16%|█▌ | 30/192 [00:08<00:40, 3.95it/s][A
16%|█▌ | 31/192 [00:09<00:45, 3.57it/s][A
17%|█▋ | 33/192 [00:09<00:30, 5.22it/s][A
18%|█▊ | 34/192 [00:09<00:33, 4.71it/s][A
18%|█▊ | 35/192 [00:10<00:42, 3.70it/s][A
19%|█▉ | 36/192 [00:10<00:52, 2.95it/s][A
19%|█▉ | 37/192 [00:11<01:03, 2.45it/s][A
20%|█▉ | 38/192 [00:11<00:55, 2.78it/s][A
20%|██ | 39/192 [00:11<00:52, 2.90it/s][A
21%|██ | 40/192 [00:12<00:48, 3.11it/s][A
21%|██▏ | 41/192 [00:12<00:38, 3.88it/s][A
22%|██▏ | 42/192 [00:12<00:41, 3.66it/s][A
22%|██▏ | 43/192 [00:13<00:52, 2.81it/s][A
23%|██▎ | 44/192 [00:13<00:49, 2.98it/s][A
23%|██▎ | 45/192 [00:13<00:40, 3.62it/s][A
24%|██▍ | 47/192 [00:13<00:26, 5.38it/s][A
25%|██▌ | 48/192 [00:13<00:30, 4.78it/s][A
26%|██▌ | 49/192 [00:14<00:40, 3.55it/s][A
26%|██▌ | 50/192 [00:14<00:39, 3.61it/s][A
27%|██▋ | 51/192 [00:14<00:37, 3.75it/s][A
27%|██▋ | 52/192 [00:15<00:43, 3.25it/s][A
28%|██▊ | 53/192 [00:15<00:43, 3.18it/s][A
29%|██▊ | 55/192 [00:15<00:27, 4.89it/s][A
30%|██▉ | 57/192 [00:15<00:21, 6.39it/s][A
30%|███ | 58/192 [00:16<00:25, 5.20it/s][A
31%|███ | 59/192 [00:16<00:38, 3.41it/s][A
31%|███▏ | 60/192 [00:17<00:39, 3.30it/s][A
32%|███▏ | 61/192 [00:17<00:32, 3.99it/s][A
32%|███▏ | 62/192 [00:17<00:35, 3.68it/s][A
33%|███▎ | 63/192 [00:18<00:44, 2.93it/s][A
33%|███▎ | 64/192 [00:18<00:39, 3.24it/s][A
34%|███▍ | 66/192 [00:18<00:32, 3.92it/s][A
35%|███▍ | 67/192 [00:19<00:33, 3.73it/s][A
36%|███▌ | 69/192 [00:19<00:29, 4.18it/s][A
36%|███▋ | 70/192 [00:20<00:37, 3.21it/s][A
37%|███▋ | 71/192 [00:20<00:40, 3.00it/s][A
38%|███▊ | 73/192 [00:20<00:34, 3.45it/s][A
39%|███▊ | 74/192 [00:21<00:34, 3.43it/s][A
39%|███▉ | 75/192 [00:21<00:28, 4.06it/s][A
40%|████ | 77/192 [00:21<00:26, 4.37it/s][A
41%|████ | 78/192 [00:22<00:33, 3.39it/s][A
41%|████ | 79/192 [00:22<00:39, 2.88it/s][A
42%|████▏ | 80/192 [00:23<00:42, 2.61it/s][A
42%|████▏ | 81/192 [00:23<00:42, 2.61it/s][A
43%|████▎ | 83/192 [00:24<00:34, 3.12it/s][A
44%|████▍ | 84/192 [00:24<00:41, 2.62it/s][A
44%|████▍ | 85/192 [00:25<00:45, 2.33it/s][A
45%|████▍ | 86/192 [00:25<00:41, 2.53it/s][A
45%|████▌ | 87/192 [00:26<00:46, 2.24it/s][A
46%|████▌ | 88/192 [00:26<00:42, 2.46it/s][A
46%|████▋ | 89/192 [00:26<00:40, 2.56it/s][A
47%|████▋ | 90/192 [00:27<00:36, 2.79it/s][A
47%|████▋ | 91/192 [00:27<00:34, 2.91it/s][A
48%|████▊ | 92/192 [00:27<00:34, 2.87it/s][A
48%|████▊ | 93/192 [00:28<00:39, 2.49it/s][A
49%|████▉ | 94/192 [00:28<00:45, 2.13it/s][A
49%|████▉ | 95/192 [00:29<00:45, 2.15it/s][A
50%|█████ | 96/192 [00:29<00:40, 2.34it/s][A
51%|█████ | 97/192 [00:30<00:44, 2.12it/s][A
51%|█████ | 98/192 [00:30<00:39, 2.37it/s][A
52%|█████▏ | 99/192 [00:30<00:37, 2.46it/s][A
52%|█████▏ | 100/192 [00:31<00:40, 2.26it/s][A
53%|█████▎ | 101/192 [00:32<00:46, 1.98it/s][A
53%|█████▎ | 102/192 [00:32<00:44, 2.02it/s][A
54%|█████▎ | 103/192 [00:33<00:43, 2.03it/s][A
54%|█████▍ | 104/192 [00:33<00:36, 2.44it/s][A
55%|█████▍ | 105/192 [00:33<00:28, 3.09it/s][A
55%|█████▌ | 106/192 [00:33<00:28, 3.01it/s][A
56%|█████▌ | 107/192 [00:34<00:28, 2.94it/s][A
56%|█████▋ | 108/192 [00:34<00:27, 3.03it/s][A
57%|█████▋ | 109/192 [00:35<00:34, 2.37it/s][A
57%|█████▋ | 110/192 [00:35<00:40, 2.04it/s][A
58%|█████▊ | 111/192 [00:36<00:48, 1.68it/s][A
58%|█████▊ | 112/192 [00:36<00:43, 1.85it/s][A
59%|█████▉ | 113/192 [00:37<00:40, 1.96it/s][A
59%|█████▉ | 114/192 [00:37<00:42, 1.84it/s][A
60%|█████▉ | 115/192 [00:38<00:43, 1.77it/s][A
60%|██████ | 116/192 [00:38<00:36, 2.10it/s][A
61%|██████ | 117/192 [00:38<00:28, 2.68it/s][A
61%|██████▏ | 118/192 [00:39<00:25, 2.96it/s][A
62%|██████▏ | 119/192 [00:39<00:27, 2.64it/s][A
62%|██████▎ | 120/192 [00:39<00:24, 2.88it/s][A
63%|██████▎ | 121/192 [00:40<00:26, 2.70it/s][A
64%|██████▎ | 122/192 [00:40<00:25, 2.79it/s][A
64%|██████▍ | 123/192 [00:41<00:25, 2.67it/s][A
65%|██████▍ | 124/192 [00:41<00:27, 2.44it/s][A
65%|██████▌ | 125/192 [00:41<00:25, 2.65it/s][A
66%|██████▌ | 127/192 [00:42<00:19, 3.35it/s][A
67%|██████▋ | 128/192 [00:42<00:18, 3.38it/s][A
67%|██████▋ | 129/192 [00:43<00:19, 3.16it/s][A
68%|██████▊ | 130/192 [00:43<00:23, 2.67it/s][A
68%|██████▊ | 131/192 [00:43<00:23, 2.61it/s][A
69%|██████▉ | 132/192 [00:44<00:20, 2.89it/s][A
69%|██████▉ | 133/192 [00:44<00:19, 3.06it/s][A
70%|██████▉ | 134/192 [00:44<00:17, 3.32it/s][A
70%|███████ | 135/192 [00:45<00:19, 2.93it/s][A
71%|███████ | 136/192 [00:45<00:20, 2.67it/s][A
71%|███████▏ | 137/192 [00:46<00:24, 2.21it/s][A
72%|███████▏ | 138/192 [00:46<00:25, 2.09it/s][A
72%|███████▏ | 139/192 [00:47<00:27, 1.92it/s][A
73%|███████▎ | 140/192 [00:47<00:23, 2.22it/s][A
73%|███████▎ | 141/192 [00:47<00:18, 2.82it/s][A
74%|███████▍ | 142/192 [00:48<00:15, 3.18it/s][A
74%|███████▍ | 143/192 [00:48<00:14, 3.35it/s][A
75%|███████▌ | 144/192 [00:48<00:14, 3.29it/s][A
76%|███████▌ | 145/192 [00:49<00:16, 2.84it/s][A
76%|███████▌ | 146/192 [00:49<00:18, 2.55it/s][A
77%|███████▋ | 147/192 [00:49<00:17, 2.56it/s][A
77%|███████▋ | 148/192 [00:50<00:16, 2.68it/s][A
78%|███████▊ | 149/192 [00:50<00:15, 2.74it/s][A
78%|███████▊ | 150/192 [00:50<00:14, 2.84it/s][A
79%|███████▊ | 151/192 [00:51<00:16, 2.42it/s][A
79%|███████▉ | 152/192 [00:51<00:15, 2.66it/s][A
80%|███████▉ | 153/192 [00:51<00:11, 3.28it/s][A
80%|████████ | 154/192 [00:52<00:11, 3.24it/s][A
81%|████████ | 155/192 [00:52<00:14, 2.48it/s][A
81%|████████▏ | 156/192 [00:53<00:15, 2.27it/s][A
82%|████████▏ | 157/192 [00:53<00:14, 2.42it/s][A
83%|████████▎ | 159/192 [00:53<00:08, 3.79it/s][A
83%|████████▎ | 160/192 [00:54<00:08, 3.56it/s][A
84%|████████▍ | 161/192 [00:54<00:10, 3.00it/s][A
84%|████████▍ | 162/192 [00:54<00:09, 3.31it/s][A
85%|████████▍ | 163/192 [00:55<00:07, 4.03it/s][A
86%|████████▌ | 165/192 [00:55<00:06, 4.35it/s][A
86%|████████▋ | 166/192 [00:55<00:07, 3.53it/s][A
87%|████████▋ | 167/192 [00:56<00:07, 3.44it/s][A
88%|████████▊ | 168/192 [00:56<00:06, 3.53it/s][A
88%|████████▊ | 169/192 [00:57<00:08, 2.63it/s][A
89%|████████▊ | 170/192 [00:57<00:09, 2.35it/s][A
89%|████████▉ | 171/192 [00:58<00:08, 2.51it/s][A
90%|████████▉ | 172/192 [00:58<00:07, 2.54it/s][A
90%|█████████ | 173/192 [00:58<00:07, 2.56it/s][A
91%|█████████ | 174/192 [00:59<00:06, 2.91it/s][A
91%|█████████ | 175/192 [00:59<00:06, 2.56it/s][A
92%|█████████▏| 176/192 [00:59<00:05, 2.70it/s][A
92%|█████████▏| 177/192 [00:59<00:04, 3.36it/s][A
93%|█████████▎| 178/192 [01:00<00:04, 3.38it/s][A
93%|█████████▎| 179/192 [01:00<00:04, 2.87it/s][A
94%|█████████▍| 180/192 [01:00<00:03, 3.19it/s][A
95%|█████████▍| 182/192 [01:01<00:02, 4.80it/s][A
96%|█████████▌| 184/192 [01:01<00:01, 6.28it/s][A
96%|█████████▋| 185/192 [01:01<00:01, 5.13it/s][A
97%|█████████▋| 186/192 [01:02<00:01, 3.71it/s][A
97%|█████████▋| 187/192 [01:02<00:01, 3.42it/s][A
98%|█████████▊| 189/192 [01:02<00:00, 4.83it/s][A
99%|█████████▉| 190/192 [01:03<00:00, 4.22it/s][A
99%|█████████▉| 191/192 [01:03<00:00, 3.86it/s][A
100%|██████████| 192/192 [01:03<00:00, 3.39it/s][A
[A 21%|██▏ | 200/936 [1:13:54<3:36:22, 17.64s/it]
100%|██████████| 192/192 [01:04<00:00, 3.39it/s][A
[Awandb: WARNING URL not available in offline run
`rope_scaling`'s beta_fast field must be a float, got 32
`rope_scaling`'s beta_slow field must be a float, got 1
21%|██▏ | 201/936 [1:14:12<7:33:44, 37.04s/it] 22%|██▏ | 202/936 [1:14:29<6:21:14, 31.16s/it] 22%|██▏ | 203/936 [1:14:49<5:39:34, 27.80s/it] 22%|██▏ | 204/936 [1:15:07<5:01:53, 24.74s/it] 22%|██▏ | 205/936 [1:15:30<4:54:42, 24.19s/it] 22%|██▏ | 205/936 [1:15:30<4:54:42, 24.19s/it] 22%|██▏ | 206/936 [1:16:32<7:13:27, 35.63s/it] 22%|██▏ | 207/936 [1:16:57<6:34:21, 32.46s/it] 22%|██▏ | 208/936 [1:17:18<5:49:55, 28.84s/it] 22%|██▏ | 209/936 [1:17:37<5:15:55, 26.07s/it] 22%|██▏ | 210/936 [1:17:51<4:30:46, 22.38s/it] 22%|██▏ | 210/936 [1:17:51<4:30:46, 22.38s/it] 23%|██▎ | 211/936 [1:18:12<4:27:12, 22.11s/it] 23%|██▎ | 212/936 [1:18:40<4:46:24, 23.74s/it] 23%|██▎ | 213/936 [1:19:00<4:33:59, 22.74s/it] 23%|██▎ | 214/936 [1:19:21<4:26:59, 22.19s/it] 23%|██▎ | 215/936 [1:19:40<4:14:24, 21.17s/it] 23%|██▎ | 215/936 [1:19:40<4:14:24, 21.17s/it] 23%|██▎ | 216/936 [1:19:55<3:50:06, 19.18s/it] 23%|██▎ | 217/936 [1:20:10<3:35:40, 18.00s/it] 23%|██▎ | 218/936 [1:20:27<3:33:47, 17.87s/it] 23%|██▎ | 219/936 [1:20:42<3:22:46, 16.97s/it] 24%|██▎ | 220/936 [1:20:56<3:09:23, 15.87s/it] 24%|██▎ | 220/936 [1:20:56<3:09:23, 15.87s/it] 24%|██▎ | 221/936 [1:21:08<2:57:33, 14.90s/it] 24%|██▎ | 222/936 [1:21:30<3:20:22, 16.84s/it] 24%|██▍ | 223/936 [1:21:48<3:25:58, 17.33s/it] 24%|██▍ | 224/936 [1:22:04<3:21:48, 17.01s/it] 24%|██▍ | 225/936 [1:22:17<3:05:43, 15.67s/it] 24%|██▍ | 225/936 [1:22:17<3:05:43, 15.67s/it] 24%|██▍ | 226/936 [1:22:25<2:38:35, 13.40s/it] 24%|██▍ | 227/936 [1:22:42<2:51:48, 14.54s/it] 24%|██▍ | 228/936 [1:23:00<3:04:51, 15.67s/it] 24%|██▍ | 229/936 [1:23:14<2:58:25, 15.14s/it] 25%|██▍ | 230/936 [1:23:27<2:50:43, 14.51s/it] 25%|██▍ | 230/936 [1:23:27<2:50:43, 14.51s/it] 25%|██▍ | 231/936 [1:23:43<2:54:09, 14.82s/it] 25%|██▍ | 232/936 [1:24:01<3:04:14, 15.70s/it] 25%|██▍ | 233/936 [1:24:18<3:08:14, 16.07s/it] 25%|██▌ | 234/936 [1:24:37<3:20:16, 17.12s/it] 25%|██▌ | 235/936 [1:24:48<2:58:15, 15.26s/it] 25%|██▌ | 235/936 [1:24:48<2:58:15, 15.26s/it] 25%|██▌ | 236/936 [1:25:06<3:08:57, 16.20s/it] 25%|██▌ | 237/936 [1:25:25<3:17:01, 16.91s/it] 25%|██▌ | 238/936 [1:25:40<3:10:43, 16.40s/it] 26%|██▌ | 239/936 [1:25:57<3:12:23, 16.56s/it] 26%|██▌ | 240/936 [1:26:11<3:02:22, 15.72s/it] 26%|██▌ | 240/936 [1:26:11<3:02:22, 15.72s/it] 26%|██▌ | 241/936 [1:26:32<3:20:44, 17.33s/it] 26%|██▌ | 242/936 [1:26:45<3:05:11, 16.01s/it] 26%|██▌ | 243/936 [1:27:00<3:02:12, 15.78s/it] 26%|██▌ | 244/936 [1:27:23<3:25:31, 17.82s/it] 26%|██▌ | 245/936 [1:27:41<3:25:16, 17.82s/it] 26%|██▌ | 245/936 [1:27:41<3:25:16, 17.82s/it] 26%|██▋ | 246/936 [1:27:57<3:21:04, 17.48s/it] 26%|██▋ | 247/936 [1:28:18<3:30:55, 18.37s/it] 26%|██▋ | 248/936 [1:28:38<3:37:36, 18.98s/it] 27%|██▋ | 249/936 [1:29:02<3:55:09, 20.54s/it] 27%|██▋ | 250/936 [1:29:09<3:06:39, 16.33s/it] 27%|██▋ | 250/936 [1:29:09<3:06:39, 16.33s/it]{'eval_loss': 0.3293338119983673, 'eval_runtime': 64.3912, 'eval_samples_per_second': 5.964, 'eval_steps_per_second': 2.982, 'epoch': 0.21}
{'loss': 0.5307, 'grad_norm': 0.5082443952560425, 'learning_rate': 0.0001849998199634131, 'epoch': 0.22}
{'loss': 0.419, 'grad_norm': 0.5360233783721924, 'learning_rate': 0.00018405580953228493, 'epoch': 0.22}
{'loss': 0.3601, 'grad_norm': 0.41893500089645386, 'learning_rate': 0.00018308555735873844, 'epoch': 0.23}
{'loss': 0.5563, 'grad_norm': 0.6814525723457336, 'learning_rate': 0.0001820893663499464, 'epoch': 0.24}
{'loss': 0.4657, 'grad_norm': 0.4897703528404236, 'learning_rate': 0.00018106754751103735, 'epoch': 0.24}
{'loss': 0.417, 'grad_norm': 0.4542348384857178, 'learning_rate': 0.00018002041984800174, 'epoch': 0.25}
{'loss': 0.4546, 'grad_norm': 0.47860705852508545, 'learning_rate': 0.00017894831026809985, 'epoch': 0.25}
{'loss': 0.4021, 'grad_norm': 0.6006109714508057, 'learning_rate': 0.00017785155347780352, 'epoch': 0.26}
{'loss': 0.4151, 'grad_norm': 0.6438106298446655, 'learning_rate': 0.00017673049187830252, 'epoch': 0.26}
{'loss': 0.5328, 'grad_norm': 0.6574239730834961, 'learning_rate': 0.00017558547545860868, 'epoch': 0.27}
0%| | 0/192 [00:00<?, ?it/s][A
1%| | 2/192 [00:00<00:46, 4.04it/s][A
2%|▏ | 3/192 [00:01<01:22, 2.30it/s][A
2%|▏ | 4/192 [00:01<01:16, 2.46it/s][A
3%|▎ | 5/192 [00:01<00:58, 3.17it/s][A
3%|▎ | 6/192 [00:01<00:55, 3.33it/s][A
4%|▎ | 7/192 [00:02<00:54, 3.39it/s][A
4%|▍ | 8/192 [00:02<00:48, 3.77it/s][A
5%|▍ | 9/192 [00:02<00:47, 3.87it/s][A
5%|▌ | 10/192 [00:03<00:52, 3.46it/s][A
6%|▌ | 11/192 [00:03<01:03, 2.85it/s][A
6%|▋ | 12/192 [00:03<00:55, 3.22it/s][A
7%|▋ | 13/192 [00:04<00:57, 3.11it/s][A
7%|▋ | 14/192 [00:04<00:55, 3.22it/s][A
8%|▊ | 15/192 [00:04<00:45, 3.88it/s][A
9%|▉ | 17/192 [00:04<00:42, 4.11it/s][A
9%|▉ | 18/192 [00:05<00:50, 3.43it/s][A
10%|▉ | 19/192 [00:05<01:01, 2.81it/s][A
10%|█ | 20/192 [00:06<01:12, 2.39it/s][A
11%|█ | 21/192 [00:06<01:11, 2.40it/s][A
12%|█▏ | 23/192 [00:07<00:57, 2.92it/s][A
12%|█▎ | 24/192 [00:07<00:57, 2.93it/s][A
13%|█▎ | 25/192 [00:07<00:47, 3.52it/s][A
14%|█▍ | 27/192 [00:08<00:33, 4.96it/s][A
15%|█▍ | 28/192 [00:08<00:34, 4.70it/s][A
15%|█▌ | 29/192 [00:08<00:36, 4.44it/s][A
16%|█▌ | 30/192 [00:08<00:41, 3.93it/s][A
16%|█▌ | 31/192 [00:09<00:45, 3.52it/s][A
17%|█▋ | 33/192 [00:09<00:30, 5.16it/s][A
18%|█▊ | 34/192 [00:09<00:33, 4.66it/s][A
18%|█▊ | 35/192 [00:10<00:42, 3.67it/s][A
19%|█▉ | 36/192 [00:10<00:53, 2.93it/s][A
19%|█▉ | 37/192 [00:11<01:03, 2.45it/s][A
20%|█▉ | 38/192 [00:11<00:55, 2.79it/s][A
20%|██ | 39/192 [00:11<00:52, 2.91it/s][A
21%|██ | 40/192 [00:12<00:48, 3.11it/s][A
21%|██▏ | 41/192 [00:12<00:39, 3.87it/s][A
22%|██▏ | 42/192 [00:12<00:41, 3.64it/s][A
22%|██▏ | 43/192 [00:13<00:53, 2.80it/s][A
23%|██▎ | 44/192 [00:13<00:50, 2.96it/s][A
23%|██▎ | 45/192 [00:13<00:41, 3.55it/s][A
24%|██▍ | 47/192 [00:13<00:27, 5.28it/s][A
25%|██▌ | 48/192 [00:13<00:30, 4.71it/s][A
26%|██▌ | 49/192 [00:14<00:41, 3.47it/s][A
26%|██▌ | 50/192 [00:14<00:39, 3.59it/s][A
27%|██▋ | 51/192 [00:14<00:37, 3.73it/s][A
27%|██▋ | 52/192 [00:15<00:43, 3.23it/s][A
28%|██▊ | 53/192 [00:15<00:43, 3.17it/s][A
29%|██▊ | 55/192 [00:15<00:28, 4.84it/s][A
30%|██▉ | 57/192 [00:16<00:21, 6.33it/s][A
30%|███ | 58/192 [00:16<00:25, 5.18it/s][A
31%|███ | 59/192 [00:17<00:42, 3.11it/s][A
31%|███▏ | 60/192 [00:17<00:38, 3.40it/s][A
32%|███▏ | 61/192 [00:17<00:32, 4.08it/s][A
32%|███▏ | 62/192 [00:17<00:34, 3.73it/s][A
33%|███▎ | 63/192 [00:18<00:43, 2.95it/s][A
33%|███▎ | 64/192 [00:18<00:39, 3.25it/s][A
34%|███▍ | 65/192 [00:18<00:31, 4.00it/s][A
34%|███▍ | 66/192 [00:18<00:32, 3.90it/s][A
35%|███▍ | 67/192 [00:19<00:34, 3.67it/s][A
36%|███▌ | 69/192 [00:19<00:29, 4.19it/s][A
36%|███▋ | 70/192 [00:20<00:38, 3.16it/s][A
37%|███▋ | 71/192 [00:20<00:40, 2.96it/s][A
38%|███▊ | 73/192 [00:20<00:34, 3.43it/s][A
39%|███▊ | 74/192 [00:21<00:34, 3.40it/s][A
39%|███▉ | 75/192 [00:21<00:29, 4.03it/s][A
40%|████ | 77/192 [00:21<00:26, 4.36it/s][A
41%|████ | 78/192 [00:22<00:33, 3.38it/s][A
41%|████ | 79/192 [00:22<00:39, 2.86it/s][A
42%|████▏ | 80/192 [00:23<00:43, 2.60it/s][A
42%|████▏ | 81/192 [00:23<00:42, 2.59it/s][A
43%|████▎ | 83/192 [00:24<00:35, 3.09it/s][A
44%|████▍ | 84/192 [00:24<00:41, 2.61it/s][A
44%|████▍ | 85/192 [00:25<00:40, 2.62it/s][A
45%|████▍ | 86/192 [00:25<00:38, 2.74it/s][A
45%|████▌ | 87/192 [00:26<00:44, 2.36it/s][A
46%|████▌ | 88/192 [00:26<00:40, 2.56it/s][A
46%|████▋ | 89/192 [00:26<00:39, 2.62it/s][A
47%|████▋ | 90/192 [00:26<00:35, 2.85it/s][A
47%|████▋ | 91/192 [00:27<00:34, 2.95it/s][A
48%|████▊ | 92/192 [00:27<00:34, 2.89it/s][A
48%|████▊ | 93/192 [00:28<00:39, 2.50it/s][A
49%|████▉ | 94/192 [00:28<00:45, 2.14it/s][A
49%|████▉ | 95/192 [00:29<00:45, 2.15it/s][A
50%|█████ | 96/192 [00:29<00:41, 2.34it/s][A
51%|█████ | 97/192 [00:30<00:45, 2.11it/s][A
51%|█████ | 98/192 [00:30<00:39, 2.37it/s][A
52%|█████▏ | 99/192 [00:30<00:38, 2.43it/s][A
52%|█████▏ | 100/192 [00:31<00:40, 2.25it/s][A
53%|█████▎ | 101/192 [00:32<00:45, 1.98it/s][A
53%|█████▎ | 102/192 [00:32<00:45, 2.00it/s][A
54%|█████▎ | 103/192 [00:33<00:44, 2.00it/s][A
54%|█████▍ | 104/192 [00:33<00:36, 2.42it/s][A
55%|█████▍ | 105/192 [00:33<00:28, 3.08it/s][A
55%|█████▌ | 106/192 [00:33<00:28, 3.00it/s][A
56%|█████▌ | 107/192 [00:34<00:29, 2.92it/s][A
56%|█████▋ | 108/192 [00:34<00:27, 3.02it/s][A
57%|█████▋ | 109/192 [00:35<00:35, 2.37it/s][A
57%|█████▋ | 110/192 [00:35<00:40, 2.03it/s][A
58%|█████▊ | 111/192 [00:36<00:48, 1.67it/s][A
58%|█████▊ | 112/192 [00:36<00:43, 1.84it/s][A
59%|█████▉ | 113/192 [00:37<00:40, 1.96it/s][A
59%|█████▉ | 114/192 [00:37<00:42, 1.84it/s][A
60%|█████▉ | 115/192 [00:38<00:43, 1.77it/s][A
60%|██████ | 116/192 [00:38<00:36, 2.10it/s][A
61%|██████ | 117/192 [00:39<00:27, 2.68it/s][A
61%|██████▏ | 118/192 [00:39<00:24, 2.96it/s][A
62%|██████▏ | 119/192 [00:39<00:27, 2.64it/s][A
62%|██████▎ | 120/192 [00:40<00:25, 2.87it/s][A
63%|██████▎ | 121/192 [00:40<00:26, 2.70it/s][A
64%|██████▎ | 122/192 [00:40<00:25, 2.78it/s][A
64%|██████▍ | 123/192 [00:41<00:25, 2.67it/s][A
65%|██████▍ | 124/192 [00:41<00:27, 2.44it/s][A
65%|██████▌ | 125/192 [00:41<00:25, 2.65it/s][A
66%|██████▌ | 127/192 [00:42<00:19, 3.35it/s][A
67%|██████▋ | 128/192 [00:42<00:18, 3.39it/s][A
67%|██████▋ | 129/192 [00:43<00:19, 3.17it/s][A
68%|██████▊ | 130/192 [00:43<00:23, 2.67it/s][A
68%|██████▊ | 131/192 [00:43<00:23, 2.61it/s][A
69%|██████▉ | 132/192 [00:44<00:20, 2.89it/s][A
69%|██████▉ | 133/192 [00:44<00:19, 3.07it/s][A
70%|██████▉ | 134/192 [00:44<00:17, 3.33it/s][A
70%|███████ | 135/192 [00:45<00:19, 2.94it/s][A
71%|███████ | 136/192 [00:45<00:20, 2.68it/s][A
71%|███████▏ | 137/192 [00:46<00:24, 2.21it/s][A
72%|███████▏ | 138/192 [00:46<00:25, 2.09it/s][A
72%|███████▏ | 139/192 [00:47<00:27, 1.91it/s][A
73%|███████▎ | 140/192 [00:47<00:23, 2.22it/s][A
73%|███████▎ | 141/192 [00:47<00:18, 2.82it/s][A
74%|███████▍ | 142/192 [00:48<00:15, 3.18it/s][A
74%|███████▍ | 143/192 [00:48<00:14, 3.34it/s][A
75%|███████▌ | 144/192 [00:48<00:14, 3.32it/s][A
76%|███████▌ | 145/192 [00:49<00:16, 2.81it/s][A
76%|███████▌ | 146/192 [00:49<00:18, 2.52it/s][A
77%|███████▋ | 147/192 [00:50<00:17, 2.55it/s][A
77%|███████▋ | 148/192 [00:50<00:16, 2.68it/s][A
78%|███████▊ | 149/192 [00:50<00:15, 2.73it/s][A
78%|███████▊ | 150/192 [00:50<00:14, 2.86it/s][A
79%|███████▊ | 151/192 [00:51<00:17, 2.39it/s][A
79%|███████▉ | 152/192 [00:51<00:14, 2.67it/s][A
80%|███████▉ | 153/192 [00:51<00:11, 3.28it/s][A
80%|████████ | 154/192 [00:52<00:11, 3.24it/s][A
81%|████████ | 155/192 [00:52<00:15, 2.45it/s][A
81%|████████▏ | 156/192 [00:53<00:15, 2.25it/s][A
82%|████████▏ | 157/192 [00:53<00:14, 2.41it/s][A
83%|████████▎ | 159/192 [00:53<00:08, 3.78it/s][A
83%|████████▎ | 160/192 [00:54<00:09, 3.55it/s][A
84%|████████▍ | 161/192 [00:54<00:10, 3.00it/s][A
84%|████████▍ | 162/192 [00:55<00:09, 3.31it/s][A
85%|████████▍ | 163/192 [00:55<00:07, 4.04it/s][A
86%|████████▌ | 165/192 [00:55<00:06, 4.36it/s][A
86%|████████▋ | 166/192 [00:55<00:07, 3.53it/s][A
87%|████████▋ | 167/192 [00:56<00:07, 3.44it/s][A
88%|████████▊ | 168/192 [00:56<00:06, 3.53it/s][A
88%|████████▊ | 169/192 [00:57<00:08, 2.63it/s][A
89%|████████▊ | 170/192 [00:57<00:09, 2.34it/s][A
89%|████████▉ | 171/192 [00:58<00:08, 2.54it/s][A
90%|████████▉ | 172/192 [00:58<00:07, 2.55it/s][A
90%|█████████ | 173/192 [00:58<00:07, 2.54it/s][A
91%|█████████ | 174/192 [00:59<00:06, 2.90it/s][A
91%|█████████ | 175/192 [00:59<00:06, 2.55it/s][A
92%|█████████▏| 176/192 [00:59<00:05, 2.70it/s][A
92%|█████████▏| 177/192 [01:00<00:04, 3.37it/s][A
93%|█████████▎| 178/192 [01:00<00:04, 3.34it/s][A
93%|█████████▎| 179/192 [01:00<00:04, 2.86it/s][A
94%|█████████▍| 180/192 [01:01<00:03, 3.18it/s][A
95%|█████████▍| 182/192 [01:01<00:02, 4.78it/s][A
96%|█████████▌| 184/192 [01:01<00:01, 6.24it/s][A
96%|█████████▋| 185/192 [01:01<00:01, 5.09it/s][A
97%|█████████▋| 186/192 [01:02<00:01, 3.71it/s][A
97%|█████████▋| 187/192 [01:02<00:01, 3.42it/s][A
98%|█████████▊| 189/192 [01:02<00:00, 4.83it/s][A
99%|█████████▉| 190/192 [01:03<00:00, 4.22it/s][A
99%|█████████▉| 191/192 [01:03<00:00, 3.86it/s][A
100%|██████████| 192/192 [01:03<00:00, 3.37it/s][A
[A 27%|██▋ | 250/936 [1:30:13<3:06:39, 16.33s/it]
100%|██████████| 192/192 [01:04<00:00, 3.37it/s][A
[A 27%|██▋ | 251/936 [1:30:29<6:46:16, 35.59s/it] 27%|██▋ | 252/936 [1:30:49<5:50:52, 30.78s/it] 27%|██▋ | 253/936 [1:31:05<4:58:44, 26.24s/it] 27%|██▋ | 254/936 [1:31:21<4:25:35, 23.37s/it] 27%|██▋ | 255/936 [1:31:37<3:58:35, 21.02s/it] 27%|██▋ | 255/936 [1:31:37<3:58:35, 21.02s/it] 27%|██▋ | 256/936 [1:31:51<3:36:26, 19.10s/it] 27%|██▋ | 257/936 [1:32:07<3:25:58, 18.20s/it] 28%|██▊ | 258/936 [1:32:27<3:29:42, 18.56s/it] 28%|██▊ | 259/936 [1:32:42<3:16:27, 17.41s/it] 28%|██▊ | 260/936 [1:32:55<3:03:34, 16.29s/it] 28%|██▊ | 260/936 [1:32:55<3:03:34, 16.29s/it] 28%|██▊ | 261/936 [1:33:11<3:01:15, 16.11s/it] 28%|██▊ | 262/936 [1:33:28<3:03:14, 16.31s/it] 28%|██▊ | 263/936 [1:33:40<2:50:57, 15.24s/it] 28%|██▊ | 264/936 [1:33:58<2:56:50, 15.79s/it] 28%|██▊ | 265/936 [1:34:09<2:42:42, 14.55s/it] 28%|██▊ | 265/936 [1:34:09<2:42:42, 14.55s/it] 28%|██▊ | 266/936 [1:34:21<2:31:38, 13.58s/it] 29%|██▊ | 267/936 [1:34:37<2:41:00, 14.44s/it] 29%|██▊ | 268/936 [1:34:53<2:45:12, 14.84s/it] 29%|██▊ | 269/936 [1:35:09<2:51:14, 15.40s/it] 29%|██▉ | 270/936 [1:35:29<3:04:18, 16.60s/it] 29%|██▉ | 270/936 [1:35:29<3:04:18, 16.60s/it] 29%|██▉ | 271/936 [1:35:44<2:58:54, 16.14s/it] 29%|██▉ | 272/936 [1:36:00<2:57:47, 16.07s/it] 29%|██▉ | 273/936 [1:36:15<2:53:31, 15.70s/it] 29%|██▉ | 274/936 [1:36:27<2:42:18, 14.71s/it] 29%|██▉ | 275/936 [1:36:45<2:52:31, 15.66s/it] 29%|██▉ | 275/936 [1:36:45<2:52:31, 15.66s/it] 29%|██▉ | 276/936 [1:37:01<2:51:55, 15.63s/it] 30%|██▉ | 277/936 [1:37:21<3:08:24, 17.15s/it] 30%|██▉ | 278/936 [1:37:26<2:28:45, 13.56s/it] 30%|██▉ | 279/936 [1:37:40<2:29:01, 13.61s/it] 30%|██▉ | 280/936 [1:37:56<2:35:14, 14.20s/it] 30%|██▉ | 280/936 [1:37:56<2:35:14, 14.20s/it] 30%|███ | 281/936 [1:38:14<2:47:08, 15.31s/it] 30%|███ | 282/936 [1:38:26<2:38:24, 14.53s/it] 30%|███ | 283/936 [1:38:40<2:34:42, 14.22s/it] 30%|███ | 284/936 [1:39:01<2:57:31, 16.34s/it] 30%|███ | 285/936 [1:39:23<3:16:20, 18.10s/it] 30%|███ | 285/936 [1:39:23<3:16:20, 18.10s/it] 31%|███ | 286/936 [1:39:36<2:58:02, 16.43s/it] 31%|███ | 287/936 [1:39:54<3:04:20, 17.04s/it] 31%|███ | 288/936 [1:40:11<3:02:48, 16.93s/it] 31%|███ | 289/936 [1:40:27<2:58:32, 16.56s/it] 31%|███ | 290/936 [1:40:37<2:39:13, 14.79s/it] 31%|███ | 290/936 [1:40:37<2:39:13, 14.79s/it] 31%|███ | 291/936 [1:40:51<2:36:13, 14.53s/it] 31%|███ | 292/936 [1:41:08<2:42:45, 15.16s/it] 31%|███▏ | 293/936 [1:41:24<2:45:46, 15.47s/it] 31%|███▏ | 294/936 [1:41:42<2:52:20, 16.11s/it] 32%|███▏ | 295/936 [1:41:56<2:47:53, 15.72s/it] 32%|███▏ | 295/936 [1:41:56<2:47:53, 15.72s/it] 32%|███▏ | 296/936 [1:42:15<2:56:30, 16.55s/it] 32%|███▏ | 297/936 [1:42:29<2:49:25, 15.91s/it] 32%|███▏ | 298/936 [1:42:46<2:50:33, 16.04s/it] 32%|███▏ | 299/936 [1:42:56<2:30:21, 14.16s/it] 32%|███▏ | 300/936 [1:43:07<2:20:16, 13.23s/it] 32%|███▏ | 300/936 [1:43:07<2:20:16, 13.23s/it]{'eval_loss': 0.3203986585140228, 'eval_runtime': 64.4396, 'eval_samples_per_second': 5.959, 'eval_steps_per_second': 2.98, 'epoch': 0.27}
{'loss': 0.3799, 'grad_norm': 0.5759228467941284, 'learning_rate': 0.0001744168616862918, 'epoch': 0.27}
{'loss': 0.4535, 'grad_norm': 0.6944848895072937, 'learning_rate': 0.00017322501539587988, 'epoch': 0.28}
{'loss': 0.4694, 'grad_norm': 0.5996747016906738, 'learning_rate': 0.00017201030867496005, 'epoch': 0.28}
{'loss': 0.3768, 'grad_norm': 0.42816469073295593, 'learning_rate': 0.00017077312074801498, 'epoch': 0.29}
{'loss': 0.2878, 'grad_norm': 0.5391706228256226, 'learning_rate': 0.00016951383785803115, 'epoch': 0.29}
{'loss': 0.5322, 'grad_norm': 0.8284974098205566, 'learning_rate': 0.00016823285314591593, 'epoch': 0.3}
{'loss': 0.4177, 'grad_norm': 0.42258185148239136, 'learning_rate': 0.00016693056652776126, 'epoch': 0.3}
{'loss': 0.5879, 'grad_norm': 0.6932430267333984, 'learning_rate': 0.00016560738456999232, 'epoch': 0.31}
{'loss': 0.4069, 'grad_norm': 0.5092376470565796, 'learning_rate': 0.00016426372036243945, 'epoch': 0.32}
{'loss': 0.4777, 'grad_norm': 0.5992222428321838, 'learning_rate': 0.00016289999338937427, 'epoch': 0.32}
0%| | 0/192 [00:00<?, ?it/s][A
1%| | 2/192 [00:00<00:46, 4.06it/s][A
2%|▏ | 3/192 [00:01<01:21, 2.31it/s][A
2%|▏ | 4/192 [00:01<01:15, 2.48it/s][A
3%|▎ | 5/192 [00:01<00:58, 3.19it/s][A
3%|▎ | 6/192 [00:01<00:55, 3.34it/s][A
4%|▎ | 7/192 [00:02<00:54, 3.41it/s][A
4%|▍ | 8/192 [00:02<00:48, 3.79it/s][A
5%|▍ | 9/192 [00:02<00:48, 3.76it/s][A
5%|▌ | 10/192 [00:03<00:53, 3.40it/s][A
6%|▌ | 11/192 [00:03<01:04, 2.82it/s][A
6%|▋ | 12/192 [00:03<00:56, 3.18it/s][A
7%|▋ | 13/192 [00:04<00:57, 3.10it/s][A
7%|▋ | 14/192 [00:04<00:55, 3.22it/s][A
8%|▊ | 15/192 [00:04<00:45, 3.88it/s][A
9%|▉ | 17/192 [00:04<00:42, 4.10it/s][A
9%|▉ | 18/192 [00:05<00:50, 3.43it/s][A
10%|▉ | 19/192 [00:05<01:01, 2.82it/s][A
10%|█ | 20/192 [00:06<01:11, 2.39it/s][A
11%|█ | 21/192 [00:06<01:11, 2.39it/s][A
12%|█▏ | 23/192 [00:07<00:57, 2.92it/s][A
12%|█▎ | 24/192 [00:07<00:57, 2.92it/s][A
13%|█▎ | 25/192 [00:07<00:47, 3.51it/s][A
14%|█▍ | 27/192 [00:08<00:33, 4.96it/s][A
15%|█▍ | 28/192 [00:08<00:34, 4.71it/s][A
15%|█▌ | 29/192 [00:08<00:36, 4.45it/s][A
16%|█▌ | 30/192 [00:08<00:41, 3.94it/s][A
16%|█▌ | 31/192 [00:09<00:45, 3.56it/s][A
17%|█▋ | 33/192 [00:09<00:30, 5.20it/s][A
18%|█▊ | 34/192 [00:09<00:33, 4.69it/s][A
18%|█▊ | 35/192 [00:10<00:42, 3.68it/s][A
19%|█▉ | 36/192 [00:10<00:53, 2.94it/s][A
19%|█▉ | 37/192 [00:11<01:03, 2.45it/s][A
20%|█▉ | 38/192 [00:11<00:55, 2.79it/s][A
20%|██ | 39/192 [00:11<00:52, 2.90it/s][A
21%|██ | 40/192 [00:12<00:48, 3.12it/s][A
21%|██▏ | 41/192 [00:12<00:38, 3.88it/s][A
22%|██▏ | 42/192 [00:12<00:41, 3.65it/s][A
22%|██▏ | 43/192 [00:13<00:53, 2.79it/s][A
23%|██▎ | 44/192 [00:13<00:50, 2.95it/s][A
23%|██▎ | 45/192 [00:13<00:40, 3.59it/s][A
24%|██▍ | 47/192 [00:13<00:27, 5.33it/s][A
25%|██▌ | 48/192 [00:13<00:30, 4.68it/s][A
26%|██▌ | 49/192 [00:14<00:40, 3.55it/s][A
26%|██▌ | 50/192 [00:14<00:39, 3.60it/s][A
27%|██▋ | 51/192 [00:14<00:37, 3.74it/s][A
27%|██▋ | 52/192 [00:15<00:43, 3.24it/s][A
28%|██▊ | 53/192 [00:15<00:43, 3.17it/s][A
29%|██▊ | 55/192 [00:15<00:28, 4.88it/s][A
30%|██▉ | 57/192 [00:16<00:21, 6.29it/s][A
30%|███ | 58/192 [00:16<00:25, 5.21it/s][A
31%|███ | 59/192 [00:16<00:39, 3.40it/s][A
31%|███▏ | 60/192 [00:17<00:39, 3.30it/s][A
32%|███▏ | 61/192 [00:17<00:32, 3.98it/s][A
32%|███▏ | 62/192 [00:17<00:35, 3.68it/s][A
33%|███▎ | 63/192 [00:18<00:44, 2.93it/s][A
33%|███▎ | 64/192 [00:18<00:39, 3.23it/s][A
34%|███▍ | 65/192 [00:18<00:31, 4.02it/s][A
34%|███▍ | 66/192 [00:18<00:32, 3.88it/s][A
35%|███▍ | 67/192 [00:19<00:34, 3.66it/s][A
36%|███▌ | 69/192 [00:19<00:29, 4.18it/s][A
36%|███▋ | 70/192 [00:20<00:38, 3.15it/s][A
37%|███▋ | 71/192 [00:20<00:41, 2.94it/s][A
38%|███▊ | 73/192 [00:20<00:34, 3.44it/s][A
39%|███▊ | 74/192 [00:21<00:34, 3.40it/s][A
39%|███▉ | 75/192 [00:21<00:28, 4.04it/s][A
40%|████ | 77/192 [00:21<00:26, 4.36it/s][A
41%|████ | 78/192 [00:22<00:33, 3.38it/s][A
41%|████ | 79/192 [00:22<00:39, 2.86it/s][A
42%|████▏ | 80/192 [00:23<00:43, 2.60it/s][A
42%|████▏ | 81/192 [00:23<00:42, 2.59it/s][A
43%|████▎ | 83/192 [00:24<00:35, 3.10it/s][A
44%|████▍ | 84/192 [00:24<00:41, 2.62it/s][A
44%|████▍ | 85/192 [00:25<00:40, 2.61it/s][A
45%|████▍ | 86/192 [00:25<00:38, 2.74it/s][A
45%|████▌ | 87/192 [00:26<00:44, 2.36it/s][A
46%|████▌ | 88/192 [00:26<00:40, 2.56it/s][A
46%|████▋ | 89/192 [00:26<00:39, 2.63it/s][A
47%|████▋ | 90/192 [00:26<00:35, 2.84it/s][A
47%|████▋ | 91/192 [00:27<00:34, 2.95it/s][A
48%|████▊ | 92/192 [00:27<00:34, 2.89it/s][A
48%|████▊ | 93/192 [00:28<00:39, 2.52it/s][A
49%|████▉ | 94/192 [00:28<00:45, 2.15it/s][A
49%|████▉ | 95/192 [00:29<00:44, 2.16it/s][A
50%|█████ | 96/192 [00:29<00:40, 2.35it/s][A
51%|█████ | 97/192 [00:30<00:44, 2.12it/s][A
51%|█████ | 98/192 [00:30<00:39, 2.37it/s][A
52%|█████▏ | 99/192 [00:30<00:37, 2.46it/s][A
52%|█████▏ | 100/192 [00:31<00:40, 2.26it/s][A
53%|█████▎ | 101/192 [00:31<00:45, 1.99it/s][A
53%|█████▎ | 102/192 [00:32<00:44, 2.01it/s][A
54%|█████▎ | 103/192 [00:32<00:44, 2.01it/s][A
54%|█████▍ | 104/192 [00:33<00:36, 2.42it/s][A
55%|█████▍ | 105/192 [00:33<00:31, 2.75it/s][A
55%|█████▌ | 106/192 [00:33<00:30, 2.80it/s][A
56%|█████▌ | 107/192 [00:34<00:30, 2.80it/s][A
56%|█████▋ | 108/192 [00:34<00:28, 2.92it/s][A
57%|█████▋ | 109/192 [00:35<00:35, 2.32it/s][A
57%|█████▋ | 110/192 [00:35<00:40, 2.01it/s][A
58%|█████▊ | 111/192 [00:36<00:49, 1.65it/s][A
58%|█████▊ | 112/192 [00:36<00:43, 1.83it/s][A
59%|█████▉ | 113/192 [00:37<00:40, 1.95it/s][A
59%|█████▉ | 114/192 [00:38<00:42, 1.83it/s][A
60%|█████▉ | 115/192 [00:38<00:43, 1.76it/s][A
60%|██████ | 116/192 [00:38<00:36, 2.11it/s][A
61%|██████ | 117/192 [00:39<00:28, 2.67it/s][A
61%|██████▏ | 118/192 [00:39<00:25, 2.95it/s][A
62%|██████▏ | 119/192 [00:39<00:27, 2.63it/s][A
62%|██████▎ | 120/192 [00:40<00:24, 2.89it/s][A
63%|██████▎ | 121/192 [00:40<00:26, 2.70it/s][A
64%|██████▎ | 122/192 [00:40<00:25, 2.79it/s][A
64%|██████▍ | 123/192 [00:41<00:25, 2.66it/s][A
65%|██████▍ | 124/192 [00:41<00:27, 2.44it/s][A
65%|██████▌ | 125/192 [00:42<00:25, 2.65it/s][A
66%|██████▌ | 127/192 [00:42<00:19, 3.34it/s][A
67%|██████▋ | 128/192 [00:42<00:18, 3.38it/s][A
67%|██████▋ | 129/192 [00:43<00:20, 3.12it/s][A
68%|██████▊ | 130/192 [00:43<00:23, 2.66it/s][A
68%|██████▊ | 131/192 [00:44<00:23, 2.59it/s][A
69%|██████▉ | 132/192 [00:44<00:20, 2.88it/s][A
69%|██████▉ | 133/192 [00:44<00:19, 3.05it/s][A
70%|██████▉ | 134/192 [00:44<00:17, 3.33it/s][A
70%|███████ | 135/192 [00:45<00:19, 2.93it/s][A
71%|███████ | 136/192 [00:45<00:20, 2.67it/s][A
71%|███████▏ | 137/192 [00:46<00:24, 2.20it/s][A
72%|███████▏ | 138/192 [00:46<00:25, 2.08it/s][A
72%|███████▏ | 139/192 [00:47<00:27, 1.92it/s][A
73%|███████▎ | 140/192 [00:47<00:23, 2.22it/s][A
73%|███████▎ | 141/192 [00:47<00:18, 2.82it/s][A
74%|███████▍ | 142/192 [00:48<00:15, 3.17it/s][A
74%|███████▍ | 143/192 [00:48<00:14, 3.33it/s][A
75%|███████▌ | 144/192 [00:48<00:14, 3.31it/s][A
76%|███████▌ | 145/192 [00:49<00:16, 2.80it/s][A
76%|███████▌ | 146/192 [00:49<00:18, 2.53it/s][A
77%|███████▋ | 147/192 [00:50<00:17, 2.51it/s][A
77%|███████▋ | 148/192 [00:50<00:16, 2.65it/s][A
78%|███████▊ | 149/192 [00:50<00:15, 2.71it/s][A
78%|███████▊ | 150/192 [00:51<00:14, 2.84it/s][A
79%|███████▊ | 151/192 [00:51<00:17, 2.40it/s][A
79%|███████▉ | 152/192 [00:51<00:15, 2.65it/s][A
80%|███████▉ | 153/192 [00:52<00:11, 3.25it/s][A
80%|████████ | 154/192 [00:52<00:11, 3.22it/s][A
81%|████████ | 155/192 [00:53<00:15, 2.47it/s][A
81%|████████▏ | 156/192 [00:53<00:15, 2.27it/s][A
82%|████████▏ | 157/192 [00:53<00:14, 2.42it/s][A
83%|████████▎ | 159/192 [00:54<00:08, 3.80it/s][A
83%|████████▎ | 160/192 [00:54<00:08, 3.57it/s][A
84%|████████▍ | 161/192 [00:54<00:10, 3.01it/s][A
84%|████████▍ | 162/192 [00:55<00:09, 3.33it/s][A
85%|████████▍ | 163/192 [00:55<00:07, 4.01it/s][A
86%|████████▌ | 165/192 [00:55<00:06, 4.32it/s][A
86%|████████▋ | 166/192 [00:56<00:07, 3.52it/s][A
87%|████████▋ | 167/192 [00:56<00:07, 3.44it/s][A
88%|████████▊ | 168/192 [00:56<00:06, 3.53it/s][A
88%|████████▊ | 169/192 [00:57<00:08, 2.63it/s][A
89%|████████▊ | 170/192 [00:57<00:09, 2.35it/s][A
89%|████████▉ | 171/192 [00:58<00:08, 2.53it/s][A
90%|████████▉ | 172/192 [00:58<00:07, 2.55it/s][A
90%|█████████ | 173/192 [00:58<00:07, 2.56it/s][A
91%|█████████ | 174/192 [00:59<00:06, 2.92it/s][A
91%|█████████ | 175/192 [00:59<00:06, 2.56it/s][A
92%|█████████▏| 176/192 [00:59<00:05, 2.71it/s][A
92%|█████████▏| 177/192 [01:00<00:04, 3.38it/s][A
93%|█████████▎| 178/192 [01:00<00:04, 3.38it/s][A
93%|█████████▎| 179/192 [01:00<00:04, 2.87it/s][A
94%|█████████▍| 180/192 [01:01<00:03, 3.19it/s][A
95%|█████████▍| 182/192 [01:01<00:02, 4.79it/s][A
96%|█████████▌| 184/192 [01:01<00:01, 6.29it/s][A
96%|█████████▋| 185/192 [01:01<00:01, 5.14it/s][A
97%|█████████▋| 186/192 [01:02<00:01, 3.72it/s][A
97%|█████████▋| 187/192 [01:02<00:01, 3.36it/s][A
98%|█████████▊| 189/192 [01:02<00:00, 4.74it/s][A
99%|█████████▉| 190/192 [01:03<00:00, 4.17it/s][A
99%|█████████▉| 191/192 [01:03<00:00, 3.80it/s][A
100%|██████████| 192/192 [01:03<00:00, 3.34it/s][A
[A 32%|███▏ | 300/936 [1:44:11<2:20:16, 13.23s/it]
100%|██████████| 192/192 [01:04<00:00, 3.34it/s][A
[Awandb: WARNING URL not available in offline run
`rope_scaling`'s beta_fast field must be a float, got 32
`rope_scaling`'s beta_slow field must be a float, got 1
32%|███▏ | 301/936 [1:44:30<6:02:49, 34.28s/it] 32%|███▏ | 302/936 [1:44:41<4:48:12, 27.27s/it] 32%|███▏ | 303/936 [1:44:53<3:59:32, 22.71s/it] 32%|███▏ | 304/936 [1:45:09<3:37:10, 20.62s/it] 33%|███▎ | 305/936 [1:45:51<4:43:50, 26.99s/it] 33%|███▎ | 305/936 [1:45:51<4:43:50, 26.99s/it] 33%|███▎ | 306/936 [1:46:28<5:15:30, 30.05s/it] 33%|███▎ | 307/936 [1:46:36<4:06:45, 23.54s/it] 33%|███▎ | 308/936 [1:46:52<3:41:19, 21.15s/it] 33%|███▎ | 309/936 [1:47:14<3:43:43, 21.41s/it] 33%|███▎ | 310/936 [1:47:32<3:33:09, 20.43s/it] 33%|███▎ | 310/936 [1:47:32<3:33:09, 20.43s/it] 33%|███▎ | 311/936 [1:47:49<3:22:45, 19.47s/it] 33%|███▎ | 312/936 [1:47:57<2:45:51, 15.95s/it] 33%|███▎ | 313/936 [1:48:20<3:08:00, 18.11s/it] 34%|███▎ | 314/936 [1:48:44<3:26:47, 19.95s/it] 34%|███▎ | 315/936 [1:49:05<3:30:08, 20.30s/it] 34%|███▎ | 315/936 [1:49:05<3:30:08, 20.30s/it]Terminated
`rope_scaling`'s beta_fast field must be a float, got 32
`rope_scaling`'s beta_slow field must be a float, got 1
`rope_scaling`'s beta_fast field must be a float, got 32
`rope_scaling`'s beta_slow field must be a float, got 1
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
⚙️ Running in WANDB offline mode
🦥 Unsloth Zoo will now patch everything to make training faster!
==((====))== Unsloth 2026.9.2: Fast Olmo3 patching. Transformers: 4.57.6.
\\ /| NVIDIA H100 80GB HBM3. Num GPUs = 1. Max memory: 79.179 GB. Platform: Linux.
O^O/ \_/ \ Torch: 2.11.0+cu128. CUDA: 9.0. CUDA Toolkit: 12.8. Triton: 3.6.0
\ / Bfloat16 = TRUE. FA [Xformers = 0.0.35. FA2 = False]
"-____-" Free license: http://github.com/unslothai/unsloth
Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
Unsloth: QLoRA and full finetuning all not selected. Switching to 16bit LoRA.
Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s]Loading checkpoint shards: 33%|███▎ | 1/3 [00:00<00:01, 1.39it/s]Loading checkpoint shards: 67%|██████▋ | 2/3 [00:01<00:00, 1.38it/s]Loading checkpoint shards: 100%|██████████| 3/3 [00:02<00:00, 1.42it/s]Loading checkpoint shards: 100%|██████████| 3/3 [00:02<00:00, 1.41it/s]
ROPE_GUARD OK train_unsloth load CompassioninMachineLearning/olmo-compassion-web-seed2-CPT-best: olmo3 theta=500000.0 scaling={'rope_type': 'yarn', 'factor': 8.0, 'original_max_position_embeddings': 8192.0, 'beta_fast': 32.0, 'beta_slow': 1.0, 'attention_factor': 1.2079441541679836}
Unsloth: Dropout = 0 is supported for fast patching. You are using dropout = 0.05.
Unsloth will patch all other layers, except LoRA matrices, causing a performance hit.
Unsloth: Explicit target_modules are constrained by the finetune_(vision|language|attention|mlp) filters; adapters attach only where both select.
Unsloth: Allowing gradients for `base_model.model.model.embed_tokens` since it's in `modules_to_save`.
Unsloth: Allowing gradients for `base_model.model.lm_head` since it's in `modules_to_save`.
Generating train split: 0 examples [00:00, ? examples/s]Generating train split: 890 examples [00:00, 5319.50 examples/s]Generating train split: 1781 examples [00:00, 2540.67 examples/s]Generating train split: 2643 examples [00:00, 3055.20 examples/s]Generating train split: 3587 examples [00:01, 3872.16 examples/s]Generating train split: 4465 examples [00:01, 3583.66 examples/s]Generating train split: 5410 examples [00:01, 2897.26 examples/s]Generating train split: 6313 examples [00:01, 3351.21 examples/s]Generating train split: 7214 examples [00:02, 3583.27 examples/s]Generating train split: 8108 examples [00:02, 3777.57 examples/s]Generating train split: 9017 examples [00:02, 2956.99 examples/s]Generating train split: 9904 examples [00:02, 3294.76 examples/s]Generating train split: 10797 examples [00:03, 3645.41 examples/s]Generating train split: 11743 examples [00:03, 2839.51 examples/s]Generating train split: 12646 examples [00:03, 3194.23 examples/s]Generating train split: 13586 examples [00:04, 3599.93 examples/s]Generating train split: 14510 examples [00:04, 3829.93 examples/s]Generating train split: 15424 examples [00:04, 3010.59 examples/s]Generating train split: 16339 examples [00:04, 3291.50 examples/s]Generating train split: 17261 examples [00:05, 3684.79 examples/s]Generating train split: 18165 examples [00:05, 2880.21 examples/s]Generating train split: 18500 examples [00:05, 3263.30 examples/s]
Generating train split: 0 examples [00:00, ? examples/s]Generating train split: 5000 examples [00:00, 8873.76 examples/s]Generating train split: 5000 examples [00:00, 8645.79 examples/s]
tokenizing (num_proc=8): 0%| | 0/23500 [00:00<?, ? examples/s]tokenizing (num_proc=8): 0%| | 10/23500 [00:00<24:15, 16.14 examples/s]tokenizing (num_proc=8): 0%| | 58/23500 [00:00<03:50, 101.91 examples/s]tokenizing (num_proc=8): 1%| | 136/23500 [00:00<01:37, 239.66 examples/s]tokenizing (num_proc=8): 1%| | 241/23500 [00:00<00:55, 419.15 examples/s]tokenizing (num_proc=8): 2%|▏ | 404/23500 [00:01<00:32, 706.85 examples/s]tokenizing (num_proc=8): 2%|▏ | 577/23500 [00:01<00:23, 959.96 examples/s]tokenizing (num_proc=8): 3%|▎ | 792/23500 [00:01<00:17, 1265.49 examples/s]tokenizing (num_proc=8): 4%|▍ | 996/23500 [00:01<00:15, 1447.71 examples/s]tokenizing (num_proc=8): 5%|▌ | 1193/23500 [00:01<00:14, 1536.99 examples/s]tokenizing (num_proc=8): 6%|▌ | 1377/23500 [00:01<00:14, 1559.21 examples/s]tokenizing (num_proc=8): 7%|▋ | 1579/23500 [00:01<00:13, 1676.85 examples/s]tokenizing (num_proc=8): 8%|▊ | 1781/23500 [00:01<00:12, 1745.30 examples/s]tokenizing (num_proc=8): 8%|▊ | 1967/23500 [00:01<00:12, 1746.78 examples/s]tokenizing (num_proc=8): 9%|▉ | 2149/23500 [00:02<00:12, 1733.47 examples/s]tokenizing (num_proc=8): 10%|▉ | 2330/23500 [00:02<00:12, 1737.12 examples/s]tokenizing (num_proc=8): 11%|█ | 2525/23500 [00:02<00:12, 1744.37 examples/s]tokenizing (num_proc=8): 12%|█▏ | 2703/23500 [00:02<00:12, 1673.21 examples/s]tokenizing (num_proc=8): 12%|█▏ | 2896/23500 [00:02<00:11, 1731.24 examples/s]tokenizing (num_proc=8): 13%|█▎ | 3113/23500 [00:02<00:11, 1788.93 examples/s]tokenizing (num_proc=8): 14%|█▍ | 3325/23500 [00:02<00:10, 1856.23 examples/s]tokenizing (num_proc=8): 15%|█▍ | 3518/23500 [00:02<00:10, 1829.49 examples/s]tokenizing (num_proc=8): 16%|█▌ | 3702/23500 [00:02<00:10, 1814.77 examples/s]tokenizing (num_proc=8): 17%|█▋ | 3900/23500 [00:02<00:10, 1843.77 examples/s]tokenizing (num_proc=8): 17%|█▋ | 4090/23500 [00:03<00:11, 1756.22 examples/s]tokenizing (num_proc=8): 18%|█▊ | 4268/23500 [00:03<00:10, 1751.36 examples/s]tokenizing (num_proc=8): 19%|█▉ | 4451/23500 [00:03<00:10, 1762.23 examples/s]tokenizing (num_proc=8): 20%|█▉ | 4650/23500 [00:03<00:10, 1804.58 examples/s]tokenizing (num_proc=8): 21%|██ | 4838/23500 [00:03<00:10, 1804.20 examples/s]tokenizing (num_proc=8): 21%|██▏ | 5050/23500 [00:03<00:09, 1880.47 examples/s]tokenizing (num_proc=8): 22%|██▏ | 5240/23500 [00:03<00:10, 1802.90 examples/s]tokenizing (num_proc=8): 23%|██▎ | 5426/23500 [00:03<00:10, 1779.91 examples/s]tokenizing (num_proc=8): 24%|██▍ | 5626/23500 [00:03<00:09, 1814.99 examples/s]tokenizing (num_proc=8): 25%|██▍ | 5815/23500 [00:04<00:09, 1799.52 examples/s]tokenizing (num_proc=8): 26%|██▌ | 6002/23500 [00:04<00:09, 1775.45 examples/s]tokenizing (num_proc=8): 26%|██▋ | 6186/23500 [00:04<00:09, 1746.92 examples/s]tokenizing (num_proc=8): 27%|██▋ | 6362/23500 [00:04<00:09, 1721.67 examples/s]tokenizing (num_proc=8): 28%|██▊ | 6536/23500 [00:04<00:10, 1681.44 examples/s]tokenizing (num_proc=8): 29%|██▊ | 6707/23500 [00:04<00:10, 1670.17 examples/s]tokenizing (num_proc=8): 29%|██▉ | 6878/23500 [00:04<00:10, 1656.82 examples/s]tokenizing (num_proc=8): 30%|██▉ | 7045/23500 [00:04<00:10, 1640.12 examples/s]tokenizing (num_proc=8): 31%|███ | 7220/23500 [00:04<00:11, 1479.40 examples/s]tokenizing (num_proc=8): 31%|███▏ | 7383/23500 [00:05<00:11, 1442.23 examples/s]tokenizing (num_proc=8): 32%|███▏ | 7534/23500 [00:05<00:12, 1327.70 examples/s]tokenizing (num_proc=8): 33%|███▎ | 7673/23500 [00:05<00:13, 1193.50 examples/s]tokenizing (num_proc=8): 33%|███▎ | 7803/23500 [00:05<00:15, 1034.42 examples/s]tokenizing (num_proc=8): 34%|███▎ | 7924/23500 [00:05<00:15, 1030.30 examples/s]tokenizing (num_proc=8): 34%|███▍ | 8039/23500 [00:05<00:16, 920.21 examples/s] tokenizing (num_proc=8): 35%|███▍ | 8148/23500 [00:05<00:16, 925.50 examples/s]tokenizing (num_proc=8): 35%|███▌ | 8248/23500 [00:06<00:16, 941.37 examples/s]tokenizing (num_proc=8): 36%|███▌ | 8375/23500 [00:06<00:14, 1016.54 examples/s]tokenizing (num_proc=8): 36%|███▌ | 8514/23500 [00:06<00:13, 1092.91 examples/s]tokenizing (num_proc=8): 37%|███▋ | 8674/23500 [00:06<00:12, 1200.73 examples/s]tokenizing (num_proc=8): 38%|███▊ | 8836/23500 [00:06<00:11, 1289.74 examples/s]tokenizing (num_proc=8): 38%|███▊ | 9019/23500 [00:06<00:10, 1390.90 examples/s]tokenizing (num_proc=8): 39%|███▉ | 9211/23500 [00:06<00:09, 1509.22 examples/s]tokenizing (num_proc=8): 40%|███▉ | 9391/23500 [00:06<00:09, 1561.49 examples/s]tokenizing (num_proc=8): 41%|████ | 9583/23500 [00:06<00:08, 1629.02 examples/s]tokenizing (num_proc=8): 42%|████▏ | 9775/23500 [00:06<00:08, 1679.28 examples/s]tokenizing (num_proc=8): 42%|████▏ | 9980/23500 [00:07<00:07, 1730.40 examples/s]tokenizing (num_proc=8): 43%|████▎ | 10164/23500 [00:07<00:07, 1721.04 examples/s]tokenizing (num_proc=8): 44%|████▍ | 10387/23500 [00:07<00:07, 1849.20 examples/s]tokenizing (num_proc=8): 45%|████▌ | 10591/23500 [00:07<00:06, 1896.27 examples/s]tokenizing (num_proc=8): 46%|████▌ | 10795/23500 [00:07<00:06, 1839.97 examples/s]tokenizing (num_proc=8): 47%|████▋ | 10998/23500 [00:07<00:06, 1876.24 examples/s]tokenizing (num_proc=8): 48%|████▊ | 11187/23500 [00:07<00:06, 1862.64 examples/s]tokenizing (num_proc=8): 48%|████▊ | 11377/23500 [00:07<00:06, 1847.88 examples/s]tokenizing (num_proc=8): 49%|████▉ | 11573/23500 [00:07<00:06, 1758.23 examples/s]tokenizing (num_proc=8): 50%|█████ | 11774/23500 [00:08<00:06, 1813.85 examples/s]tokenizing (num_proc=8): 51%|█████ | 11964/23500 [00:08<00:06, 1781.53 examples/s]tokenizing (num_proc=8): 52%|█████▏ | 12149/23500 [00:08<00:06, 1787.83 examples/s]tokenizing (num_proc=8): 53%|█████▎ | 12349/23500 [00:08<00:06, 1791.87 examples/s]tokenizing (num_proc=8): 53%|█████▎ | 12539/23500 [00:08<00:06, 1804.79 examples/s]tokenizing (num_proc=8): 54%|█████▍ | 12720/23500 [00:08<00:06, 1741.90 examples/s]tokenizing (num_proc=8): 55%|█████▌ | 12945/23500 [00:08<00:05, 1843.07 examples/s]tokenizing (num_proc=8): 56%|█████▌ | 13149/23500 [00:08<00:05, 1862.13 examples/s]tokenizing (num_proc=8): 57%|█████▋ | 13350/23500 [00:08<00:05, 1882.77 examples/s]tokenizing (num_proc=8): 58%|█████▊ | 13550/23500 [00:09<00:05, 1869.50 examples/s]tokenizing (num_proc=8): 59%|█████▊ | 13750/23500 [00:09<00:05, 1831.79 examples/s]tokenizing (num_proc=8): 59%|█████▉ | 13947/23500 [00:09<00:05, 1817.30 examples/s]tokenizing (num_proc=8): 60%|██████ | 14148/23500 [00:09<00:05, 1862.38 examples/s]tokenizing (num_proc=8): 61%|██████ | 14343/23500 [00:09<00:05, 1744.44 examples/s]tokenizing (num_proc=8): 62%|██████▏ | 14529/23500 [00:09<00:05, 1740.88 examples/s]tokenizing (num_proc=8): 63%|██████▎ | 14718/23500 [00:09<00:05, 1746.84 examples/s]tokenizing (num_proc=8): 63%|██████▎ | 14904/23500 [00:09<00:05, 1701.03 examples/s]tokenizing (num_proc=8): 64%|██████▍ | 15076/23500 [00:09<00:05, 1621.07 examples/s]tokenizing (num_proc=8): 65%|██████▍ | 15247/23500 [00:10<00:05, 1608.35 examples/s]tokenizing (num_proc=8): 66%|██████▌ | 15416/23500 [00:10<00:05, 1547.45 examples/s]tokenizing (num_proc=8): 66%|██████▋ | 15572/23500 [00:10<00:05, 1499.76 examples/s]tokenizing (num_proc=8): 67%|██████▋ | 15746/23500 [00:10<00:04, 1552.41 examples/s]tokenizing (num_proc=8): 68%|██████▊ | 15906/23500 [00:10<00:05, 1489.24 examples/s]tokenizing (num_proc=8): 68%|██████▊ | 16063/23500 [00:10<00:05, 1374.94 examples/s]tokenizing (num_proc=8): 69%|██████▉ | 16206/23500 [00:10<00:05, 1329.77 examples/s]tokenizing (num_proc=8): 70%|██████▉ | 16353/23500 [00:10<00:05, 1314.97 examples/s]tokenizing (num_proc=8): 70%|███████ | 16563/23500 [00:10<00:04, 1496.21 examples/s]tokenizing (num_proc=8): 71%|███████▏ | 16790/23500 [00:11<00:04, 1664.04 examples/s]tokenizing (num_proc=8): 72%|███████▏ | 16972/23500 [00:11<00:03, 1703.02 examples/s]tokenizing (num_proc=8): 73%|███████▎ | 17155/23500 [00:11<00:03, 1684.34 examples/s]tokenizing (num_proc=8): 74%|███████▎ | 17325/23500 [00:11<00:03, 1660.20 examples/s]tokenizing (num_proc=8): 74%|███████▍ | 17494/23500 [00:11<00:03, 1646.05 examples/s]tokenizing (num_proc=8): 75%|███████▌ | 17661/23500 [00:11<00:03, 1640.22 examples/s]tokenizing (num_proc=8): 76%|███████▌ | 17857/23500 [00:11<00:03, 1719.45 examples/s]tokenizing (num_proc=8): 77%|███████▋ | 18093/23500 [00:11<00:02, 1887.91 examples/s]tokenizing (num_proc=8): 78%|███████▊ | 18284/23500 [00:11<00:02, 1869.45 examples/s]tokenizing (num_proc=8): 79%|███████▊ | 18481/23500 [00:12<00:02, 1873.78 examples/s]tokenizing (num_proc=8): 79%|███████▉ | 18672/23500 [00:12<00:02, 1864.89 examples/s]tokenizing (num_proc=8): 80%|████████ | 18864/23500 [00:12<00:02, 1799.66 examples/s]tokenizing (num_proc=8): 81%|████████ | 19061/23500 [00:12<00:02, 1839.68 examples/s]tokenizing (num_proc=8): 82%|████████▏ | 19256/23500 [00:12<00:02, 1774.05 examples/s]tokenizing (num_proc=8): 83%|████████▎ | 19441/23500 [00:12<00:02, 1794.71 examples/s]tokenizing (num_proc=8): 84%|████████▎ | 19643/23500 [00:12<00:02, 1854.12 examples/s]tokenizing (num_proc=8): 85%|████████▍ | 19866/23500 [00:12<00:01, 1950.55 examples/s]tokenizing (num_proc=8): 85%|████████▌ | 20064/23500 [00:12<00:01, 1915.38 examples/s]tokenizing (num_proc=8): 86%|████████▌ | 20265/23500 [00:12<00:01, 1921.94 examples/s]tokenizing (num_proc=8): 87%|████████▋ | 20461/23500 [00:13<00:01, 1883.27 examples/s]tokenizing (num_proc=8): 88%|████████▊ | 20664/23500 [00:13<00:01, 1905.08 examples/s]tokenizing (num_proc=8): 89%|████████▉ | 20863/23500 [00:13<00:01, 1885.52 examples/s]tokenizing (num_proc=8): 90%|████████▉ | 21058/23500 [00:13<00:01, 1838.83 examples/s]tokenizing (num_proc=8): 90%|█████████ | 21256/23500 [00:13<00:01, 1860.24 examples/s]tokenizing (num_proc=8): 91%|█████████▏| 21460/23500 [00:13<00:01, 1902.17 examples/s]tokenizing (num_proc=8): 92%|█████████▏| 21651/23500 [00:13<00:01, 1820.95 examples/s]tokenizing (num_proc=8): 93%|█████████▎| 21843/23500 [00:13<00:00, 1694.84 examples/s]tokenizing (num_proc=8): 94%|█████████▍| 22032/23500 [00:13<00:00, 1691.06 examples/s]tokenizing (num_proc=8): 95%|█████████▍| 22223/23500 [00:14<00:00, 1744.39 examples/s]tokenizing (num_proc=8): 95%|█████████▌| 22400/23500 [00:14<00:00, 1686.33 examples/s]tokenizing (num_proc=8): 96%|█████████▌| 22576/23500 [00:14<00:00, 1686.92 examples/s]tokenizing (num_proc=8): 97%|█████████▋| 22751/23500 [00:14<00:00, 1545.13 examples/s]tokenizing (num_proc=8): 98%|█████████▊| 22925/23500 [00:14<00:00, 1476.08 examples/s]tokenizing (num_proc=8): 98%|█████████▊| 23076/23500 [00:14<00:00, 1458.47 examples/s]tokenizing (num_proc=8): 99%|█████████▉| 23229/23500 [00:14<00:00, 1227.95 examples/s]tokenizing (num_proc=8): 99%|█████████▉| 23361/23500 [00:15<00:00, 1092.76 examples/s]tokenizing (num_proc=8): 100%|█████████▉| 23483/23500 [00:15<00:00, 814.35 examples/s] tokenizing (num_proc=8): 100%|██████████| 23500/23500 [00:15<00:00, 1491.48 examples/s]
Filter: 0%| | 0/23500 [00:00<?, ? examples/s]Filter: 4%|▍ | 1000/23500 [00:00<00:04, 4784.37 examples/s]Filter: 9%|▊ | 2000/23500 [00:00<00:03, 5445.31 examples/s]Filter: 13%|█▎ | 3000/23500 [00:00<00:03, 5588.24 examples/s]Filter: 17%|█▋ | 4000/23500 [00:00<00:03, 5908.47 examples/s]Filter: 21%|██▏ | 5000/23500 [00:00<00:03, 6121.82 examples/s]Filter: 26%|██▌ | 6000/23500 [00:01<00:02, 6368.29 examples/s]Filter: 30%|██▉ | 7000/23500 [00:01<00:02, 6473.33 examples/s]Filter: 34%|███▍ | 8000/23500 [00:01<00:02, 6452.18 examples/s]Filter: 38%|███▊ | 9000/23500 [00:01<00:02, 6569.25 examples/s]Filter: 43%|████▎ | 10000/23500 [00:01<00:02, 6589.54 examples/s]Filter: 47%|████▋ | 11000/23500 [00:01<00:01, 6655.93 examples/s]Filter: 51%|█████ | 12000/23500 [00:01<00:01, 6528.26 examples/s]Filter: 55%|█████▌ | 13000/23500 [00:02<00:01, 6549.41 examples/s]Filter: 60%|█████▉ | 14000/23500 [00:02<00:01, 6373.99 examples/s]Filter: 64%|██████▍ | 15000/23500 [00:02<00:01, 6471.53 examples/s]Filter: 68%|██████▊ | 16000/23500 [00:02<00:01, 6454.40 examples/s]Filter: 72%|███████▏ | 17000/23500 [00:02<00:01, 6447.25 examples/s]Filter: 77%|███████▋ | 18000/23500 [00:02<00:00, 6594.78 examples/s]Filter: 81%|████████ | 19000/23500 [00:02<00:00, 6550.96 examples/s]Filter: 85%|████████▌ | 20000/23500 [00:03<00:00, 6608.87 examples/s]Filter: 89%|████████▉ | 21000/23500 [00:03<00:00, 6603.36 examples/s]Filter: 94%|█████████▎| 22000/23500 [00:03<00:00, 6578.00 examples/s]Filter: 98%|█████████▊| 23000/23500 [00:03<00:00, 6622.20 examples/s]Filter: 100%|██████████| 23500/23500 [00:03<00:00, 6408.46 examples/s]
Filter: 0%| | 0/23500 [00:00<?, ? examples/s]Filter: 4%|▍ | 1000/23500 [00:00<00:03, 6425.12 examples/s]Filter: 9%|▊ | 2000/23500 [00:00<00:03, 6171.01 examples/s]Filter: 13%|█▎ | 3000/23500 [00:00<00:03, 5854.18 examples/s]Filter: 17%|█▋ | 4000/23500 [00:00<00:03, 5976.34 examples/s]Filter: 21%|██▏ | 5000/23500 [00:00<00:03, 5936.87 examples/s]Filter: 26%|██▌ | 6000/23500 [00:00<00:02, 6106.53 examples/s]Filter: 30%|██▉ | 7000/23500 [00:01<00:02, 6146.79 examples/s]Filter: 34%|███▍ | 8000/23500 [00:01<00:02, 5968.91 examples/s]Filter: 38%|███▊ | 9000/23500 [00:01<00:02, 6090.12 examples/s]Filter: 43%|████▎ | 10000/23500 [00:01<00:02, 5865.71 examples/s]Filter: 47%|████▋ | 11000/23500 [00:01<00:02, 6009.88 examples/s]Filter: 51%|█████ | 12000/23500 [00:01<00:01, 5981.25 examples/s]Filter: 55%|█████▌ | 13000/23500 [00:02<00:01, 6001.61 examples/s]Filter: 60%|█████▉ | 14000/23500 [00:02<00:01, 5983.39 examples/s]Filter: 64%|██████▍ | 15000/23500 [00:02<00:01, 5792.35 examples/s]Filter: 68%|██████▊ | 16000/23500 [00:02<00:01, 5833.27 examples/s]Filter: 72%|███████▏ | 17000/23500 [00:02<00:01, 5834.18 examples/s]Filter: 77%|███████▋ | 18000/23500 [00:03<00:00, 6012.67 examples/s]Filter: 81%|████████ | 19000/23500 [00:03<00:00, 5998.50 examples/s]Filter: 85%|████████▌ | 20000/23500 [00:03<00:00, 5932.09 examples/s]Filter: 89%|████████▉ | 21000/23500 [00:03<00:00, 6032.11 examples/s]Filter: 94%|█████████▎| 22000/23500 [00:03<00:00, 5791.11 examples/s]Filter: 98%|█████████▊| 23000/23500 [00:03<00:00, 5951.17 examples/s]Filter: 100%|██████████| 23500/23500 [00:03<00:00, 5960.52 examples/s]
supervised tokens: tool 4221981, replay 10741108 (71.8% replay)
replay capped to 25% of supervised tokens: kept 1406715 replay tokens, 1209 replay rows
rows: 23500 raw -> 15347 kept; by source: {'recovery': 3750, 'xlam': 5000, 'replay:compassion_12185_cleaned': 1209, 'apigen': 4888, 'efficiency': 500}
[manifest] git=534143b028a9765e904b4d6d1b22205027804f19 data_sha256={'train_olmo-compassion-web-s2_replay50.jsonl': 'b43d87068505', 'xlam5000_max1call.jsonl': 'a8aae6cf5c36'}
wandb: WARNING git root /workspace/anchor/olmo-compassion-web-s2/wandb does not exist
wandb: Tracking run with wandb version 0.30.0
wandb: W&B syncing is set to `offline` in this directory. Run `wandb online` or set WANDB_MODE=online to enable cloud syncing.
wandb: Run data is saved locally in /workspace/anchor/olmo-compassion-web-s2/wandb/wandb/offline-run-20261016_031236-b77g3dzm
wandb: View this run in the terminal with `wandb leet`
wandb: Detected [huggingface_hub.inference] in use.
wandb: Use W&B Weave for improved LLM call tracing. Install Weave with `pip install weave` then add `import weave` to the top of your script.
wandb: For more information, check out the docs at: https://weave-docs.wandb.ai
==((====))== Unsloth - 2x faster free finetuning | Num GPUs used = 1
\\ /| Num examples = 14,963 | Num Epochs = 1 | Total steps = 936
O^O/ \_/ \ Batch size per device = 2 | Gradient accumulation steps = 8
\ / Data Parallel GPUs = 1 | Total batch size (2 x 8 x 1) = 16
"-____-" Trainable parameters = 981,385,216 of 8,279,396,352 (11.85% trained)
Unsloth: Setting lr = 2.00e-05 instead of 2.00e-04 for embed_tokens.
Unsloth: Setting lr = 2.00e-05 instead of 2.00e-04 for lm_head.
0%| | 0/936 [00:00<?, ?it/s] 0%| | 1/936 [00:46<12:02:54, 46.39s/it] 0%| | 2/936 [01:08<8:23:04, 32.32s/it] 0%| | 3/936 [01:32<7:19:53, 28.29s/it] 0%| | 4/936 [01:59<7:11:13, 27.76s/it] 1%| | 5/936 [02:18<6:23:36, 24.72s/it] 1%| | 5/936 [02:18<6:23:36, 24.72s/it] 1%| | 6/936 [02:42<6:20:15, 24.53s/it] 1%| | 7/936 [03:06<6:17:27, 24.38s/it] 1%| | 8/936 [03:29<6:06:23, 23.69s/it] 1%| | 9/936 [03:54<6:12:09, 24.09s/it] 1%| | 10/936 [04:20<6:23:27, 24.85s/it] 1%| | 10/936 [04:20<6:23:27, 24.85s/it] 1%| | 11/936 [04:47<6:33:18, 25.51s/it] 1%|▏ | 12/936 [05:08<6:10:41, 24.07s/it] 1%|▏ | 13/936 [05:28<5:53:17, 22.97s/it] 1%|▏ | 14/936 [05:50<5:45:10, 22.46s/it] 2%|▏ | 15/936 [06:10<5:36:18, 21.91s/it] 2%|▏ | 15/936 [06:10<5:36:18, 21.91s/it] 2%|▏ | 16/936 [06:29<5:22:24, 21.03s/it] 2%|▏ | 17/936 [06:54<5:39:32, 22.17s/it] 2%|▏ | 18/936 [07:15<5:32:15, 21.72s/it] 2%|▏ | 19/936 [07:35<5:26:45, 21.38s/it] 2%|▏ | 20/936 [07:59<5:37:14, 22.09s/it] 2%|▏ | 20/936 [07:59<5:37:14, 22.09s/it] 2%|▏ | 21/936 [08:19<5:28:37, 21.55s/it] 2%|▏ | 22/936 [08:43<5:38:20, 22.21s/it] 2%|▏ | 23/936 [09:08<5:48:53, 22.93s/it] 3%|▎ | 24/936 [09:26<5:27:09, 21.52s/it] 3%|▎ | 25/936 [09:52<5:47:40, 22.90s/it] 3%|▎ | 25/936 [09:52<5:47:40, 22.90s/it] 3%|▎ | 26/936 [10:16<5:52:29, 23.24s/it] 3%|▎ | 27/936 [10:40<5:54:19, 23.39s/it] 3%|▎ | 28/936 [11:01<5:45:12, 22.81s/it] 3%|▎ | 29/936 [11:25<5:46:56, 22.95s/it] 3%|▎ | 30/936 [11:48<5:49:09, 23.12s/it] 3%|▎ | 30/936 [11:48<5:49:09, 23.12s/it] 3%|▎ | 31/936 [12:09<5:39:52, 22.53s/it] 3%|▎ | 32/936 [12:36<5:57:27, 23.72s/it] 4%|▎ | 33/936 [12:55<5:35:50, 22.31s/it] 4%|▎ | 34/936 [13:20<5:46:39, 23.06s/it] 4%|▎ | 35/936 [13:44<5:50:35, 23.35s/it] 4%|▎ | 35/936 [13:44<5:50:35, 23.35s/it] 4%|▍ | 36/936 [14:08<5:54:31, 23.64s/it] 4%|▍ | 37/936 [14:30<5:46:50, 23.15s/it] 4%|▍ | 38/936 [14:49<5:27:24, 21.88s/it] 4%|▍ | 39/936 [15:11<5:29:39, 22.05s/it] 4%|▍ | 40/936 [15:36<5:41:10, 22.85s/it] 4%|▍ | 40/936 [15:36<5:41:10, 22.85s/it] 4%|▍ | 41/936 [15:56<5:27:25, 21.95s/it] 4%|▍ | 42/936 [16:19<5:31:12, 22.23s/it] 5%|▍ | 43/936 [16:37<5:15:30, 21.20s/it] 5%|▍ | 44/936 [17:01<5:23:53, 21.79s/it] 5%|▍ | 45/936 [17:21<5:16:05, 21.29s/it] 5%|▍ | 45/936 [17:21<5:16:05, 21.29s/it] 5%|▍ | 46/936 [17:45<5:30:27, 22.28s/it] 5%|▌ | 47/936 [18:04<5:13:14, 21.14s/it] 5%|▌ | 48/936 [18:25<5:12:50, 21.14s/it] 5%|▌ | 49/936 [18:43<4:58:09, 20.17s/it] 5%|▌ | 50/936 [19:02<4:54:46, 19.96s/it] 5%|▌ | 50/936 [19:02<4:54:46, 19.96s/it]Unsloth: Will smartly offload gradients to save VRAM!
Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
{'loss': 1.1207, 'grad_norm': 1.163063645362854, 'learning_rate': 1.7021276595744682e-05, 'epoch': 0.01}
{'loss': 0.9711, 'grad_norm': 1.4067025184631348, 'learning_rate': 3.829787234042553e-05, 'epoch': 0.01}
{'loss': 0.8716, 'grad_norm': 0.8022639155387878, 'learning_rate': 5.9574468085106384e-05, 'epoch': 0.02}
{'loss': 0.6679, 'grad_norm': 1.227128267288208, 'learning_rate': 8.085106382978723e-05, 'epoch': 0.02}
{'loss': 0.6026, 'grad_norm': 1.3577823638916016, 'learning_rate': 0.00010212765957446809, 'epoch': 0.03}
{'loss': 0.7036, 'grad_norm': 1.6052050590515137, 'learning_rate': 0.00012340425531914893, 'epoch': 0.03}
{'loss': 0.5792, 'grad_norm': 1.4057849645614624, 'learning_rate': 0.0001446808510638298, 'epoch': 0.04}
{'loss': 0.5134, 'grad_norm': 1.3657870292663574, 'learning_rate': 0.00016595744680851065, 'epoch': 0.04}
{'loss': 0.5381, 'grad_norm': 1.0391675233840942, 'learning_rate': 0.0001872340425531915, 'epoch': 0.05}
{'loss': 0.5001, 'grad_norm': 0.9683766961097717, 'learning_rate': 0.00019999750239122662, 'epoch': 0.05}
Filter: 0%| | 0/384 [00:00<?, ? examples/s][A
Filter: 100%|██████████| 384/384 [00:00<00:00, 1437.93 examples/s][AFilter: 100%|██████████| 384/384 [00:00<00:00, 1425.00 examples/s]
0%| | 0/192 [00:00<?, ?it/s][A
1%| | 2/192 [00:00<01:32, 2.04it/s][A
2%|▏ | 3/192 [00:02<02:27, 1.28it/s][A
2%|▏ | 4/192 [00:03<02:33, 1.23it/s][A
3%|▎ | 5/192 [00:03<02:23, 1.30it/s][A
3%|▎ | 6/192 [00:03<01:51, 1.67it/s][A
4%|▎ | 7/192 [00:04<02:02, 1.51it/s][A
4%|▍ | 8/192 [00:05<02:05, 1.47it/s][A
5%|▍ | 9/192 [00:05<01:39, 1.84it/s][A
5%|▌ | 10/192 [00:06<02:09, 1.41it/s][A
6%|▌ | 11/192 [00:07<02:25, 1.24it/s][A
6%|▋ | 12/192 [00:08<02:34, 1.17it/s][A
7%|▋ | 13/192 [00:09<02:32, 1.17it/s][A
7%|▋ | 14/192 [00:09<02:01, 1.47it/s][A
8%|▊ | 15/192 [00:10<02:13, 1.33it/s][A
8%|▊ | 16/192 [00:11<02:05, 1.40it/s][A
9%|▉ | 17/192 [00:11<01:45, 1.66it/s][A
9%|▉ | 18/192 [00:13<02:15, 1.29it/s][A
10%|▉ | 19/192 [00:14<02:27, 1.17it/s][A
10%|█ | 20/192 [00:15<02:51, 1.00it/s][A
11%|█ | 21/192 [00:16<02:59, 1.05s/it][A
12%|█▏ | 23/192 [00:17<02:23, 1.18it/s][A
12%|█▎ | 24/192 [00:18<02:32, 1.10it/s][A
13%|█▎ | 25/192 [00:19<02:19, 1.20it/s][A
14%|█▎ | 26/192 [00:20<02:17, 1.21it/s][A
14%|█▍ | 27/192 [00:20<02:08, 1.29it/s][A
15%|█▍ | 28/192 [00:21<02:16, 1.20it/s][A
15%|█▌ | 29/192 [00:22<01:48, 1.50it/s][A
16%|█▌ | 30/192 [00:22<01:32, 1.76it/s][A
16%|█▌ | 31/192 [00:23<01:56, 1.38it/s][A
17%|█▋ | 32/192 [00:24<01:59, 1.34it/s][A
18%|█▊ | 34/192 [00:25<01:43, 1.52it/s][A
18%|█▊ | 35/192 [00:26<01:54, 1.38it/s][A
19%|█▉ | 36/192 [00:27<02:15, 1.15it/s][A
19%|█▉ | 37/192 [00:28<02:30, 1.03it/s][A
20%|█▉ | 38/192 [00:29<02:29, 1.03it/s][A
20%|██ | 39/192 [00:30<02:21, 1.08it/s][A
21%|██ | 40/192 [00:31<02:13, 1.13it/s][A
22%|██▏ | 42/192 [00:31<01:26, 1.74it/s][A