File size: 112,725 Bytes
22e5596 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 | `rope_scaling`'s beta_fast field must be a float, got 32
`rope_scaling`'s beta_slow field must be a float, got 1
`rope_scaling`'s beta_fast field must be a float, got 32
`rope_scaling`'s beta_slow field must be a float, got 1
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
⚙️ Running in WANDB offline mode
🦥 Unsloth Zoo will now patch everything to make training faster!
==((====))== Unsloth 2026.9.2: Fast Olmo3 patching. Transformers: 4.57.6.
\\ /| NVIDIA H100 80GB HBM3. Num GPUs = 1. Max memory: 79.179 GB. Platform: Linux.
O^O/ \_/ \ Torch: 2.11.0+cu128. CUDA: 9.0. CUDA Toolkit: 12.8. Triton: 3.6.0
\ / Bfloat16 = TRUE. FA [Xformers = 0.0.35. FA2 = False]
"-____-" Free license: http://github.com/unslothai/unsloth
Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!
Unsloth: QLoRA and full finetuning all not selected. Switching to 16bit LoRA.
Loading checkpoint shards: 0%| | 0/3 [00:00<?, ?it/s]
Loading checkpoint shards: 33%|███▎ | 1/3 [00:00<00:01, 1.13it/s]
Loading checkpoint shards: 67%|██████▋ | 2/3 [00:01<00:00, 1.15it/s]
Loading checkpoint shards: 100%|██████████| 3/3 [00:02<00:00, 1.20it/s]
Loading checkpoint shards: 100%|██████████| 3/3 [00:02<00:00, 1.19it/s]
ROPE_GUARD OK train_unsloth load allenai/Olmo-3-1025-7B: olmo3 theta=500000.0 scaling={'rope_type': 'yarn', 'factor': 8.0, 'original_max_position_embeddings': 8192.0, 'beta_fast': 32.0, 'beta_slow': 1.0, 'attention_factor': 1.2079441541679836}
Unsloth: Dropout = 0 is supported for fast patching. You are using dropout = 0.05.
Unsloth will patch all other layers, except LoRA matrices, causing a performance hit.
Unsloth: Explicit target_modules are constrained by the finetune_(vision|language|attention|mlp) filters; adapters attach only where both select.
Unsloth: Allowing gradients for `base_model.model.model.embed_tokens` since it's in `modules_to_save`.
Unsloth: Allowing gradients for `base_model.model.lm_head` since it's in `modules_to_save`.
Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 666 examples [00:00, 1082.16 examples/s]
Generating train split: 1308 examples [00:00, 1707.96 examples/s]
Generating train split: 1991 examples [00:01, 2054.25 examples/s]
Generating train split: 2648 examples [00:01, 1733.29 examples/s]
Generating train split: 3299 examples [00:01, 1995.85 examples/s]
Generating train split: 3983 examples [00:02, 1680.16 examples/s]
Generating train split: 4619 examples [00:02, 1895.30 examples/s]
Generating train split: 5267 examples [00:02, 2111.71 examples/s]
Generating train split: 5946 examples [00:03, 1816.35 examples/s]
Generating train split: 6655 examples [00:03, 2029.22 examples/s]
Generating train split: 7317 examples [00:04, 1712.43 examples/s]
Generating train split: 7996 examples [00:04, 1933.20 examples/s]
Generating train split: 8636 examples [00:04, 2050.00 examples/s]
Generating train split: 9250 examples [00:05, 1705.56 examples/s]
Generating train split: 9250 examples [00:05, 1810.59 examples/s]
Generating train split: 0 examples [00:00, ? examples/s]
Generating train split: 5000 examples [00:00, 8010.55 examples/s]
Generating train split: 5000 examples [00:00, 7809.35 examples/s]
tokenizing (num_proc=8): 0%| | 0/14250 [00:00<?, ? examples/s]
tokenizing (num_proc=8): 0%| | 7/14250 [00:00<20:37, 11.51 examples/s]
tokenizing (num_proc=8): 0%| | 43/14250 [00:00<03:04, 77.13 examples/s]
tokenizing (num_proc=8): 1%| | 115/14250 [00:00<01:06, 213.11 examples/s]
tokenizing (num_proc=8): 2%|▏ | 254/14250 [00:00<00:29, 472.41 examples/s]
tokenizing (num_proc=8): 3%|▎ | 426/14250 [00:01<00:17, 772.42 examples/s]
tokenizing (num_proc=8): 4%|▍ | 581/14250 [00:01<00:14, 971.30 examples/s]
tokenizing (num_proc=8): 5%|▌ | 767/14250 [00:01<00:11, 1185.65 examples/s]
tokenizing (num_proc=8): 7%|▋ | 957/14250 [00:01<00:09, 1363.39 examples/s]
tokenizing (num_proc=8): 8%|▊ | 1138/14250 [00:01<00:08, 1464.92 examples/s]
tokenizing (num_proc=8): 9%|▉ | 1298/14250 [00:01<00:08, 1462.99 examples/s]
tokenizing (num_proc=8): 10%|█ | 1485/14250 [00:01<00:08, 1536.51 examples/s]
tokenizing (num_proc=8): 12%|█▏ | 1654/14250 [00:01<00:08, 1510.57 examples/s]
tokenizing (num_proc=8): 13%|█▎ | 1811/14250 [00:01<00:08, 1505.85 examples/s]
tokenizing (num_proc=8): 14%|█▍ | 2009/14250 [00:01<00:07, 1623.43 examples/s]
tokenizing (num_proc=8): 15%|█▌ | 2184/14250 [00:02<00:07, 1642.48 examples/s]
tokenizing (num_proc=8): 17%|█▋ | 2370/14250 [00:02<00:06, 1701.44 examples/s]
tokenizing (num_proc=8): 18%|█▊ | 2544/14250 [00:02<00:07, 1585.09 examples/s]
tokenizing (num_proc=8): 19%|█▉ | 2737/14250 [00:02<00:07, 1627.67 examples/s]
tokenizing (num_proc=8): 20%|██ | 2917/14250 [00:02<00:06, 1654.15 examples/s]
tokenizing (num_proc=8): 22%|██▏ | 3100/14250 [00:02<00:06, 1665.20 examples/s]
tokenizing (num_proc=8): 23%|██▎ | 3268/14250 [00:02<00:06, 1602.08 examples/s]
tokenizing (num_proc=8): 24%|██▍ | 3469/14250 [00:02<00:06, 1690.23 examples/s]
tokenizing (num_proc=8): 26%|██▌ | 3650/14250 [00:02<00:06, 1685.48 examples/s]
tokenizing (num_proc=8): 27%|██▋ | 3826/14250 [00:03<00:06, 1647.17 examples/s]
tokenizing (num_proc=8): 28%|██▊ | 3997/14250 [00:03<00:06, 1577.13 examples/s]
tokenizing (num_proc=8): 29%|██▉ | 4157/14250 [00:03<00:06, 1558.27 examples/s]
tokenizing (num_proc=8): 30%|███ | 4320/14250 [00:03<00:06, 1520.91 examples/s]
tokenizing (num_proc=8): 31%|███▏ | 4481/14250 [00:03<00:06, 1544.44 examples/s]
tokenizing (num_proc=8): 33%|███▎ | 4636/14250 [00:03<00:06, 1535.47 examples/s]
tokenizing (num_proc=8): 34%|███▎ | 4793/14250 [00:03<00:06, 1544.38 examples/s]
tokenizing (num_proc=8): 35%|███▍ | 4970/14250 [00:03<00:05, 1578.49 examples/s]
tokenizing (num_proc=8): 36%|███▌ | 5134/14250 [00:03<00:05, 1551.05 examples/s]
tokenizing (num_proc=8): 37%|███▋ | 5342/14250 [00:04<00:05, 1660.28 examples/s]
tokenizing (num_proc=8): 39%|███▊ | 5509/14250 [00:04<00:05, 1583.78 examples/s]
tokenizing (num_proc=8): 40%|███▉ | 5669/14250 [00:04<00:05, 1527.58 examples/s]
tokenizing (num_proc=8): 41%|████ | 5853/14250 [00:04<00:05, 1592.46 examples/s]
tokenizing (num_proc=8): 42%|████▏ | 6025/14250 [00:04<00:05, 1539.09 examples/s]
tokenizing (num_proc=8): 43%|████▎ | 6188/14250 [00:04<00:05, 1521.44 examples/s]
tokenizing (num_proc=8): 45%|████▍ | 6348/14250 [00:04<00:05, 1494.84 examples/s]
tokenizing (num_proc=8): 46%|████▌ | 6501/14250 [00:04<00:05, 1491.33 examples/s]
tokenizing (num_proc=8): 47%|████▋ | 6667/14250 [00:04<00:05, 1501.41 examples/s]
tokenizing (num_proc=8): 48%|████▊ | 6824/14250 [00:05<00:04, 1499.45 examples/s]
tokenizing (num_proc=8): 49%|████▉ | 6979/14250 [00:05<00:05, 1448.80 examples/s]
tokenizing (num_proc=8): 50%|█████ | 7130/14250 [00:05<00:04, 1429.87 examples/s]
tokenizing (num_proc=8): 51%|█████ | 7300/14250 [00:05<00:04, 1486.08 examples/s]
tokenizing (num_proc=8): 53%|█████▎ | 7482/14250 [00:05<00:04, 1554.99 examples/s]
tokenizing (num_proc=8): 54%|█████▎ | 7640/14250 [00:05<00:04, 1410.70 examples/s]
tokenizing (num_proc=8): 55%|█████▍ | 7791/14250 [00:05<00:05, 1227.83 examples/s]
tokenizing (num_proc=8): 56%|█████▌ | 7925/14250 [00:06<00:06, 983.42 examples/s]
tokenizing (num_proc=8): 56%|█████▋ | 8036/14250 [00:06<00:10, 582.00 examples/s]
tokenizing (num_proc=8): 57%|█████▋ | 8131/14250 [00:06<00:09, 612.78 examples/s]
tokenizing (num_proc=8): 58%|█████▊ | 8222/14250 [00:06<00:09, 657.87 examples/s]
tokenizing (num_proc=8): 59%|█████▊ | 8347/14250 [00:06<00:07, 773.66 examples/s]
tokenizing (num_proc=8): 59%|█████▉ | 8465/14250 [00:06<00:06, 849.51 examples/s]
tokenizing (num_proc=8): 60%|██████ | 8618/14250 [00:06<00:05, 1007.43 examples/s]
tokenizing (num_proc=8): 62%|██████▏ | 8791/14250 [00:07<00:04, 1172.57 examples/s]
tokenizing (num_proc=8): 63%|██████▎ | 8932/14250 [00:07<00:04, 1227.26 examples/s]
tokenizing (num_proc=8): 64%|██████▎ | 9075/14250 [00:07<00:04, 1276.87 examples/s]
tokenizing (num_proc=8): 65%|██████▍ | 9234/14250 [00:07<00:03, 1363.29 examples/s]
tokenizing (num_proc=8): 66%|██████▌ | 9387/14250 [00:07<00:03, 1396.37 examples/s]
tokenizing (num_proc=8): 67%|██████▋ | 9558/14250 [00:07<00:03, 1453.05 examples/s]
tokenizing (num_proc=8): 68%|██████▊ | 9711/14250 [00:07<00:03, 1460.08 examples/s]
tokenizing (num_proc=8): 69%|██████▉ | 9881/14250 [00:07<00:02, 1517.12 examples/s]
tokenizing (num_proc=8): 71%|███████ | 10093/14250 [00:07<00:02, 1666.38 examples/s]
tokenizing (num_proc=8): 72%|███████▏ | 10287/14250 [00:08<00:02, 1716.59 examples/s]
tokenizing (num_proc=8): 73%|███████▎ | 10464/14250 [00:08<00:02, 1642.87 examples/s]
tokenizing (num_proc=8): 75%|███████▍ | 10645/14250 [00:08<00:02, 1640.68 examples/s]
tokenizing (num_proc=8): 76%|███████▌ | 10820/14250 [00:08<00:02, 1631.71 examples/s]
tokenizing (num_proc=8): 77%|███████▋ | 10995/14250 [00:08<00:02, 1612.62 examples/s]
tokenizing (num_proc=8): 78%|███████▊ | 11176/14250 [00:08<00:01, 1615.53 examples/s]
tokenizing (num_proc=8): 80%|███████▉ | 11352/14250 [00:08<00:01, 1628.01 examples/s]
tokenizing (num_proc=8): 81%|████████ | 11518/14250 [00:08<00:01, 1633.52 examples/s]
tokenizing (num_proc=8): 82%|████████▏ | 11706/14250 [00:08<00:01, 1679.80 examples/s]
tokenizing (num_proc=8): 83%|████████▎ | 11894/14250 [00:08<00:01, 1724.75 examples/s]
tokenizing (num_proc=8): 85%|████████▍ | 12078/14250 [00:09<00:01, 1751.27 examples/s]
tokenizing (num_proc=8): 86%|████████▌ | 12262/14250 [00:09<00:01, 1719.10 examples/s]
tokenizing (num_proc=8): 87%|████████▋ | 12463/14250 [00:09<00:01, 1750.59 examples/s]
tokenizing (num_proc=8): 89%|████████▊ | 12640/14250 [00:09<00:00, 1683.74 examples/s]
tokenizing (num_proc=8): 90%|████████▉ | 12821/14250 [00:09<00:00, 1692.39 examples/s]
tokenizing (num_proc=8): 91%|█████████ | 13002/14250 [00:09<00:00, 1722.06 examples/s]
tokenizing (num_proc=8): 93%|█████████▎| 13183/14250 [00:09<00:00, 1671.07 examples/s]
tokenizing (num_proc=8): 94%|█████████▍| 13360/14250 [00:09<00:00, 1557.49 examples/s]
tokenizing (num_proc=8): 95%|█████████▍| 13519/14250 [00:10<00:00, 1452.19 examples/s]
tokenizing (num_proc=8): 96%|█████████▌| 13671/14250 [00:10<00:00, 1366.25 examples/s]
tokenizing (num_proc=8): 97%|█████████▋| 13818/14250 [00:10<00:00, 1238.60 examples/s]
tokenizing (num_proc=8): 98%|█████████▊| 13948/14250 [00:10<00:00, 1094.77 examples/s]
tokenizing (num_proc=8): 99%|█████████▉| 14072/14250 [00:10<00:00, 989.95 examples/s]
tokenizing (num_proc=8): 99%|█████████▉| 14177/14250 [00:10<00:00, 884.58 examples/s]
tokenizing (num_proc=8): 100%|██████████| 14250/14250 [00:11<00:00, 1245.71 examples/s]
Filter: 0%| | 0/14250 [00:00<?, ? examples/s]
Filter: 7%|▋ | 1000/14250 [00:00<00:03, 3895.81 examples/s]
Filter: 14%|█▍ | 2000/14250 [00:00<00:02, 4620.30 examples/s]
Filter: 21%|██ | 3000/14250 [00:00<00:02, 4453.77 examples/s]
Filter: 28%|██▊ | 4000/14250 [00:00<00:02, 4681.14 examples/s]
Filter: 35%|███▌ | 5000/14250 [00:01<00:01, 4759.22 examples/s]
Filter: 42%|████▏ | 6000/14250 [00:01<00:01, 4693.79 examples/s]
Filter: 49%|████▉ | 7000/14250 [00:01<00:01, 4815.32 examples/s]
Filter: 56%|█████▌ | 8000/14250 [00:01<00:01, 4963.41 examples/s]
Filter: 63%|██████▎ | 9000/14250 [00:01<00:01, 4978.38 examples/s]
Filter: 70%|███████ | 10000/14250 [00:02<00:00, 5111.16 examples/s]
Filter: 77%|███████▋ | 11000/14250 [00:02<00:00, 5018.46 examples/s]
Filter: 84%|████████▍ | 12000/14250 [00:02<00:00, 4852.32 examples/s]
Filter: 91%|█████████ | 13000/14250 [00:02<00:00, 4956.03 examples/s]
Filter: 98%|█████████▊| 14000/14250 [00:02<00:00, 4742.63 examples/s]
Filter: 100%|██████████| 14250/14250 [00:02<00:00, 4829.84 examples/s]
Filter: 0%| | 0/14250 [00:00<?, ? examples/s]
Filter: 7%|▋ | 1000/14250 [00:00<00:02, 4719.50 examples/s]
Filter: 14%|█▍ | 2000/14250 [00:00<00:02, 4881.84 examples/s]
Filter: 21%|██ | 3000/14250 [00:00<00:02, 4545.55 examples/s]
Filter: 28%|██▊ | 4000/14250 [00:00<00:02, 4662.41 examples/s]
Filter: 35%|███▌ | 5000/14250 [00:01<00:01, 4656.56 examples/s]
Filter: 42%|████▏ | 6000/14250 [00:01<00:01, 4753.82 examples/s]
Filter: 49%|████▉ | 7000/14250 [00:01<00:01, 4746.76 examples/s]
Filter: 56%|█████▌ | 8000/14250 [00:01<00:01, 4691.69 examples/s]
Filter: 63%|██████▎ | 9000/14250 [00:01<00:01, 4833.64 examples/s]
Filter: 70%|███████ | 10000/14250 [00:02<00:00, 4612.92 examples/s]
Filter: 77%|███████▋ | 11000/14250 [00:02<00:00, 4673.15 examples/s]
Filter: 84%|████████▍ | 12000/14250 [00:02<00:00, 4669.64 examples/s]
Filter: 91%|█████████ | 13000/14250 [00:02<00:00, 4606.14 examples/s]
Filter: 98%|█████████▊| 14000/14250 [00:03<00:00, 4419.02 examples/s]
Filter: 100%|██████████| 14250/14250 [00:03<00:00, 4627.10 examples/s]
supervised tokens: tool 4221981, replay 0 (0.0% replay)
rows: 14250 raw -> 14138 kept; by source: {'apigen': 4888, 'recovery': 3750, 'xlam': 5000, 'efficiency': 500}
[manifest] git=534143b028a9765e904b4d6d1b22205027804f19 data_sha256={'train_olmo-pretrained_replay50.jsonl': '64d5d5af76e7', 'xlam5000_max1call.jsonl': 'a8aae6cf5c36'}
wandb: WARNING git root /workspace/anchor/olmo-pretrained/wandb does not exist
wandb: Tracking run with wandb version 0.30.0
wandb: W&B syncing is set to `offline` in this directory. Run `wandb online` or set WANDB_MODE=online to enable cloud syncing.
wandb: Run data is saved locally in /workspace/anchor/olmo-pretrained/wandb/wandb/offline-run-20261016_031238-pqlcax1d
wandb: View this run in the terminal with `wandb leet`
wandb: Detected [huggingface_hub.inference] in use.
wandb: Use W&B Weave for improved LLM call tracing. Install Weave with `pip install weave` then add `import weave` to the top of your script.
wandb: For more information, check out the docs at: https://weave-docs.wandb.ai
==((====))== Unsloth - 2x faster free finetuning | Num GPUs used = 1
\\ /| Num examples = 13,784 | Num Epochs = 1 | Total steps = 862
O^O/ \_/ \ Batch size per device = 2 | Gradient accumulation steps = 8
\ / Data Parallel GPUs = 1 | Total batch size (2 x 8 x 1) = 16
"-____-" Trainable parameters = 981,385,216 of 8,279,396,352 (11.85% trained)
Unsloth: Setting lr = 2.00e-05 instead of 2.00e-04 for embed_tokens.
Unsloth: Setting lr = 2.00e-05 instead of 2.00e-04 for lm_head.
0%| | 0/862 [00:00<?, ?it/s]
0%| | 1/862 [00:44<10:42:30, 44.77s/it]
0%| | 2/862 [01:12<8:15:15, 34.55s/it]
0%| | 3/862 [01:34<6:57:15, 29.15s/it]
0%| | 4/862 [01:59<6:30:55, 27.34s/it]
1%| | 5/862 [02:23<6:12:21, 26.07s/it]
1%| | 5/862 [02:23<6:12:21, 26.07s/it]
1%| | 6/862 [02:43<5:43:34, 24.08s/it]
1%| | 7/862 [03:10<5:57:20, 25.08s/it]
1%| | 8/862 [03:35<5:57:27, 25.11s/it]
1%| | 9/862 [03:58<5:44:53, 24.26s/it]
1%| | 10/862 [04:21<5:40:37, 23.99s/it]
1%| | 10/862 [04:21<5:40:37, 23.99s/it]
1%|▏ | 11/862 [04:42<5:28:19, 23.15s/it]
1%|▏ | 12/862 [05:06<5:30:25, 23.32s/it]
2%|▏ | 13/862 [05:26<5:16:39, 22.38s/it]
2%|▏ | 14/862 [05:49<5:18:53, 22.56s/it]
2%|▏ | 15/862 [06:13<5:24:49, 23.01s/it]
2%|▏ | 15/862 [06:13<5:24:49, 23.01s/it]
2%|▏ | 16/862 [06:38<5:31:17, 23.50s/it]
2%|▏ | 17/862 [07:00<5:24:03, 23.01s/it]
2%|▏ | 18/862 [07:18<5:04:18, 21.63s/it]
2%|▏ | 19/862 [07:41<5:09:50, 22.05s/it]
2%|▏ | 20/862 [08:06<5:19:10, 22.74s/it]
2%|▏ | 20/862 [08:06<5:19:10, 22.74s/it]
2%|▏ | 21/862 [08:31<5:28:57, 23.47s/it]
3%|▎ | 22/862 [08:53<5:25:10, 23.23s/it]
3%|▎ | 23/862 [09:09<4:52:17, 20.90s/it]
3%|▎ | 24/862 [09:31<4:55:56, 21.19s/it]
3%|▎ | 25/862 [09:52<4:56:06, 21.23s/it]
3%|▎ | 25/862 [09:52<4:56:06, 21.23s/it]
3%|▎ | 26/862 [10:13<4:54:07, 21.11s/it]
3%|▎ | 27/862 [10:37<5:04:40, 21.89s/it]
3%|▎ | 28/862 [10:51<4:34:34, 19.75s/it]
3%|▎ | 29/862 [11:15<4:51:40, 21.01s/it]
3%|▎ | 30/862 [11:39<5:01:39, 21.75s/it]
3%|▎ | 30/862 [11:39<5:01:39, 21.75s/it]
4%|▎ | 31/862 [12:02<5:07:38, 22.21s/it]
4%|▎ | 32/862 [12:21<4:51:57, 21.11s/it]
4%|▍ | 33/862 [12:46<5:10:15, 22.45s/it]
4%|▍ | 34/862 [13:06<4:57:46, 21.58s/it]
4%|▍ | 35/862 [13:25<4:49:17, 20.99s/it]
4%|▍ | 35/862 [13:25<4:49:17, 20.99s/it]
4%|▍ | 36/862 [13:48<4:57:37, 21.62s/it]
4%|▍ | 37/862 [14:08<4:48:28, 20.98s/it]
4%|▍ | 38/862 [14:33<5:06:16, 22.30s/it]
5%|▍ | 39/862 [14:56<5:05:56, 22.30s/it]
5%|▍ | 40/862 [15:16<4:56:12, 21.62s/it]
5%|▍ | 40/862 [15:16<4:56:12, 21.62s/it]
5%|▍ | 41/862 [15:37<4:54:10, 21.50s/it]
5%|▍ | 42/862 [16:00<5:00:48, 22.01s/it]
5%|▍ | 43/862 [16:26<5:15:50, 23.14s/it]
5%|▌ | 44/862 [16:44<4:53:24, 21.52s/it]
5%|▌ | 45/862 [17:08<5:06:27, 22.51s/it]
5%|▌ | 45/862 [17:08<5:06:27, 22.51s/it]
5%|▌ | 46/862 [17:28<4:53:48, 21.60s/it]
5%|▌ | 47/862 [17:55<5:16:23, 23.29s/it]
6%|▌ | 48/862 [18:17<5:09:44, 22.83s/it]
6%|▌ | 49/862 [18:40<5:10:24, 22.91s/it]
6%|▌ | 50/862 [19:03<5:12:02, 23.06s/it]
6%|▌ | 50/862 [19:03<5:12:02, 23.06s/it]Unsloth: Will smartly offload gradients to save VRAM!
Unsloth: Double buffering enabled (parallel H2D + compute) for backward pass.
{'loss': 0.9941, 'grad_norm': 1.8992259502410889, 'learning_rate': 1.8181818181818182e-05, 'epoch': 0.01}
{'loss': 0.8727, 'grad_norm': 1.5108100175857544, 'learning_rate': 4.0909090909090915e-05, 'epoch': 0.01}
{'loss': 0.7164, 'grad_norm': 1.313894271850586, 'learning_rate': 6.363636363636364e-05, 'epoch': 0.02}
{'loss': 0.5324, 'grad_norm': 1.5354955196380615, 'learning_rate': 8.636363636363637e-05, 'epoch': 0.02}
{'loss': 0.4492, 'grad_norm': 1.4390828609466553, 'learning_rate': 0.00010909090909090909, 'epoch': 0.03}
{'loss': 0.4172, 'grad_norm': 1.8266173601150513, 'learning_rate': 0.0001318181818181818, 'epoch': 0.03}
{'loss': 0.3966, 'grad_norm': 1.5203430652618408, 'learning_rate': 0.00015454545454545454, 'epoch': 0.04}
{'loss': 0.3584, 'grad_norm': 1.6018481254577637, 'learning_rate': 0.00017727272727272728, 'epoch': 0.05}
{'loss': 0.3571, 'grad_norm': 1.3058902025222778, 'learning_rate': 0.0002, 'epoch': 0.05}
{'loss': 0.2975, 'grad_norm': 1.388102650642395, 'learning_rate': 0.00019998156301911284, 'epoch': 0.06}
Filter: 0%| | 0/354 [00:00<?, ? examples/s][A
Filter: 100%|██████████| 354/354 [00:00<00:00, 1481.01 examples/s][A
Filter: 100%|██████████| 354/354 [00:00<00:00, 1467.34 examples/s]
0%| | 0/177 [00:00<?, ?it/s][A
1%| | 2/177 [00:01<02:04, 1.40it/s][A
2%|▏ | 3/177 [00:02<02:28, 1.17it/s][A
2%|▏ | 4/177 [00:03<02:37, 1.10it/s][A
3%|▎ | 5/177 [00:04<02:33, 1.12it/s][A
3%|▎ | 6/177 [00:04<01:58, 1.44it/s][A
4%|▍ | 7/177 [00:05<02:18, 1.23it/s][A
5%|▍ | 8/177 [00:06<02:18, 1.22it/s][A
5%|▌ | 9/177 [00:07<02:20, 1.20it/s][A
6%|▌ | 10/177 [00:08<02:21, 1.18it/s][A
6%|▌ | 11/177 [00:09<02:30, 1.10it/s][A
7%|▋ | 12/177 [00:09<01:55, 1.42it/s][A
7%|▋ | 13/177 [00:09<01:39, 1.65it/s][A
8%|▊ | 14/177 [00:10<02:00, 1.36it/s][A
8%|▊ | 15/177 [00:11<02:03, 1.31it/s][A
9%|▉ | 16/177 [00:12<02:13, 1.20it/s][A
10%|▉ | 17/177 [00:13<01:46, 1.50it/s][A
10%|█ | 18/177 [00:13<01:53, 1.41it/s][A
11%|█ | 19/177 [00:14<02:07, 1.24it/s][A
11%|█▏ | 20/177 [00:15<02:05, 1.26it/s][A
12%|█▏ | 21/177 [00:16<02:10, 1.19it/s][A
12%|█▏ | 22/177 [00:18<02:35, 1.00s/it][A
13%|█▎ | 23/177 [00:19<02:54, 1.13s/it][A
14%|█▎ | 24/177 [00:20<02:56, 1.15s/it][A
14%|█▍ | 25/177 [00:20<02:16, 1.12it/s][A
15%|█▍ | 26/177 [00:21<02:09, 1.17it/s][A
15%|█▌ | 27/177 [00:22<02:06, 1.19it/s][A
16%|█▌ | 28/177 [00:23<02:01, 1.22it/s][A
16%|█▋ | 29/177 [00:24<02:22, 1.04it/s][A
17%|█▋ | 30/177 [00:25<02:36, 1.06s/it][A
18%|█▊ | 31/177 [00:27<02:39, 1.09s/it][A
18%|█▊ | 32/177 [00:27<02:26, 1.01s/it][A
19%|█▊ | 33/177 [00:28<02:19, 1.03it/s][A
19%|█▉ | 34/177 [00:29<02:01, 1.18it/s][A
20%|█▉ | 35/177 [00:30<02:07, 1.11it/s][A
20%|██ | 36/177 [00:30<01:38, 1.43it/s][A
21%|██ | 37/177 [00:31<01:44, 1.34it/s][A
21%|██▏ | 38/177 [00:32<01:43, 1.34it/s][A
22%|██▏ | 39/177 [00:33<01:57, 1.17it/s][A
23%|██▎ | 40/177 [00:34<02:07, 1.07it/s][A
23%|██▎ | 41/177 [00:35<02:03, 1.10it/s][A
24%|██▎ | 42/177 [00:36<02:07, 1.06it/s][A
24%|██▍ | 43/177 [00:37<02:30, 1.12s/it][A
25%|██▍ | 44/177 [00:39<02:34, 1.16s/it][A
25%|██▌ | 45/177 [00:40<02:35, 1.18s/it][A
26%|██▌ | 46/177 [00:41<02:35, 1.19s/it][A
27%|██▋ | 47/177 [00:42<02:38, 1.22s/it][A
27%|██▋ | 48/177 [00:43<02:37, 1.22s/it][A
28%|██▊ | 49/177 [00:45<02:37, 1.23s/it][A
28%|██▊ | 50/177 [00:46<02:35, 1.22s/it][A
29%|██▉ | 51/177 [00:47<02:39, 1.26s/it][A
29%|██▉ | 52/177 [00:48<02:12, 1.06s/it][A
30%|██▉ | 53/177 [00:49<02:12, 1.07s/it][A
31%|███ | 54/177 [00:50<02:00, 1.02it/s][A
31%|███ | 55/177 [00:51<01:55, 1.05it/s][A
32%|███▏ | 56/177 [00:51<01:50, 1.10it/s][A
32%|███▏ | 57/177 [00:52<01:42, 1.17it/s][A
33%|███▎ | 58/177 [00:53<01:36, 1.23it/s][A
33%|███▎ | 59/177 [00:54<01:36, 1.22it/s][A
34%|███▍ | 60/177 [00:54<01:27, 1.34it/s][A
34%|███▍ | 61/177 [00:55<01:28, 1.31it/s][A
35%|███▌ | 62/177 [00:56<01:34, 1.22it/s][A
36%|███▌ | 63/177 [00:57<01:43, 1.10it/s][A
36%|███▌ | 64/177 [00:58<01:46, 1.06it/s][A
37%|███▋ | 65/177 [00:59<01:42, 1.09it/s][A
37%|███▋ | 66/177 [01:00<01:38, 1.13it/s][A
38%|███▊ | 67/177 [01:00<01:11, 1.54it/s][A
38%|███▊ | 68/177 [01:01<01:14, 1.47it/s][A
39%|███▉ | 69/177 [01:01<00:59, 1.80it/s][A
40%|████ | 71/177 [01:02<00:54, 1.94it/s][A
41%|████ | 72/177 [01:02<00:47, 2.19it/s][A
41%|████ | 73/177 [01:03<00:52, 1.97it/s][A
42%|████▏ | 74/177 [01:04<01:00, 1.72it/s][A
42%|████▏ | 75/177 [01:05<01:12, 1.40it/s][A
43%|████▎ | 76/177 [01:05<01:08, 1.47it/s][A
44%|████▎ | 77/177 [01:06<01:20, 1.24it/s][A
44%|████▍ | 78/177 [01:07<01:26, 1.15it/s][A
45%|████▍ | 79/177 [01:08<01:25, 1.15it/s][A
45%|████▌ | 80/177 [01:09<01:21, 1.19it/s][A
46%|████▌ | 81/177 [01:10<01:20, 1.19it/s][A
46%|████▋ | 82/177 [01:11<01:27, 1.08it/s][A
47%|████▋ | 83/177 [01:13<01:47, 1.14s/it][A
47%|████▋ | 84/177 [01:14<01:37, 1.05s/it][A
48%|████▊ | 85/177 [01:14<01:24, 1.09it/s][A
49%|████▊ | 86/177 [01:15<01:14, 1.23it/s][A
49%|████▉ | 87/177 [01:16<01:14, 1.21it/s][A
50%|████▉ | 88/177 [01:16<01:07, 1.32it/s][A
50%|█████ | 89/177 [01:17<01:02, 1.40it/s][A
51%|█████ | 90/177 [01:17<00:57, 1.51it/s][A
52%|█████▏ | 92/177 [01:18<00:50, 1.70it/s][A
53%|█████▎ | 93/177 [01:19<00:41, 2.00it/s][A
53%|█████▎ | 94/177 [01:20<00:55, 1.50it/s][A
54%|█████▎ | 95/177 [01:21<01:03, 1.30it/s][A
54%|█████▍ | 96/177 [01:22<01:10, 1.15it/s][A
55%|█████▍ | 97/177 [01:22<01:00, 1.33it/s][A
55%|█████▌ | 98/177 [01:23<01:04, 1.22it/s][A
56%|█████▌ | 99/177 [01:24<01:10, 1.10it/s][A
56%|█████▋ | 100/177 [01:25<01:01, 1.25it/s][A
57%|█████▋ | 101/177 [01:26<01:05, 1.15it/s][A
58%|█████▊ | 102/177 [01:27<01:11, 1.05it/s][A
58%|█████▊ | 103/177 [01:28<01:07, 1.09it/s][A
59%|█████▉ | 104/177 [01:29<01:07, 1.08it/s][A
59%|█████▉ | 105/177 [01:30<01:06, 1.09it/s][A
60%|█████▉ | 106/177 [01:31<01:03, 1.11it/s][A
60%|██████ | 107/177 [01:32<01:05, 1.08it/s][A
61%|██████ | 108/177 [01:32<01:01, 1.12it/s][A
62%|██████▏ | 109/177 [01:33<00:54, 1.24it/s][A
63%|██████▎ | 111/177 [01:34<00:42, 1.55it/s][A
63%|██████▎ | 112/177 [01:35<00:44, 1.47it/s][A
64%|██████▍ | 113/177 [01:35<00:39, 1.62it/s][A
64%|██████▍ | 114/177 [01:36<00:46, 1.35it/s][A
65%|██████▍ | 115/177 [01:37<00:46, 1.33it/s][A
66%|██████▌ | 116/177 [01:38<00:49, 1.24it/s][A
66%|██████▌ | 117/177 [01:39<00:45, 1.33it/s][A
67%|██████▋ | 118/177 [01:40<00:48, 1.21it/s][A
67%|██████▋ | 119/177 [01:41<00:55, 1.04it/s][A
68%|██████▊ | 120/177 [01:42<00:55, 1.02it/s][A
68%|██████▊ | 121/177 [01:42<00:40, 1.39it/s][A
69%|██████▉ | 122/177 [01:43<00:37, 1.47it/s][A
69%|██████▉ | 123/177 [01:43<00:37, 1.44it/s][A
70%|███████ | 124/177 [01:44<00:42, 1.24it/s][A
71%|███████ | 125/177 [01:46<00:50, 1.03it/s][A
71%|███████ | 126/177 [01:47<00:51, 1.00s/it][A
72%|███████▏ | 127/177 [01:47<00:41, 1.21it/s][A
72%|███████▏ | 128/177 [01:48<00:43, 1.12it/s][A
73%|███████▎ | 129/177 [01:49<00:42, 1.13it/s][A
73%|███████▎ | 130/177 [01:50<00:42, 1.10it/s][A
74%|███████▍ | 131/177 [01:51<00:42, 1.08it/s][A
75%|███████▍ | 132/177 [01:52<00:41, 1.07it/s][A
75%|███████▌ | 133/177 [01:52<00:32, 1.34it/s][A
76%|███████▌ | 134/177 [01:53<00:34, 1.23it/s][A
76%|███████▋ | 135/177 [01:54<00:27, 1.51it/s][A
77%|███████▋ | 137/177 [01:54<00:15, 2.56it/s][A
78%|███████▊ | 138/177 [01:54<00:17, 2.26it/s][A
79%|███████▊ | 139/177 [01:55<00:20, 1.84it/s][A
79%|███████▉ | 140/177 [01:56<00:22, 1.67it/s][A
80%|███████▉ | 141/177 [01:57<00:29, 1.24it/s][A
80%|████████ | 142/177 [01:58<00:31, 1.12it/s][A
81%|████████ | 143/177 [01:59<00:26, 1.26it/s][A
81%|████████▏ | 144/177 [02:00<00:27, 1.19it/s][A
82%|████████▏ | 145/177 [02:01<00:31, 1.02it/s][A
82%|████████▏ | 146/177 [02:02<00:30, 1.02it/s][A
83%|████████▎ | 147/177 [02:03<00:30, 1.01s/it][A
84%|████████▎ | 148/177 [02:04<00:25, 1.14it/s][A
84%|████████▍ | 149/177 [02:04<00:20, 1.37it/s][A
85%|████████▍ | 150/177 [02:05<00:18, 1.45it/s][A
85%|████████▌ | 151/177 [02:06<00:21, 1.21it/s][A
86%|████████▌ | 152/177 [02:07<00:22, 1.11it/s][A
86%|████████▋ | 153/177 [02:08<00:20, 1.16it/s][A
87%|████████▋ | 154/177 [02:08<00:15, 1.51it/s][A
88%|████████▊ | 155/177 [02:09<00:15, 1.39it/s][A
88%|████████▊ | 156/177 [02:10<00:15, 1.35it/s][A
89%|████████▊ | 157/177 [02:11<00:15, 1.31it/s][A
89%|████████▉ | 158/177 [02:11<00:14, 1.29it/s][A
90%|████████▉ | 159/177 [02:12<00:14, 1.28it/s][A
90%|█████████ | 160/177 [02:13<00:12, 1.38it/s][A
91%|█████████ | 161/177 [02:14<00:12, 1.33it/s][A
92%|█████████▏| 162/177 [02:14<00:10, 1.44it/s][A
92%|█████████▏| 163/177 [02:15<00:09, 1.43it/s][A
93%|█████████▎| 164/177 [02:16<00:09, 1.32it/s][A
93%|█████████▎| 165/177 [02:17<00:11, 1.09it/s][A
94%|█████████▍| 166/177 [02:18<00:10, 1.06it/s][A
94%|█████████▍| 167/177 [02:19<00:09, 1.02it/s][A
95%|█████████▍| 168/177 [02:20<00:09, 1.05s/it][A
95%|█████████▌| 169/177 [02:21<00:08, 1.09s/it][A
96%|█████████▌| 170/177 [02:23<00:07, 1.09s/it][A
97%|█████████▋| 171/177 [02:23<00:05, 1.16it/s][A
97%|█████████▋| 172/177 [02:23<00:03, 1.39it/s][A
98%|█████████▊| 173/177 [02:24<00:03, 1.30it/s][A
98%|█████████▊| 174/177 [02:25<00:02, 1.40it/s][A
99%|█████████▉| 176/177 [02:26<00:00, 1.54it/s][A
100%|██████████| 177/177 [02:27<00:00, 1.34it/s][A
[A
6%|▌ | 50/862 [21:36<5:12:02, 23.06s/it]
100%|██████████| 177/177 [02:27<00:00, 1.34it/s][A
[A
6%|▌ | 51/862 [21:54<15:11:06, 67.41s/it]
6%|▌ | 52/862 [22:16<12:04:25, 53.66s/it]
6%|▌ | 53/862 [22:41<10:06:34, 44.99s/it]
6%|▋ | 54/862 [23:04<8:38:55, 38.53s/it]
6%|▋ | 55/862 [23:27<7:37:06, 33.99s/it]
6%|▋ | 55/862 [23:27<7:37:06, 33.99s/it]
6%|▋ | 56/862 [23:48<6:43:25, 30.03s/it]
7%|▋ | 57/862 [24:10<6:10:36, 27.62s/it]
7%|▋ | 58/862 [24:30<5:38:49, 25.29s/it]
7%|▋ | 59/862 [24:49<5:12:30, 23.35s/it]
7%|▋ | 60/862 [25:10<5:02:52, 22.66s/it]
7%|▋ | 60/862 [25:10<5:02:52, 22.66s/it]
7%|▋ | 61/862 [25:29<4:48:18, 21.60s/it]
7%|▋ | 62/862 [25:49<4:41:25, 21.11s/it]
7%|▋ | 63/862 [26:13<4:50:19, 21.80s/it]
7%|▋ | 64/862 [26:34<4:48:05, 21.66s/it]
8%|▊ | 65/862 [26:54<4:39:53, 21.07s/it]
8%|▊ | 65/862 [26:54<4:39:53, 21.07s/it]
8%|▊ | 66/862 [27:09<4:17:15, 19.39s/it]
8%|▊ | 67/862 [27:31<4:26:41, 20.13s/it]
8%|▊ | 68/862 [27:51<4:24:46, 20.01s/it]
8%|▊ | 69/862 [28:11<4:26:46, 20.18s/it]
8%|▊ | 70/862 [28:26<4:06:32, 18.68s/it]
8%|▊ | 70/862 [28:26<4:06:32, 18.68s/it]
8%|▊ | 71/862 [28:45<4:07:16, 18.76s/it]
8%|▊ | 72/862 [29:08<4:21:11, 19.84s/it]
8%|▊ | 73/862 [29:28<4:23:54, 20.07s/it]
9%|▊ | 74/862 [29:45<4:10:43, 19.09s/it]
9%|▊ | 75/862 [30:04<4:11:16, 19.16s/it]
9%|▊ | 75/862 [30:04<4:11:16, 19.16s/it]
9%|▉ | 76/862 [30:32<4:43:28, 21.64s/it]
9%|▉ | 77/862 [30:56<4:51:52, 22.31s/it]
9%|▉ | 78/862 [31:15<4:39:30, 21.39s/it]
9%|▉ | 79/862 [31:39<4:51:33, 22.34s/it]
9%|▉ | 80/862 [31:58<4:36:42, 21.23s/it]
9%|▉ | 80/862 [31:58<4:36:42, 21.23s/it]
9%|▉ | 81/862 [32:15<4:18:42, 19.88s/it]
10%|▉ | 82/862 [32:39<4:35:19, 21.18s/it]
10%|▉ | 83/862 [32:56<4:19:50, 20.01s/it]
10%|▉ | 84/862 [33:13<4:06:52, 19.04s/it]
10%|▉ | 85/862 [33:31<4:03:53, 18.83s/it]
10%|▉ | 85/862 [33:31<4:03:53, 18.83s/it]
10%|▉ | 86/862 [33:49<3:57:15, 18.35s/it]
10%|█ | 87/862 [34:03<3:40:22, 17.06s/it]
10%|█ | 88/862 [34:23<3:51:10, 17.92s/it]
10%|█ | 89/862 [34:48<4:20:04, 20.19s/it]
10%|█ | 90/862 [35:11<4:29:48, 20.97s/it]
10%|█ | 90/862 [35:11<4:29:48, 20.97s/it]
11%|█ | 91/862 [35:33<4:32:16, 21.19s/it]
11%|█ | 92/862 [35:52<4:23:15, 20.51s/it]
11%|█ | 93/862 [36:14<4:31:28, 21.18s/it]
11%|█ | 94/862 [36:33<4:19:49, 20.30s/it]
11%|█ | 95/862 [36:56<4:31:41, 21.25s/it]
11%|█ | 95/862 [36:56<4:31:41, 21.25s/it]
11%|█ | 96/862 [37:19<4:37:46, 21.76s/it]
11%|█▏ | 97/862 [37:36<4:18:24, 20.27s/it]
11%|█▏ | 98/862 [37:56<4:17:27, 20.22s/it]
11%|█▏ | 99/862 [38:12<4:03:00, 19.11s/it]
12%|█▏ | 100/862 [38:33<4:09:12, 19.62s/it]
12%|█▏ | 100/862 [38:33<4:09:12, 19.62s/it]{'eval_loss': 0.2423536628484726, 'eval_runtime': 152.5048, 'eval_samples_per_second': 2.321, 'eval_steps_per_second': 1.161, 'epoch': 0.06}
{'loss': 0.3098, 'grad_norm': 1.3211257457733154, 'learning_rate': 0.0001999262588748966, 'epoch': 0.06}
{'loss': 0.27, 'grad_norm': 1.0737800598144531, 'learning_rate': 0.00019983410796018032, 'epoch': 0.07}
{'loss': 0.2597, 'grad_norm': 1.062996745109558, 'learning_rate': 0.00019970514425465705, 'epoch': 0.08}
{'loss': 0.2213, 'grad_norm': 0.8997095227241516, 'learning_rate': 0.00019953941531235424, 'epoch': 0.08}
{'loss': 0.2616, 'grad_norm': 0.6930145025253296, 'learning_rate': 0.00019933698224409876, 'epoch': 0.09}
{'loss': 0.2218, 'grad_norm': 0.5634636282920837, 'learning_rate': 0.0001990979196949828, 'epoch': 0.09}
{'loss': 0.238, 'grad_norm': 0.545459508895874, 'learning_rate': 0.00019882231581683937, 'epoch': 0.1}
{'loss': 0.2145, 'grad_norm': 0.4912775456905365, 'learning_rate': 0.0001985102722357371, 'epoch': 0.1}
{'loss': 0.2336, 'grad_norm': 0.4108012914657593, 'learning_rate': 0.0001981619040145068, 'epoch': 0.11}
{'loss': 0.1963, 'grad_norm': 0.521858274936676, 'learning_rate': 0.00019777733961031326, 'epoch': 0.12}
0%| | 0/177 [00:00<?, ?it/s][A
1%| | 2/177 [00:00<00:49, 3.52it/s][A
2%|▏ | 3/177 [00:01<01:19, 2.18it/s][A
2%|▏ | 4/177 [00:01<01:08, 2.53it/s][A
3%|▎ | 5/177 [00:01<00:50, 3.42it/s][A
3%|▎ | 6/177 [00:01<00:50, 3.37it/s][A
4%|▍ | 7/177 [00:02<00:52, 3.24it/s][A
5%|▍ | 8/177 [00:02<00:52, 3.19it/s][A
5%|▌ | 9/177 [00:02<00:55, 3.02it/s][A
6%|▌ | 10/177 [00:03<00:56, 2.95it/s][A
6%|▌ | 11/177 [00:03<01:06, 2.50it/s][A
7%|▋ | 12/177 [00:04<00:57, 2.85it/s][A
7%|▋ | 13/177 [00:04<00:59, 2.76it/s][A
8%|▊ | 14/177 [00:04<00:56, 2.89it/s][A
8%|▊ | 15/177 [00:04<00:44, 3.65it/s][A
9%|▉ | 16/177 [00:05<00:43, 3.70it/s][A
10%|▉ | 17/177 [00:05<00:44, 3.62it/s][A
11%|█ | 19/177 [00:05<00:37, 4.22it/s][A
11%|█▏ | 20/177 [00:06<00:38, 4.04it/s][A
12%|█▏ | 21/177 [00:06<00:46, 3.36it/s][A
12%|█▏ | 22/177 [00:07<01:01, 2.53it/s][A
13%|█▎ | 23/177 [00:07<01:15, 2.05it/s][A
14%|█▎ | 24/177 [00:08<01:14, 2.06it/s][A
14%|█▍ | 25/177 [00:08<01:05, 2.33it/s][A
15%|█▍ | 26/177 [00:09<00:57, 2.62it/s][A
15%|█▌ | 27/177 [00:09<00:53, 2.81it/s][A
16%|█▌ | 28/177 [00:09<00:48, 3.06it/s][A
16%|█▋ | 29/177 [00:10<00:58, 2.53it/s][A
17%|█▋ | 30/177 [00:10<01:05, 2.24it/s][A
18%|█▊ | 31/177 [00:11<01:14, 1.96it/s][A
18%|█▊ | 32/177 [00:11<01:05, 2.20it/s][A
19%|█▊ | 33/177 [00:11<00:51, 2.80it/s][A
20%|█▉ | 35/177 [00:12<00:38, 3.65it/s][A
20%|██ | 36/177 [00:12<00:37, 3.79it/s][A
21%|██ | 37/177 [00:12<00:31, 4.38it/s][A
21%|██▏ | 38/177 [00:12<00:32, 4.31it/s][A
22%|██▏ | 39/177 [00:13<00:46, 2.99it/s][A
23%|██▎ | 40/177 [00:13<00:47, 2.88it/s][A
23%|██▎ | 41/177 [00:14<00:47, 2.87it/s][A
24%|██▎ | 42/177 [00:14<00:53, 2.50it/s][A
24%|██▍ | 43/177 [00:15<01:01, 2.19it/s][A
25%|██▍ | 44/177 [00:15<00:59, 2.23it/s][A
25%|██▌ | 45/177 [00:16<01:00, 2.17it/s][A
26%|██▌ | 46/177 [00:16<01:06, 1.98it/s][A
27%|██▋ | 47/177 [00:17<01:03, 2.05it/s][A
27%|██▋ | 48/177 [00:17<01:03, 2.04it/s][A
28%|██▊ | 49/177 [00:18<01:04, 1.97it/s][A
28%|██▊ | 50/177 [00:18<01:02, 2.02it/s][A
29%|██▉ | 51/177 [00:19<01:08, 1.85it/s][A
29%|██▉ | 52/177 [00:19<01:09, 1.80it/s][A
30%|██▉ | 53/177 [00:20<01:10, 1.77it/s][A
31%|███ | 54/177 [00:20<00:58, 2.11it/s][A
31%|███ | 55/177 [00:21<00:54, 2.26it/s][A
32%|███▏ | 56/177 [00:21<00:49, 2.47it/s][A
32%|███▏ | 57/177 [00:21<00:42, 2.84it/s][A
33%|███▎ | 58/177 [00:21<00:36, 3.28it/s][A
33%|███▎ | 59/177 [00:21<00:28, 4.08it/s][A
34%|███▍ | 61/177 [00:22<00:18, 6.20it/s][A
35%|███▌ | 62/177 [00:22<00:20, 5.54it/s][A
36%|███▌ | 63/177 [00:22<00:32, 3.53it/s][A
36%|███▌ | 64/177 [00:23<00:40, 2.82it/s][A
37%|███▋ | 65/177 [00:23<00:39, 2.85it/s][A
38%|███▊ | 67/177 [00:23<00:25, 4.30it/s][A
38%|███▊ | 68/177 [00:24<00:25, 4.26it/s][A
39%|███▉ | 69/177 [00:24<00:26, 4.11it/s][A
40%|████ | 71/177 [00:24<00:24, 4.32it/s][A
41%|████ | 72/177 [00:25<00:25, 4.11it/s][A
41%|████ | 73/177 [00:25<00:22, 4.61it/s][A
42%|████▏ | 74/177 [00:25<00:24, 4.27it/s][A
42%|████▏ | 75/177 [00:25<00:26, 3.90it/s][A
44%|████▎ | 77/177 [00:26<00:25, 4.00it/s][A
44%|████▍ | 78/177 [00:26<00:31, 3.14it/s][A
45%|████▍ | 79/177 [00:27<00:32, 3.04it/s][A
45%|████▌ | 80/177 [00:27<00:30, 3.19it/s][A
46%|████▌ | 81/177 [00:27<00:30, 3.18it/s][A
46%|████▋ | 82/177 [00:28<00:30, 3.12it/s][A
47%|████▋ | 83/177 [00:28<00:38, 2.44it/s][A
47%|████▋ | 84/177 [00:29<00:35, 2.59it/s][A
49%|████▊ | 86/177 [00:29<00:22, 4.07it/s][A
50%|████▉ | 88/177 [00:29<00:16, 5.36it/s][A
50%|█████ | 89/177 [00:29<00:14, 5.93it/s][A
51%|█████▏ | 91/177 [00:29<00:10, 7.88it/s][A
53%|█████▎ | 93/177 [00:30<00:13, 6.25it/s][A
53%|█████▎ | 94/177 [00:30<00:17, 4.71it/s][A
54%|█████▎ | 95/177 [00:31<00:23, 3.47it/s][A
54%|█████▍ | 96/177 [00:31<00:25, 3.18it/s][A
55%|█████▍ | 97/177 [00:32<00:28, 2.84it/s][A
55%|█████▌ | 98/177 [00:32<00:30, 2.58it/s][A
56%|█████▌ | 99/177 [00:33<00:35, 2.22it/s][A
56%|█████▋ | 100/177 [00:33<00:37, 2.04it/s][A
57%|█████▋ | 101/177 [00:34<00:37, 2.02it/s][A
58%|█████▊ | 102/177 [00:34<00:35, 2.13it/s][A
58%|█████▊ | 103/177 [00:34<00:31, 2.34it/s][A
59%|█████▉ | 104/177 [00:35<00:26, 2.73it/s][A
59%|█████▉ | 105/177 [00:35<00:27, 2.66it/s][A
60%|█████▉ | 106/177 [00:35<00:26, 2.71it/s][A
60%|██████ | 107/177 [00:36<00:23, 2.95it/s][A
62%|██████▏ | 109/177 [00:36<00:14, 4.60it/s][A
63%|██████▎ | 111/177 [00:36<00:14, 4.71it/s][A
63%|██████▎ | 112/177 [00:37<00:14, 4.37it/s][A
64%|██████▍ | 113/177 [00:37<00:17, 3.61it/s][A
64%|██████▍ | 114/177 [00:37<00:18, 3.39it/s][A
65%|██████▍ | 115/177 [00:38<00:18, 3.44it/s][A
66%|██████▌ | 116/177 [00:38<00:16, 3.66it/s][A
66%|██████▌ | 117/177 [00:38<00:13, 4.37it/s][A
67%|██████▋ | 118/177 [00:38<00:14, 4.07it/s][A
67%|██████▋ | 119/177 [00:39<00:19, 2.97it/s][A
68%|██████▊ | 120/177 [00:39<00:18, 3.10it/s][A
68%|██████▊ | 121/177 [00:39<00:14, 3.85it/s][A
69%|██████▉ | 123/177 [00:39<00:11, 4.78it/s][A
70%|███████ | 124/177 [00:40<00:12, 4.13it/s][A
71%|███████ | 125/177 [00:40<00:17, 3.02it/s][A
71%|███████ | 126/177 [00:41<00:16, 3.00it/s][A
72%|███████▏ | 127/177 [00:41<00:17, 2.79it/s][A
72%|███████▏ | 128/177 [00:42<00:17, 2.88it/s][A
73%|███████▎ | 129/177 [00:42<00:16, 2.85it/s][A
73%|███████▎ | 130/177 [00:42<00:18, 2.58it/s][A
74%|███████▍ | 131/177 [00:43<00:18, 2.42it/s][A
75%|███████▍ | 132/177 [00:43<00:15, 2.86it/s][A
75%|███████▌ | 133/177 [00:43<00:14, 2.93it/s][A
76%|███████▌ | 134/177 [00:44<00:16, 2.65it/s][A
76%|███████▋ | 135/177 [00:44<00:15, 2.74it/s][A
77%|███████▋ | 137/177 [00:44<00:09, 4.30it/s][A
79%|███████▊ | 139/177 [00:44<00:06, 5.70it/s][A
79%|███████▉ | 140/177 [00:45<00:07, 5.22it/s][A
80%|███████▉ | 141/177 [00:45<00:09, 3.71it/s][A
80%|████████ | 142/177 [00:46<00:11, 3.17it/s][A
81%|████████ | 143/177 [00:46<00:12, 2.65it/s][A
81%|████████▏ | 144/177 [00:47<00:13, 2.51it/s][A
82%|████████▏ | 145/177 [00:47<00:14, 2.20it/s][A
82%|████████▏ | 146/177 [00:48<00:14, 2.15it/s][A
83%|████████▎ | 147/177 [00:48<00:15, 1.91it/s][A
84%|████████▎ | 148/177 [00:49<00:15, 1.88it/s][A
84%|████████▍ | 149/177 [00:49<00:13, 2.04it/s][A
85%|████████▌ | 151/177 [00:50<00:09, 2.63it/s][A
86%|████████▌ | 152/177 [00:50<00:09, 2.68it/s][A
86%|████████▋ | 153/177 [00:50<00:08, 2.90it/s][A
87%|████████▋ | 154/177 [00:51<00:06, 3.29it/s][A
88%|████████▊ | 155/177 [00:51<00:05, 3.94it/s][A
89%|████████▊ | 157/177 [00:51<00:03, 5.65it/s][A
89%|████████▉ | 158/177 [00:51<00:03, 4.86it/s][A
90%|████████▉ | 159/177 [00:52<00:04, 4.38it/s][A
91%|█████████ | 161/177 [00:52<00:02, 6.02it/s][A
92%|█████████▏| 163/177 [00:52<00:02, 6.46it/s][A
93%|█████████▎| 164/177 [00:52<00:02, 4.92it/s][A
93%|█████████▎| 165/177 [00:53<00:03, 3.46it/s][A
94%|█████████▍| 166/177 [00:53<00:03, 2.92it/s][A
94%|█████████▍| 167/177 [00:54<00:03, 2.92it/s][A
95%|█████████▍| 168/177 [00:54<00:03, 2.71it/s][A
95%|█████████▌| 169/177 [00:55<00:03, 2.62it/s][A
96%|█████████▌| 170/177 [00:55<00:02, 2.78it/s][A
97%|█████████▋| 171/177 [00:55<00:02, 2.85it/s][A
97%|█████████▋| 172/177 [00:56<00:01, 2.78it/s][A
98%|█████████▊| 173/177 [00:56<00:01, 2.73it/s][A
99%|█████████▉| 175/177 [00:56<00:00, 4.20it/s][A
99%|█████████▉| 176/177 [00:57<00:00, 3.90it/s][A
100%|██████████| 177/177 [00:57<00:00, 3.63it/s][A
[A
12%|█▏ | 100/862 [39:31<4:09:12, 19.62s/it]
100%|██████████| 177/177 [00:57<00:00, 3.63it/s][A
[Awandb: WARNING URL not available in offline run
`rope_scaling`'s beta_fast field must be a float, got 32
`rope_scaling`'s beta_slow field must be a float, got 1
12%|█▏ | 101/862 [39:59<8:18:49, 39.33s/it]
12%|█▏ | 102/862 [40:17<7:00:20, 33.19s/it]
12%|█▏ | 103/862 [40:37<6:09:28, 29.21s/it]
12%|█▏ | 104/862 [41:38<8:07:17, 38.57s/it]
12%|█▏ | 105/862 [42:07<7:31:11, 35.76s/it]
12%|█▏ | 105/862 [42:07<7:31:11, 35.76s/it]
12%|█▏ | 106/862 [42:25<6:23:19, 30.42s/it]
12%|█▏ | 107/862 [42:48<5:55:01, 28.21s/it]
13%|█▎ | 108/862 [43:01<4:58:43, 23.77s/it]
13%|█▎ | 109/862 [43:15<4:19:05, 20.64s/it]
13%|█▎ | 110/862 [43:28<3:51:39, 18.48s/it]
13%|█▎ | 110/862 [43:28<3:51:39, 18.48s/it]
13%|█▎ | 111/862 [43:46<3:48:38, 18.27s/it]
13%|█▎ | 112/862 [44:07<3:59:49, 19.19s/it]
13%|█▎ | 113/862 [44:36<4:36:45, 22.17s/it]
13%|█▎ | 114/862 [44:54<4:19:00, 20.78s/it]
13%|█▎ | 115/862 [45:10<4:00:31, 19.32s/it]
13%|█▎ | 115/862 [45:10<4:00:31, 19.32s/it]
13%|█▎ | 116/862 [45:23<3:37:13, 17.47s/it]
14%|█▎ | 117/862 [45:45<3:52:11, 18.70s/it]
14%|█▎ | 118/862 [46:03<3:50:20, 18.58s/it]
14%|█▍ | 119/862 [46:19<3:39:45, 17.75s/it]
14%|█▍ | 120/862 [46:39<3:49:27, 18.55s/it]
14%|█▍ | 120/862 [46:39<3:49:27, 18.55s/it]
14%|█▍ | 121/862 [46:58<3:50:50, 18.69s/it]
14%|█▍ | 122/862 [47:15<3:43:05, 18.09s/it]
14%|█▍ | 123/862 [47:28<3:23:47, 16.55s/it]
14%|█▍ | 124/862 [47:50<3:43:51, 18.20s/it]
15%|█▍ | 125/862 [48:12<4:00:07, 19.55s/it]
15%|█▍ | 125/862 [48:12<4:00:07, 19.55s/it]
15%|█▍ | 126/862 [48:30<3:52:26, 18.95s/it]
15%|█▍ | 127/862 [48:50<3:54:34, 19.15s/it]
15%|█▍ | 128/862 [49:06<3:42:15, 18.17s/it]
15%|█▍ | 129/862 [49:26<3:48:45, 18.72s/it]
15%|█▌ | 130/862 [49:45<3:52:27, 19.05s/it]
15%|█▌ | 130/862 [49:45<3:52:27, 19.05s/it]
15%|█▌ | 131/862 [50:06<3:58:18, 19.56s/it]
15%|█▌ | 132/862 [50:24<3:50:09, 18.92s/it]
15%|█▌ | 133/862 [50:37<3:30:08, 17.30s/it]
16%|█▌ | 134/862 [50:59<3:48:02, 18.79s/it]
16%|█▌ | 135/862 [51:15<3:34:54, 17.74s/it]
16%|█▌ | 135/862 [51:15<3:34:54, 17.74s/it]
16%|█▌ | 136/862 [51:34<3:40:21, 18.21s/it]
16%|█▌ | 137/862 [51:51<3:34:13, 17.73s/it]
16%|█▌ | 138/862 [52:08<3:33:55, 17.73s/it]
16%|█▌ | 139/862 [52:29<3:42:57, 18.50s/it]
16%|█▌ | 140/862 [52:47<3:44:12, 18.63s/it]
16%|█▌ | 140/862 [52:47<3:44:12, 18.63s/it]
16%|█▋ | 141/862 [53:09<3:54:57, 19.55s/it]
16%|█▋ | 142/862 [53:36<4:20:01, 21.67s/it]
17%|█▋ | 143/862 [53:51<3:56:47, 19.76s/it]
17%|█▋ | 144/862 [54:07<3:43:33, 18.68s/it]
17%|█▋ | 145/862 [54:30<3:56:02, 19.75s/it]
17%|█▋ | 145/862 [54:30<3:56:02, 19.75s/it]
17%|█▋ | 146/862 [54:44<3:35:18, 18.04s/it]
17%|█▋ | 147/862 [55:03<3:38:32, 18.34s/it]
17%|█▋ | 148/862 [55:22<3:40:31, 18.53s/it]
17%|█▋ | 149/862 [55:40<3:39:20, 18.46s/it]
17%|█▋ | 150/862 [56:03<3:55:15, 19.83s/it]
17%|█▋ | 150/862 [56:03<3:55:15, 19.83s/it]{'eval_loss': 0.1702147275209427, 'eval_runtime': 57.6184, 'eval_samples_per_second': 6.144, 'eval_steps_per_second': 3.072, 'epoch': 0.12}
{'loss': 0.2433, 'grad_norm': 0.5323580503463745, 'learning_rate': 0.00019735672082728782, 'epoch': 0.12}
{'loss': 0.2063, 'grad_norm': 0.5639608502388, 'learning_rate': 0.0001969002027642398, 'epoch': 0.13}
{'loss': 0.2208, 'grad_norm': 0.583233654499054, 'learning_rate': 0.00019640795375746518, 'epoch': 0.13}
{'loss': 0.204, 'grad_norm': 0.5018272399902344, 'learning_rate': 0.00019588015531867464, 'epoch': 0.14}
{'loss': 0.2155, 'grad_norm': 0.48238405585289, 'learning_rate': 0.00019531700206806274, 'epoch': 0.15}
{'loss': 0.208, 'grad_norm': 0.4539037048816681, 'learning_rate': 0.00019471870166254377, 'epoch': 0.15}
{'loss': 0.2292, 'grad_norm': 0.5225255489349365, 'learning_rate': 0.00019408547471918061, 'epoch': 0.16}
{'loss': 0.2074, 'grad_norm': 0.4645092785358429, 'learning_rate': 0.00019341755473383432, 'epoch': 0.16}
{'loss': 0.2301, 'grad_norm': 0.47805657982826233, 'learning_rate': 0.00019271518799506492, 'epoch': 0.17}
{'loss': 0.1916, 'grad_norm': 0.43033567070961, 'learning_rate': 0.00019197863349331522, 'epoch': 0.17}
0%| | 0/177 [00:00<?, ?it/s][A
1%| | 2/177 [00:00<00:50, 3.50it/s][A
2%|▏ | 3/177 [00:01<01:20, 2.17it/s][A
2%|▏ | 4/177 [00:01<01:08, 2.52it/s][A
3%|▎ | 5/177 [00:01<00:50, 3.39it/s][A
3%|▎ | 6/177 [00:01<00:51, 3.35it/s][A
4%|▍ | 7/177 [00:02<00:52, 3.22it/s][A
5%|▍ | 8/177 [00:02<00:53, 3.18it/s][A
5%|▌ | 9/177 [00:03<00:55, 3.01it/s][A
6%|▌ | 10/177 [00:03<00:56, 2.93it/s][A
6%|▌ | 11/177 [00:03<01:06, 2.50it/s][A
7%|▋ | 12/177 [00:04<00:57, 2.86it/s][A
7%|▋ | 13/177 [00:04<00:59, 2.75it/s][A
8%|▊ | 14/177 [00:04<00:56, 2.88it/s][A
8%|▊ | 15/177 [00:04<00:45, 3.55it/s][A
9%|▉ | 16/177 [00:05<00:44, 3.63it/s][A
10%|▉ | 17/177 [00:05<00:44, 3.57it/s][A
11%|█ | 19/177 [00:05<00:37, 4.19it/s][A
11%|█▏ | 20/177 [00:06<00:39, 4.02it/s][A
12%|█▏ | 21/177 [00:06<00:46, 3.36it/s][A
12%|█▏ | 22/177 [00:07<01:01, 2.51it/s][A
13%|█▎ | 23/177 [00:08<01:15, 2.04it/s][A
14%|█▎ | 24/177 [00:08<01:14, 2.05it/s][A
14%|█▍ | 25/177 [00:08<01:05, 2.31it/s][A
15%|█▍ | 26/177 [00:09<00:57, 2.60it/s][A
15%|█▌ | 27/177 [00:09<00:53, 2.79it/s][A
16%|█▌ | 28/177 [00:09<00:48, 3.06it/s][A
16%|█▋ | 29/177 [00:10<00:58, 2.53it/s][A
17%|█▋ | 30/177 [00:10<01:05, 2.24it/s][A
18%|█▊ | 31/177 [00:11<01:14, 1.96it/s][A
18%|█▊ | 32/177 [00:11<01:05, 2.20it/s][A
19%|█▊ | 33/177 [00:11<00:51, 2.81it/s][A
20%|█▉ | 35/177 [00:12<00:38, 3.65it/s][A
20%|██ | 36/177 [00:12<00:37, 3.80it/s][A
21%|██ | 37/177 [00:12<00:31, 4.40it/s][A
21%|██▏ | 38/177 [00:12<00:32, 4.34it/s][A
22%|██▏ | 39/177 [00:13<00:46, 2.99it/s][A
23%|██▎ | 40/177 [00:13<00:47, 2.89it/s][A
23%|██▎ | 41/177 [00:14<00:47, 2.87it/s][A
24%|██▎ | 42/177 [00:14<00:53, 2.50it/s][A
24%|██▍ | 43/177 [00:15<01:01, 2.19it/s][A
25%|██▍ | 44/177 [00:15<00:59, 2.23it/s][A
25%|██▌ | 45/177 [00:16<01:01, 2.16it/s][A
26%|██▌ | 46/177 [00:16<01:01, 2.12it/s][A
27%|██▋ | 47/177 [00:17<01:06, 1.97it/s][A
27%|██▋ | 48/177 [00:17<01:05, 1.97it/s][A
28%|██▊ | 49/177 [00:18<01:06, 1.92it/s][A
28%|██▊ | 50/177 [00:18<01:04, 1.97it/s][A
29%|██▉ | 51/177 [00:19<01:09, 1.82it/s][A
29%|██▉ | 52/177 [00:20<01:10, 1.78it/s][A
30%|██▉ | 53/177 [00:20<01:10, 1.75it/s][A
31%|███ | 54/177 [00:20<00:58, 2.09it/s][A
31%|███ | 55/177 [00:21<00:54, 2.25it/s][A
32%|███▏ | 56/177 [00:21<00:49, 2.47it/s][A
32%|███▏ | 57/177 [00:21<00:42, 2.84it/s][A
33%|███▎ | 58/177 [00:21<00:36, 3.28it/s][A
33%|███▎ | 59/177 [00:22<00:28, 4.09it/s][A
34%|███▍ | 61/177 [00:22<00:18, 6.22it/s][A
35%|███▌ | 62/177 [00:22<00:20, 5.56it/s][A
36%|███▌ | 63/177 [00:23<00:32, 3.54it/s][A
36%|███▌ | 64/177 [00:23<00:39, 2.83it/s][A
37%|███▋ | 65/177 [00:23<00:39, 2.85it/s][A
38%|███▊ | 67/177 [00:24<00:25, 4.32it/s][A
38%|███▊ | 68/177 [00:24<00:25, 4.29it/s][A
39%|███▉ | 69/177 [00:24<00:26, 4.13it/s][A
40%|████ | 71/177 [00:25<00:24, 4.32it/s][A
41%|████ | 72/177 [00:25<00:25, 4.10it/s][A
41%|████ | 73/177 [00:25<00:22, 4.61it/s][A
42%|████▏ | 74/177 [00:25<00:24, 4.26it/s][A
42%|████▏ | 75/177 [00:26<00:26, 3.90it/s][A
44%|████▎ | 77/177 [00:26<00:24, 4.01it/s][A
44%|████▍ | 78/177 [00:27<00:31, 3.14it/s][A
45%|████▍ | 79/177 [00:27<00:32, 3.03it/s][A
45%|████▌ | 80/177 [00:27<00:30, 3.19it/s][A
46%|████▌ | 81/177 [00:28<00:30, 3.18it/s][A
46%|████▋ | 82/177 [00:28<00:30, 3.12it/s][A
47%|████▋ | 83/177 [00:28<00:38, 2.43it/s][A
47%|████▋ | 84/177 [00:29<00:35, 2.59it/s][A
49%|████▊ | 86/177 [00:29<00:22, 4.06it/s][A
50%|████▉ | 88/177 [00:29<00:16, 5.36it/s][A
50%|█████ | 89/177 [00:29<00:14, 5.93it/s][A
51%|█████▏ | 91/177 [00:29<00:10, 7.86it/s][A
53%|█████▎ | 93/177 [00:30<00:13, 6.22it/s][A
53%|█████▎ | 94/177 [00:30<00:17, 4.70it/s][A
54%|█████▎ | 95/177 [00:31<00:23, 3.47it/s][A
54%|█████▍ | 96/177 [00:31<00:25, 3.16it/s][A
55%|█████▍ | 97/177 [00:32<00:32, 2.44it/s][A
55%|█████▌ | 98/177 [00:32<00:30, 2.58it/s][A
56%|█████▌ | 99/177 [00:33<00:34, 2.23it/s][A
56%|█████▋ | 100/177 [00:33<00:36, 2.11it/s][A
57%|█████▋ | 101/177 [00:34<00:37, 2.02it/s][A
58%|█████▊ | 102/177 [00:34<00:35, 2.11it/s][A
58%|█████▊ | 103/177 [00:35<00:31, 2.32it/s][A
59%|█████▉ | 104/177 [00:35<00:26, 2.72it/s][A
59%|█████▉ | 105/177 [00:35<00:27, 2.65it/s][A
60%|█████▉ | 106/177 [00:36<00:26, 2.70it/s][A
60%|██████ | 107/177 [00:36<00:23, 2.93it/s][A
62%|██████▏ | 109/177 [00:36<00:14, 4.57it/s][A
63%|██████▎ | 111/177 [00:36<00:14, 4.68it/s][A
63%|██████▎ | 112/177 [00:37<00:14, 4.35it/s][A
64%|██████▍ | 113/177 [00:37<00:17, 3.58it/s][A
64%|██████▍ | 114/177 [00:38<00:18, 3.38it/s][A
65%|██████▍ | 115/177 [00:38<00:18, 3.42it/s][A
66%|██████▌ | 116/177 [00:38<00:16, 3.66it/s][A
66%|██████▌ | 117/177 [00:38<00:13, 4.39it/s][A
67%|██████▋ | 118/177 [00:38<00:14, 4.09it/s][A
67%|██████▋ | 119/177 [00:39<00:19, 2.97it/s][A
68%|██████▊ | 120/177 [00:39<00:18, 3.09it/s][A
68%|██████▊ | 121/177 [00:39<00:14, 3.83it/s][A
69%|██████▉ | 123/177 [00:40<00:11, 4.75it/s][A
70%|███████ | 124/177 [00:40<00:12, 4.10it/s][A
71%|███████ | 125/177 [00:41<00:17, 2.99it/s][A
71%|███████ | 126/177 [00:41<00:17, 2.98it/s][A
72%|███████▏ | 127/177 [00:41<00:18, 2.77it/s][A
72%|███████▏ | 128/177 [00:42<00:17, 2.87it/s][A
73%|███████▎ | 129/177 [00:42<00:16, 2.85it/s][A
73%|███████▎ | 130/177 [00:43<00:18, 2.59it/s][A
74%|███████▍ | 131/177 [00:43<00:19, 2.42it/s][A
75%|███████▍ | 132/177 [00:43<00:15, 2.85it/s][A
75%|███████▌ | 133/177 [00:44<00:15, 2.93it/s][A
76%|███████▌ | 134/177 [00:44<00:16, 2.62it/s][A
76%|███████▋ | 135/177 [00:44<00:15, 2.73it/s][A
77%|███████▋ | 137/177 [00:44<00:09, 4.29it/s][A
79%|███████▊ | 139/177 [00:45<00:06, 5.68it/s][A
79%|███████▉ | 140/177 [00:45<00:07, 5.23it/s][A
80%|███████▉ | 141/177 [00:45<00:09, 3.71it/s][A
80%|████████ | 142/177 [00:46<00:11, 3.17it/s][A
81%|████████ | 143/177 [00:46<00:12, 2.65it/s][A
81%|████████▏ | 144/177 [00:47<00:13, 2.51it/s][A
82%|████████▏ | 145/177 [00:47<00:14, 2.20it/s][A
82%|████████▏ | 146/177 [00:48<00:14, 2.15it/s][A
83%|████████▎ | 147/177 [00:49<00:14, 2.00it/s][A
84%|████████▎ | 148/177 [00:49<00:14, 1.94it/s][A
84%|████████▍ | 149/177 [00:50<00:13, 2.08it/s][A
85%|████████▌ | 151/177 [00:50<00:09, 2.66it/s][A
86%|████████▌ | 152/177 [00:50<00:09, 2.70it/s][A
86%|████████▋ | 153/177 [00:51<00:08, 2.90it/s][A
87%|████████▋ | 154/177 [00:51<00:07, 3.27it/s][A
88%|████████▊ | 155/177 [00:51<00:05, 3.84it/s][A
89%|████████▊ | 157/177 [00:51<00:03, 5.36it/s][A
89%|████████▉ | 158/177 [00:51<00:04, 4.74it/s][A
90%|████████▉ | 159/177 [00:52<00:04, 4.16it/s][A
91%|█████████ | 161/177 [00:52<00:02, 5.62it/s][A
92%|█████████▏| 163/177 [00:52<00:02, 5.90it/s][A
93%|█████████▎| 164/177 [00:53<00:02, 4.66it/s][A
93%|█████████▎| 165/177 [00:53<00:03, 3.30it/s][A
94%|█████████▍| 166/177 [00:54<00:03, 2.85it/s][A
94%|█████████▍| 167/177 [00:54<00:03, 2.85it/s][A
95%|█████████▍| 168/177 [00:55<00:03, 2.66it/s][A
95%|█████████▌| 169/177 [00:55<00:03, 2.58it/s][A
96%|█████████▌| 170/177 [00:55<00:02, 2.75it/s][A
97%|█████████▋| 171/177 [00:56<00:02, 2.82it/s][A
97%|█████████▋| 172/177 [00:56<00:01, 2.76it/s][A
98%|█████████▊| 173/177 [00:56<00:01, 2.72it/s][A
99%|█████████▉| 175/177 [00:57<00:00, 4.18it/s][A
99%|█████████▉| 176/177 [00:57<00:00, 3.88it/s][A
100%|██████████| 177/177 [00:57<00:00, 3.64it/s][A
[A
17%|█▋ | 150/862 [57:01<3:55:15, 19.83s/it]
100%|██████████| 177/177 [00:57<00:00, 3.64it/s][A
[A
18%|█▊ | 151/862 [57:22<7:27:21, 37.75s/it]
18%|█▊ | 152/862 [57:42<6:22:24, 32.32s/it]
18%|█▊ | 153/862 [58:06<5:50:44, 29.68s/it]
18%|█▊ | 154/862 [58:26<5:19:01, 27.04s/it]
18%|█▊ | 155/862 [58:44<4:43:55, 24.09s/it]
18%|█▊ | 155/862 [58:44<4:43:55, 24.09s/it]
18%|█▊ | 156/862 [59:06<4:35:27, 23.41s/it]
18%|█▊ | 157/862 [59:24<4:16:18, 21.81s/it]
18%|█▊ | 158/862 [59:44<4:10:49, 21.38s/it]
18%|█▊ | 159/862 [1:00:00<3:53:14, 19.91s/it]
19%|█▊ | 160/862 [1:00:14<3:29:48, 17.93s/it]
19%|█▊ | 160/862 [1:00:14<3:29:48, 17.93s/it]
19%|█▊ | 161/862 [1:00:37<3:47:19, 19.46s/it]
19%|█▉ | 162/862 [1:00:49<3:19:55, 17.14s/it]
19%|█▉ | 163/862 [1:01:09<3:30:08, 18.04s/it]
19%|█▉ | 164/862 [1:01:29<3:36:40, 18.63s/it]
19%|█▉ | 165/862 [1:01:46<3:32:47, 18.32s/it]
19%|█▉ | 165/862 [1:01:46<3:32:47, 18.32s/it]
19%|█▉ | 166/862 [1:02:01<3:18:20, 17.10s/it]
19%|█▉ | 167/862 [1:02:12<2:59:18, 15.48s/it]
19%|█▉ | 168/862 [1:02:31<3:11:46, 16.58s/it]
20%|█▉ | 169/862 [1:02:48<3:12:26, 16.66s/it]
20%|█▉ | 170/862 [1:03:07<3:20:34, 17.39s/it]
20%|█▉ | 170/862 [1:03:07<3:20:34, 17.39s/it]
20%|█▉ | 171/862 [1:03:26<3:25:45, 17.87s/it]
20%|█▉ | 172/862 [1:03:43<3:22:05, 17.57s/it]
20%|██ | 173/862 [1:03:55<3:01:00, 15.76s/it]
20%|██ | 174/862 [1:04:08<2:50:50, 14.90s/it]
20%|██ | 175/862 [1:04:27<3:04:32, 16.12s/it]
20%|██ | 175/862 [1:04:27<3:04:32, 16.12s/it]
20%|██ | 176/862 [1:04:38<2:47:13, 14.63s/it]
21%|██ | 177/862 [1:04:53<2:50:23, 14.93s/it]
21%|██ | 178/862 [1:05:13<3:05:55, 16.31s/it]
21%|██ | 179/862 [1:05:30<3:07:24, 16.46s/it]
21%|██ | 180/862 [1:05:47<3:08:54, 16.62s/it]
21%|██ | 180/862 [1:05:47<3:08:54, 16.62s/it]
21%|██ | 181/862 [1:06:03<3:09:22, 16.68s/it]
21%|██ | 182/862 [1:06:20<3:07:12, 16.52s/it]
21%|██ | 183/862 [1:06:36<3:05:21, 16.38s/it]
21%|██▏ | 184/862 [1:07:00<3:32:38, 18.82s/it]
21%|██▏ | 185/862 [1:07:18<3:28:28, 18.48s/it]
21%|██▏ | 185/862 [1:07:18<3:28:28, 18.48s/it]
22%|██▏ | 186/862 [1:07:37<3:28:54, 18.54s/it]
22%|██▏ | 187/862 [1:07:51<3:15:56, 17.42s/it]
22%|██▏ | 188/862 [1:08:10<3:18:28, 17.67s/it]
22%|██▏ | 189/862 [1:08:32<3:35:34, 19.22s/it]
22%|██▏ | 190/862 [1:08:56<3:51:16, 20.65s/it]
22%|██▏ | 190/862 [1:08:56<3:51:16, 20.65s/it]
22%|██▏ | 191/862 [1:09:14<3:41:03, 19.77s/it]
22%|██▏ | 192/862 [1:09:32<3:33:28, 19.12s/it]
22%|██▏ | 193/862 [1:09:45<3:12:46, 17.29s/it]
23%|██▎ | 194/862 [1:09:56<2:51:10, 15.38s/it]
23%|██▎ | 195/862 [1:10:12<2:52:26, 15.51s/it]
23%|██▎ | 195/862 [1:10:12<2:52:26, 15.51s/it]
23%|██▎ | 196/862 [1:10:30<3:03:26, 16.53s/it]
23%|██▎ | 197/862 [1:10:46<3:00:45, 16.31s/it]
23%|██▎ | 198/862 [1:11:07<3:13:45, 17.51s/it]
23%|██▎ | 199/862 [1:11:24<3:13:01, 17.47s/it]
23%|██▎ | 200/862 [1:11:36<2:56:00, 15.95s/it]
23%|██▎ | 200/862 [1:11:36<2:56:00, 15.95s/it]{'eval_loss': 0.1569332629442215, 'eval_runtime': 57.9274, 'eval_samples_per_second': 6.111, 'eval_steps_per_second': 3.056, 'epoch': 0.17}
{'loss': 0.2054, 'grad_norm': 0.5715239644050598, 'learning_rate': 0.00019120816282541063, 'epoch': 0.18}
{'loss': 0.2156, 'grad_norm': 0.5529009699821472, 'learning_rate': 0.00019040406009441073, 'epoch': 0.19}
{'loss': 0.227, 'grad_norm': 0.5217124819755554, 'learning_rate': 0.00018956662180484913, 'epoch': 0.19}
{'loss': 0.2191, 'grad_norm': 0.4713853895664215, 'learning_rate': 0.00018869615675340068, 'epoch': 0.2}
{'loss': 0.2052, 'grad_norm': 0.44989559054374695, 'learning_rate': 0.00018779298591501564, 'epoch': 0.2}
{'loss': 0.2009, 'grad_norm': 0.5454024076461792, 'learning_rate': 0.00018685744232456374, 'epoch': 0.21}
{'loss': 0.175, 'grad_norm': 0.5243939161300659, 'learning_rate': 0.0001858898709540309, 'epoch': 0.21}
{'loss': 0.1861, 'grad_norm': 0.4877641499042511, 'learning_rate': 0.0001848906285853145, 'epoch': 0.22}
{'loss': 0.1775, 'grad_norm': 0.45391249656677246, 'learning_rate': 0.0001838600836786635, 'epoch': 0.23}
{'loss': 0.1791, 'grad_norm': 0.49695494771003723, 'learning_rate': 0.0001827986162368129, 'epoch': 0.23}
0%| | 0/177 [00:00<?, ?it/s][A
1%| | 2/177 [00:00<00:49, 3.52it/s][A
2%|▏ | 3/177 [00:01<01:20, 2.17it/s][A
2%|▏ | 4/177 [00:01<01:08, 2.52it/s][A
3%|▎ | 5/177 [00:01<00:52, 3.26it/s][A
3%|▎ | 6/177 [00:02<00:52, 3.26it/s][A
4%|▍ | 7/177 [00:02<00:53, 3.17it/s][A
5%|▍ | 8/177 [00:02<00:53, 3.14it/s][A
5%|▌ | 9/177 [00:03<00:56, 2.99it/s][A
6%|▌ | 10/177 [00:03<00:57, 2.92it/s][A
6%|▌ | 11/177 [00:03<01:06, 2.50it/s][A
7%|▋ | 12/177 [00:04<00:58, 2.84it/s][A
7%|▋ | 13/177 [00:04<00:59, 2.76it/s][A
8%|▊ | 14/177 [00:04<00:56, 2.89it/s][A
8%|▊ | 15/177 [00:04<00:44, 3.65it/s][A
9%|▉ | 16/177 [00:05<00:43, 3.70it/s][A
10%|▉ | 17/177 [00:05<00:44, 3.62it/s][A
11%|█ | 19/177 [00:05<00:37, 4.19it/s][A
11%|█▏ | 20/177 [00:06<00:39, 3.96it/s][A
12%|█▏ | 21/177 [00:06<00:47, 3.30it/s][A
12%|█▏ | 22/177 [00:07<01:01, 2.51it/s][A
13%|█▎ | 23/177 [00:08<01:16, 2.02it/s][A
14%|█▎ | 24/177 [00:08<01:14, 2.04it/s][A
14%|█▍ | 25/177 [00:08<01:07, 2.26it/s][A
15%|█▍ | 26/177 [00:09<00:58, 2.59it/s][A
15%|█▌ | 27/177 [00:09<00:53, 2.78it/s][A
16%|█▌ | 28/177 [00:09<00:48, 3.05it/s][A
16%|█▋ | 29/177 [00:10<00:58, 2.53it/s][A
17%|█▋ | 30/177 [00:10<01:05, 2.24it/s][A
18%|█▊ | 31/177 [00:11<01:14, 1.96it/s][A
18%|█▊ | 32/177 [00:11<01:05, 2.21it/s][A
19%|█▊ | 33/177 [00:11<00:51, 2.81it/s][A
20%|█▉ | 35/177 [00:12<00:38, 3.66it/s][A
20%|██ | 36/177 [00:12<00:37, 3.79it/s][A
21%|██ | 37/177 [00:12<00:31, 4.38it/s][A
21%|██▏ | 38/177 [00:12<00:32, 4.31it/s][A
22%|██▏ | 39/177 [00:13<00:46, 2.98it/s][A
23%|██▎ | 40/177 [00:13<00:47, 2.88it/s][A
23%|██▎ | 41/177 [00:14<00:47, 2.86it/s][A
24%|██▎ | 42/177 [00:14<00:54, 2.50it/s][A
24%|██▍ | 43/177 [00:15<01:01, 2.19it/s][A
25%|██▍ | 44/177 [00:15<00:59, 2.22it/s][A
25%|██▌ | 45/177 [00:16<01:00, 2.17it/s][A
26%|██▌ | 46/177 [00:16<01:01, 2.13it/s][A
27%|██▋ | 47/177 [00:17<01:05, 2.00it/s][A
27%|██▋ | 48/177 [00:17<01:04, 1.99it/s][A
28%|██▊ | 49/177 [00:18<01:05, 1.94it/s][A
28%|██▊ | 50/177 [00:18<01:03, 1.99it/s][A
29%|██▉ | 51/177 [00:19<01:08, 1.83it/s][A
29%|██▉ | 52/177 [00:20<01:09, 1.79it/s][A
30%|██▉ | 53/177 [00:20<01:10, 1.76it/s][A
31%|███ | 54/177 [00:20<00:58, 2.10it/s][A
31%|███ | 55/177 [00:21<00:54, 2.25it/s][A
32%|███▏ | 56/177 [00:21<00:49, 2.46it/s][A
32%|███▏ | 57/177 [00:21<00:44, 2.73it/s][A
33%|███▎ | 58/177 [00:21<00:35, 3.32it/s][A
33%|███▎ | 59/177 [00:22<00:32, 3.65it/s][A
34%|███▍ | 61/177 [00:22<00:20, 5.68it/s][A
35%|███▌ | 62/177 [00:22<00:22, 5.22it/s][A
36%|███▌ | 63/177 [00:23<00:33, 3.43it/s][A
36%|███▌ | 64/177 [00:23<00:40, 2.77it/s][A
37%|███▋ | 65/177 [00:24<00:39, 2.80it/s][A
38%|███▊ | 67/177 [00:24<00:25, 4.24it/s][A
38%|███▊ | 68/177 [00:24<00:25, 4.23it/s][A
39%|███▉ | 69/177 [00:24<00:26, 4.11it/s][A
40%|████ | 71/177 [00:25<00:24, 4.28it/s][A
41%|████ | 72/177 [00:25<00:25, 4.11it/s][A
41%|████ | 73/177 [00:25<00:22, 4.61it/s][A
42%|████▏ | 74/177 [00:25<00:24, 4.26it/s][A
42%|████▏ | 75/177 [00:26<00:26, 3.89it/s][A
44%|████▎ | 77/177 [00:26<00:24, 4.01it/s][A
44%|████▍ | 78/177 [00:27<00:31, 3.15it/s][A
45%|████▍ | 79/177 [00:27<00:32, 3.04it/s][A
45%|████▌ | 80/177 [00:27<00:30, 3.19it/s][A
46%|████▌ | 81/177 [00:28<00:30, 3.18it/s][A
46%|████▋ | 82/177 [00:28<00:30, 3.12it/s][A
47%|████▋ | 83/177 [00:29<00:38, 2.43it/s][A
47%|████▋ | 84/177 [00:29<00:36, 2.58it/s][A
49%|████▊ | 86/177 [00:29<00:22, 4.05it/s][A
50%|████▉ | 88/177 [00:29<00:16, 5.31it/s][A
50%|█████ | 89/177 [00:29<00:14, 5.89it/s][A
51%|█████▏ | 91/177 [00:30<00:11, 7.79it/s][A
53%|█████▎ | 93/177 [00:30<00:13, 6.23it/s][A
53%|█████▎ | 94/177 [00:30<00:17, 4.69it/s][A
54%|█████▎ | 95/177 [00:31<00:23, 3.47it/s][A
54%|█████▍ | 96/177 [00:31<00:25, 3.16it/s][A
55%|█████▍ | 97/177 [00:32<00:28, 2.82it/s][A
55%|█████▌ | 98/177 [00:32<00:30, 2.55it/s][A
56%|█████▌ | 99/177 [00:33<00:35, 2.20it/s][A
56%|█████▋ | 100/177 [00:33<00:36, 2.09it/s][A
57%|█████▋ | 101/177 [00:34<00:37, 2.02it/s][A
58%|█████▊ | 102/177 [00:34<00:35, 2.11it/s][A
58%|█████▊ | 103/177 [00:35<00:31, 2.33it/s][A
59%|█████▉ | 104/177 [00:35<00:26, 2.72it/s][A
59%|█████▉ | 105/177 [00:35<00:27, 2.65it/s][A
60%|█████▉ | 106/177 [00:36<00:26, 2.70it/s][A
60%|██████ | 107/177 [00:36<00:23, 2.94it/s][A
62%|██████▏ | 109/177 [00:36<00:14, 4.59it/s][A
63%|██████▎ | 111/177 [00:37<00:14, 4.68it/s][A
63%|██████▎ | 112/177 [00:37<00:14, 4.37it/s][A
64%|██████▍ | 113/177 [00:37<00:17, 3.59it/s][A
64%|██████▍ | 114/177 [00:38<00:18, 3.38it/s][A
65%|██████▍ | 115/177 [00:38<00:18, 3.43it/s][A
66%|██████▌ | 116/177 [00:38<00:16, 3.66it/s][A
66%|██████▌ | 117/177 [00:38<00:13, 4.40it/s][A
67%|██████▋ | 118/177 [00:38<00:14, 4.09it/s][A
67%|██████▋ | 119/177 [00:39<00:19, 2.98it/s][A
68%|██████▊ | 120/177 [00:39<00:18, 3.10it/s][A
68%|██████▊ | 121/177 [00:39<00:14, 3.85it/s][A
69%|██████▉ | 123/177 [00:40<00:11, 4.77it/s][A
70%|███████ | 124/177 [00:40<00:12, 4.12it/s][A
71%|███████ | 125/177 [00:41<00:17, 3.01it/s][A
71%|███████ | 126/177 [00:41<00:17, 2.99it/s][A
72%|███████▏ | 127/177 [00:41<00:18, 2.77it/s][A
72%|███████▏ | 128/177 [00:42<00:17, 2.81it/s][A
73%|███████▎ | 129/177 [00:42<00:16, 2.82it/s][A
73%|███████▎ | 130/177 [00:43<00:18, 2.57it/s][A
74%|███████▍ | 131/177 [00:43<00:19, 2.38it/s][A
75%|███████▍ | 132/177 [00:43<00:16, 2.74it/s][A
75%|███████▌ | 133/177 [00:44<00:15, 2.81it/s][A
76%|███████▌ | 134/177 [00:44<00:16, 2.57it/s][A
76%|███████▋ | 135/177 [00:45<00:15, 2.65it/s][A
77%|███████▋ | 137/177 [00:45<00:09, 4.09it/s][A
79%|███████▊ | 139/177 [00:45<00:07, 5.43it/s][A
79%|███████▉ | 140/177 [00:45<00:07, 5.07it/s][A
80%|███████▉ | 141/177 [00:46<00:09, 3.66it/s][A
80%|████████ | 142/177 [00:46<00:11, 3.14it/s][A
81%|████████ | 143/177 [00:47<00:12, 2.63it/s][A
81%|████████▏ | 144/177 [00:47<00:13, 2.50it/s][A
82%|████████▏ | 145/177 [00:48<00:14, 2.19it/s][A
82%|████████▏ | 146/177 [00:48<00:14, 2.15it/s][A
83%|████████▎ | 147/177 [00:49<00:15, 2.00it/s][A
84%|████████▎ | 148/177 [00:49<00:14, 1.94it/s][A
84%|████████▍ | 149/177 [00:50<00:13, 2.08it/s][A
85%|████████▌ | 151/177 [00:50<00:09, 2.67it/s][A
86%|████████▌ | 152/177 [00:51<00:09, 2.72it/s][A
86%|████████▋ | 153/177 [00:51<00:08, 2.92it/s][A
87%|████████▋ | 154/177 [00:51<00:06, 3.31it/s][A
88%|████████▊ | 155/177 [00:51<00:05, 3.96it/s][A
89%|████████▊ | 157/177 [00:51<00:03, 5.69it/s][A
89%|████████▉ | 158/177 [00:52<00:03, 4.87it/s][A
90%|████████▉ | 159/177 [00:52<00:04, 4.38it/s][A
91%|█████████ | 161/177 [00:52<00:02, 6.03it/s][A
92%|█████████▏| 163/177 [00:52<00:02, 6.45it/s][A
93%|█████████▎| 164/177 [00:53<00:02, 4.91it/s][A
93%|█████████▎| 165/177 [00:53<00:03, 3.45it/s][A
94%|█████████▍| 166/177 [00:54<00:03, 2.92it/s][A
94%|█████████▍| 167/177 [00:54<00:03, 2.92it/s][A
95%|█████████▍| 168/177 [00:55<00:03, 2.71it/s][A
95%|█████████▌| 169/177 [00:55<00:03, 2.62it/s][A
96%|█████████▌| 170/177 [00:55<00:02, 2.78it/s][A
97%|█████████▋| 171/177 [00:56<00:02, 2.85it/s][A
97%|█████████▋| 172/177 [00:56<00:01, 2.78it/s][A
98%|█████████▊| 173/177 [00:56<00:01, 2.74it/s][A
99%|█████████▉| 175/177 [00:57<00:00, 4.21it/s][A
99%|█████████▉| 176/177 [00:57<00:00, 3.90it/s][A
100%|██████████| 177/177 [00:57<00:00, 3.66it/s][A
[A
23%|██▎ | 200/862 [1:12:34<2:56:00, 15.95s/it]
100%|██████████| 177/177 [00:57<00:00, 3.66it/s][A
[Awandb: WARNING URL not available in offline run
`rope_scaling`'s beta_fast field must be a float, got 32
`rope_scaling`'s beta_slow field must be a float, got 1
23%|██▎ | 201/862 [1:12:52<6:12:46, 33.84s/it]
23%|██▎ | 202/862 [1:13:19<5:51:25, 31.95s/it]
24%|██▎ | 203/862 [1:14:05<6:36:52, 36.13s/it]
24%|██▎ | 204/862 [1:14:49<7:02:16, 38.51s/it]
24%|██▍ | 205/862 [1:15:10<6:03:43, 33.22s/it]
24%|██▍ | 205/862 [1:15:10<6:03:43, 33.22s/it]
24%|██▍ | 206/862 [1:15:31<5:22:17, 29.48s/it]
24%|██▍ | 207/862 [1:15:53<4:58:34, 27.35s/it]
24%|██▍ | 208/862 [1:16:09<4:19:59, 23.85s/it]
24%|██▍ | 209/862 [1:16:40<4:44:17, 26.12s/it]
24%|██▍ | 210/862 [1:16:58<4:15:58, 23.56s/it]
24%|██▍ | 210/862 [1:16:58<4:15:58, 23.56s/it]
24%|██▍ | 211/862 [1:17:16<3:57:09, 21.86s/it]
25%|██▍ | 212/862 [1:17:28<3:26:21, 19.05s/it]
25%|██▍ | 213/862 [1:17:43<3:12:50, 17.83s/it]
25%|██▍ | 214/862 [1:17:57<2:57:37, 16.45s/it]
25%|██▍ | 215/862 [1:18:08<2:40:25, 14.88s/it]
25%|██▍ | 215/862 [1:18:08<2:40:25, 14.88s/it]
25%|██▌ | 216/862 [1:18:25<2:46:12, 15.44s/it]
25%|██▌ | 217/862 [1:18:40<2:46:09, 15.46s/it]
25%|██▌ | 218/862 [1:18:57<2:51:56, 16.02s/it]
25%|██▌ | 219/862 [1:19:09<2:37:44, 14.72s/it]
26%|██▌ | 220/862 [1:19:24<2:38:33, 14.82s/it]
26%|██▌ | 220/862 [1:19:24<2:38:33, 14.82s/it]
26%|██▌ | 221/862 [1:19:47<3:02:28, 17.08s/it]
26%|██▌ | 222/862 [1:19:59<2:48:16, 15.78s/it]
26%|██▌ | 223/862 [1:20:19<2:59:42, 16.87s/it]
26%|██▌ | 224/862 [1:20:36<3:00:20, 16.96s/it]
26%|██▌ | 225/862 [1:20:48<2:45:04, 15.55s/it]
26%|██▌ | 225/862 [1:20:48<2:45:04, 15.55s/it]
26%|██▌ | 226/862 [1:20:57<2:22:05, 13.41s/it]
26%|██▋ | 227/862 [1:21:13<2:31:22, 14.30s/it]
26%|██▋ | 228/862 [1:21:27<2:29:56, 14.19s/it]
27%|██▋ | 229/862 [1:21:43<2:36:02, 14.79s/it]
27%|██▋ | 230/862 [1:21:57<2:33:46, 14.60s/it]
27%|██▋ | 230/862 [1:21:57<2:33:46, 14.60s/it]
27%|██▋ | 231/862 [1:22:11<2:31:44, 14.43s/it]
27%|██▋ | 232/862 [1:22:34<2:57:54, 16.94s/it]
27%|██▋ | 233/862 [1:22:49<2:51:51, 16.39s/it]
27%|██▋ | 234/862 [1:23:04<2:45:18, 15.79s/it]
27%|██▋ | 235/862 [1:23:31<3:20:55, 19.23s/it]
27%|██▋ | 235/862 [1:23:31<3:20:55, 19.23s/it]
27%|██▋ | 236/862 [1:23:44<3:00:26, 17.30s/it]
27%|██▋ | 237/862 [1:24:04<3:09:25, 18.18s/it]
28%|██▊ | 238/862 [1:24:19<2:59:09, 17.23s/it]
28%|██▊ | 239/862 [1:24:40<3:09:53, 18.29s/it]
28%|██▊ | 240/862 [1:24:58<3:10:34, 18.38s/it]
28%|██▊ | 240/862 [1:24:58<3:10:34, 18.38s/it]
28%|██▊ | 241/862 [1:25:11<2:52:17, 16.65s/it]
28%|██▊ | 242/862 [1:25:29<2:55:59, 17.03s/it]
28%|██▊ | 243/862 [1:25:47<3:00:30, 17.50s/it]
28%|██▊ | 244/862 [1:26:02<2:52:50, 16.78s/it]
28%|██▊ | 245/862 [1:26:14<2:36:15, 15.20s/it]
28%|██▊ | 245/862 [1:26:14<2:36:15, 15.20s/it]
29%|██▊ | 246/862 [1:26:27<2:29:29, 14.56s/it]
29%|██▊ | 247/862 [1:26:41<2:27:27, 14.39s/it]
29%|██▉ | 248/862 [1:26:54<2:24:03, 14.08s/it]
29%|██▉ | 249/862 [1:27:14<2:39:40, 15.63s/it]
29%|██▉ | 250/862 [1:27:31<2:44:38, 16.14s/it]
29%|██▉ | 250/862 [1:27:31<2:44:38, 16.14s/it]{'eval_loss': 0.14798370003700256, 'eval_runtime': 57.9291, 'eval_samples_per_second': 6.111, 'eval_steps_per_second': 3.055, 'epoch': 0.23}
{'loss': 0.2095, 'grad_norm': 0.3668426275253296, 'learning_rate': 0.0001817066176648615, 'epoch': 0.24}
{'loss': 0.2096, 'grad_norm': 0.48783811926841736, 'learning_rate': 0.0001805844906259452, 'epoch': 0.24}
{'loss': 0.1579, 'grad_norm': 0.3652338981628418, 'learning_rate': 0.00017943264889275944, 'epoch': 0.25}
{'loss': 0.169, 'grad_norm': 0.4943305552005768, 'learning_rate': 0.00017825151719498466, 'epoch': 0.26}
{'loss': 0.1633, 'grad_norm': 0.4297771453857422, 'learning_rate': 0.0001770415310626715, 'epoch': 0.26}
{'loss': 0.1713, 'grad_norm': 0.5426889657974243, 'learning_rate': 0.00017580313666564395, 'epoch': 0.27}
{'loss': 0.1921, 'grad_norm': 0.5065458416938782, 'learning_rate': 0.0001745367906489786, 'epoch': 0.27}
{'loss': 0.2107, 'grad_norm': 0.452774316072464, 'learning_rate': 0.00017324295996462146, 'epoch': 0.28}
{'loss': 0.1468, 'grad_norm': 0.37844130396842957, 'learning_rate': 0.00017192212169920459, 'epoch': 0.28}
{'loss': 0.1421, 'grad_norm': 0.3819119930267334, 'learning_rate': 0.00017057476289812504, 'epoch': 0.29}
0%| | 0/177 [00:00<?, ?it/s][A
1%| | 2/177 [00:00<00:50, 3.49it/s][A
2%|▏ | 3/177 [00:01<01:22, 2.11it/s][A
2%|▏ | 4/177 [00:01<01:10, 2.47it/s][A
3%|▎ | 5/177 [00:01<00:52, 3.29it/s][A
3%|▎ | 6/177 [00:02<00:52, 3.27it/s][A
4%|▍ | 7/177 [00:02<00:54, 3.11it/s][A
5%|▍ | 8/177 [00:02<00:54, 3.08it/s][A
5%|▌ | 9/177 [00:03<00:57, 2.92it/s][A
6%|▌ | 10/177 [00:03<00:58, 2.87it/s][A
6%|▌ | 11/177 [00:03<01:07, 2.47it/s][A
7%|▋ | 12/177 [00:04<00:58, 2.83it/s][A
7%|▋ | 13/177 [00:04<00:59, 2.75it/s][A
8%|▊ | 14/177 [00:04<00:56, 2.88it/s][A
8%|▊ | 15/177 [00:05<00:44, 3.65it/s][A
9%|▉ | 16/177 [00:05<00:43, 3.71it/s][A
10%|▉ | 17/177 [00:05<00:44, 3.62it/s][A
11%|█ | 19/177 [00:05<00:37, 4.23it/s][A
11%|█▏ | 20/177 [00:06<00:38, 4.06it/s][A
12%|█▏ | 21/177 [00:06<00:46, 3.37it/s][A
12%|█▏ | 22/177 [00:07<01:01, 2.54it/s][A
13%|█▎ | 23/177 [00:08<01:14, 2.06it/s][A
14%|█▎ | 24/177 [00:08<01:14, 2.06it/s][A
14%|█▍ | 25/177 [00:08<01:05, 2.33it/s][A
15%|█▍ | 26/177 [00:09<00:57, 2.61it/s][A
15%|█▌ | 27/177 [00:09<00:53, 2.80it/s][A
16%|█▌ | 28/177 [00:09<00:48, 3.06it/s][A
16%|█▋ | 29/177 [00:10<00:58, 2.54it/s][A
17%|█▋ | 30/177 [00:10<01:05, 2.25it/s][A
18%|█▊ | 31/177 [00:11<01:14, 1.97it/s][A
18%|█▊ | 32/177 [00:11<01:05, 2.21it/s][A
19%|█▊ | 33/177 [00:11<00:51, 2.82it/s][A
20%|█▉ | 35/177 [00:12<00:38, 3.67it/s][A
20%|██ | 36/177 [00:12<00:37, 3.80it/s][A
21%|██ | 37/177 [00:12<00:31, 4.40it/s][A
21%|██▏ | 38/177 [00:12<00:32, 4.33it/s][A
22%|██▏ | 39/177 [00:13<00:46, 2.98it/s][A
23%|██▎ | 40/177 [00:13<00:47, 2.88it/s][A
23%|██▎ | 41/177 [00:14<00:47, 2.87it/s][A
24%|██▎ | 42/177 [00:14<00:53, 2.50it/s][A
24%|██▍ | 43/177 [00:15<01:01, 2.20it/s][A
25%|██▍ | 44/177 [00:15<00:59, 2.23it/s][A
25%|██▌ | 45/177 [00:16<01:00, 2.18it/s][A
26%|██▌ | 46/177 [00:16<01:01, 2.14it/s][A
27%|██▋ | 47/177 [00:17<01:04, 2.00it/s][A
27%|██▋ | 48/177 [00:17<01:04, 2.00it/s][A
28%|██▊ | 49/177 [00:18<01:05, 1.95it/s][A
28%|██▊ | 50/177 [00:18<01:03, 1.99it/s][A
29%|██▉ | 51/177 [00:19<01:08, 1.84it/s][A
29%|██▉ | 52/177 [00:19<01:09, 1.79it/s][A
30%|██▉ | 53/177 [00:20<01:10, 1.76it/s][A
31%|███ | 54/177 [00:20<00:58, 2.11it/s][A
31%|███ | 55/177 [00:21<00:53, 2.26it/s][A
32%|███▏ | 56/177 [00:21<00:48, 2.48it/s][A
32%|███▏ | 57/177 [00:21<00:42, 2.85it/s][A
33%|███▎ | 58/177 [00:21<00:36, 3.30it/s][A
33%|███▎ | 59/177 [00:22<00:28, 4.10it/s][A
34%|███▍ | 61/177 [00:22<00:18, 6.22it/s][A
35%|███▌ | 62/177 [00:22<00:20, 5.55it/s][A
36%|███▌ | 63/177 [00:22<00:32, 3.54it/s][A
36%|███▌ | 64/177 [00:23<00:40, 2.82it/s][A
37%|███▋ | 65/177 [00:23<00:39, 2.85it/s][A
38%|███▊ | 67/177 [00:24<00:25, 4.30it/s][A
38%|███▊ | 68/177 [00:24<00:25, 4.27it/s][A
39%|███▉ | 69/177 [00:24<00:26, 4.13it/s][A
40%|████ | 71/177 [00:24<00:24, 4.33it/s][A
41%|████ | 72/177 [00:25<00:25, 4.11it/s][A
41%|████ | 73/177 [00:25<00:22, 4.62it/s][A
42%|████▏ | 74/177 [00:25<00:24, 4.27it/s][A
42%|████▏ | 75/177 [00:26<00:26, 3.91it/s][A
44%|████▎ | 77/177 [00:26<00:30, 3.31it/s][A
44%|████▍ | 78/177 [00:27<00:33, 2.98it/s][A
45%|████▍ | 79/177 [00:27<00:33, 2.91it/s][A
45%|████▌ | 80/177 [00:27<00:31, 3.08it/s][A
46%|████▌ | 81/177 [00:28<00:30, 3.10it/s][A
46%|████▋ | 82/177 [00:28<00:31, 3.00it/s][A
47%|████▋ | 83/177 [00:29<00:38, 2.43it/s][A
47%|████▋ | 84/177 [00:29<00:35, 2.59it/s][A
49%|████▊ | 86/177 [00:29<00:22, 4.07it/s][A
50%|████▉ | 88/177 [00:29<00:16, 5.38it/s][A
50%|█████ | 89/177 [00:29<00:14, 5.95it/s][A
51%|█████▏ | 91/177 [00:29<00:10, 7.85it/s][A
53%|█████▎ | 93/177 [00:30<00:13, 6.25it/s][A
53%|█████▎ | 94/177 [00:30<00:17, 4.69it/s][A
54%|█████▎ | 95/177 [00:31<00:23, 3.46it/s][A
54%|█████▍ | 96/177 [00:31<00:25, 3.15it/s][A
55%|█████▍ | 97/177 [00:32<00:28, 2.83it/s][A
55%|█████▌ | 98/177 [00:32<00:30, 2.56it/s][A
56%|█████▌ | 99/177 [00:33<00:35, 2.20it/s][A
56%|█████▋ | 100/177 [00:33<00:36, 2.09it/s][A
57%|█████▋ | 101/177 [00:34<00:37, 2.02it/s][A
58%|█████▊ | 102/177 [00:34<00:35, 2.11it/s][A
58%|█████▊ | 103/177 [00:35<00:31, 2.33it/s][A
59%|█████▉ | 104/177 [00:35<00:26, 2.72it/s][A
59%|█████▉ | 105/177 [00:35<00:27, 2.64it/s][A
60%|█████▉ | 106/177 [00:36<00:26, 2.69it/s][A
60%|██████ | 107/177 [00:36<00:24, 2.85it/s][A
62%|██████▏ | 109/177 [00:36<00:15, 4.28it/s][A
63%|██████▎ | 111/177 [00:37<00:14, 4.50it/s][A
63%|██████▎ | 112/177 [00:37<00:15, 4.10it/s][A
64%|██████▍ | 113/177 [00:37<00:18, 3.43it/s][A
64%|██████▍ | 114/177 [00:38<00:19, 3.27it/s][A
65%|██████▍ | 115/177 [00:38<00:18, 3.32it/s][A
66%|██████▌ | 116/177 [00:38<00:17, 3.54it/s][A
66%|██████▌ | 117/177 [00:38<00:14, 4.22it/s][A
67%|██████▋ | 118/177 [00:39<00:14, 3.97it/s][A
67%|██████▋ | 119/177 [00:39<00:19, 2.93it/s][A
68%|██████▊ | 120/177 [00:39<00:18, 3.07it/s][A
68%|██████▊ | 121/177 [00:40<00:14, 3.81it/s][A
69%|██████▉ | 123/177 [00:40<00:11, 4.75it/s][A
70%|███████ | 124/177 [00:40<00:12, 4.10it/s][A
71%|███████ | 125/177 [00:41<00:17, 3.00it/s][A
71%|███████ | 126/177 [00:41<00:17, 2.99it/s][A
72%|███████▏ | 127/177 [00:42<00:17, 2.78it/s][A
72%|███████▏ | 128/177 [00:42<00:16, 2.88it/s][A
73%|███████▎ | 129/177 [00:42<00:16, 2.87it/s][A
73%|███████▎ | 130/177 [00:43<00:18, 2.60it/s][A
74%|███████▍ | 131/177 [00:43<00:18, 2.42it/s][A
75%|███████▍ | 132/177 [00:43<00:15, 2.86it/s][A
75%|███████▌ | 133/177 [00:44<00:14, 2.94it/s][A
76%|███████▌ | 134/177 [00:44<00:16, 2.63it/s][A
76%|███████▋ | 135/177 [00:45<00:15, 2.75it/s][A
77%|███████▋ | 137/177 [00:45<00:09, 4.33it/s][A
79%|███████▊ | 139/177 [00:45<00:06, 5.72it/s][A
79%|███████▉ | 140/177 [00:45<00:07, 5.26it/s][A
80%|███████▉ | 141/177 [00:46<00:09, 3.72it/s][A
80%|████████ | 142/177 [00:46<00:11, 3.17it/s][A
81%|████████ | 143/177 [00:47<00:12, 2.66it/s][A
81%|████████▏ | 144/177 [00:47<00:13, 2.51it/s][A
82%|████████▏ | 145/177 [00:48<00:14, 2.20it/s][A
82%|████████▏ | 146/177 [00:48<00:14, 2.15it/s][A
83%|████████▎ | 147/177 [00:49<00:15, 2.00it/s][A
84%|████████▎ | 148/177 [00:49<00:14, 1.93it/s][A
84%|████████▍ | 149/177 [00:50<00:13, 2.08it/s][A
85%|████████▌ | 151/177 [00:50<00:09, 2.66it/s][A
86%|████████▌ | 152/177 [00:51<00:09, 2.71it/s][A
86%|████████▋ | 153/177 [00:51<00:08, 2.93it/s][A
87%|████████▋ | 154/177 [00:51<00:06, 3.32it/s][A
88%|████████▊ | 155/177 [00:51<00:05, 3.98it/s][A
89%|████████▊ | 157/177 [00:51<00:03, 5.71it/s][A
89%|████████▉ | 158/177 [00:52<00:03, 4.89it/s][A
90%|████████▉ | 159/177 [00:52<00:04, 4.39it/s][A
91%|█████████ | 161/177 [00:52<00:02, 6.05it/s][A
92%|█████████▏| 163/177 [00:52<00:02, 6.50it/s][A
93%|█████████▎| 164/177 [00:53<00:02, 4.94it/s][A
93%|█████████▎| 165/177 [00:53<00:03, 3.47it/s][A
94%|█████████▍| 166/177 [00:54<00:03, 2.93it/s][A
94%|█████████▍| 167/177 [00:54<00:03, 2.92it/s][A
95%|█████████▍| 168/177 [00:55<00:03, 2.71it/s][A
95%|█████████▌| 169/177 [00:55<00:03, 2.62it/s][A
96%|█████████▌| 170/177 [00:55<00:02, 2.79it/s][A
97%|█████████▋| 171/177 [00:56<00:02, 2.85it/s][A
97%|█████████▋| 172/177 [00:56<00:01, 2.79it/s][A
98%|█████████▊| 173/177 [00:56<00:01, 2.74it/s][A
99%|█████████▉| 175/177 [00:57<00:00, 4.20it/s][A
99%|█████████▉| 176/177 [00:57<00:00, 3.89it/s][A
100%|██████████| 177/177 [00:57<00:00, 3.65it/s][A
[A
29%|██▉ | 250/862 [1:28:29<2:44:38, 16.14s/it]
100%|██████████| 177/177 [00:57<00:00, 3.65it/s][A
[A
29%|██▉ | 251/862 [1:28:50<5:56:21, 34.99s/it]
29%|██▉ | 252/862 [1:29:10<5:11:30, 30.64s/it]
29%|██▉ | 253/862 [1:29:25<4:23:29, 25.96s/it]
29%|██▉ | 254/862 [1:29:49<4:15:26, 25.21s/it]
30%|██▉ | 255/862 [1:29:58<3:26:01, 20.36s/it]
30%|██▉ | 255/862 [1:29:58<3:26:01, 20.36s/it]
30%|██▉ | 256/862 [1:30:10<2:59:58, 17.82s/it]
30%|██▉ | 257/862 [1:30:29<3:04:39, 18.31s/it]
30%|██▉ | 258/862 [1:30:48<3:05:17, 18.41s/it]
30%|███ | 259/862 [1:31:05<3:02:10, 18.13s/it]
30%|███ | 260/862 [1:31:29<3:19:22, 19.87s/it]
30%|███ | 260/862 [1:31:29<3:19:22, 19.87s/it]
30%|███ | 261/862 [1:31:50<3:20:35, 20.03s/it]
30%|███ | 262/862 [1:32:05<3:06:01, 18.60s/it]
31%|███ | 263/862 [1:32:16<2:42:33, 16.28s/it]
31%|███ | 264/862 [1:32:33<2:43:34, 16.41s/it]
31%|███ | 265/862 [1:32:49<2:44:40, 16.55s/it]
31%|███ | 265/862 [1:32:49<2:44:40, 16.55s/it]
31%|███ | 266/862 [1:33:00<2:27:20, 14.83s/it]
31%|███ | 267/862 [1:33:09<2:09:37, 13.07s/it]
31%|███ | 268/862 [1:33:32<2:39:20, 16.09s/it]
31%|███ | 269/862 [1:33:48<2:39:02, 16.09s/it]
31%|███▏ | 270/862 [1:34:04<2:36:51, 15.90s/it]
31%|███▏ | 270/862 [1:34:04<2:36:51, 15.90s/it]
31%|███▏ | 271/862 [1:34:24<2:48:06, 17.07s/it]
32%|███▏ | 272/862 [1:34:40<2:45:05, 16.79s/it]
32%|███▏ | 273/862 [1:34:52<2:31:32, 15.44s/it]
32%|███▏ | 274/862 [1:35:08<2:31:14, 15.43s/it]
32%|███▏ | 275/862 [1:35:23<2:32:29, 15.59s/it]
32%|███▏ | 275/862 [1:35:23<2:32:29, 15.59s/it]
32%|███▏ | 276/862 [1:35:43<2:44:53, 16.88s/it]
32%|███▏ | 277/862 [1:36:03<2:51:15, 17.57s/it]
32%|███▏ | 278/862 [1:36:16<2:39:15, 16.36s/it]
32%|███▏ | 279/862 [1:36:35<2:46:26, 17.13s/it]
32%|███▏ | 280/862 [1:36:58<3:02:07, 18.78s/it]
32%|███▏ | 280/862 [1:36:58<3:02:07, 18.78s/it]
33%|███▎ | 281/862 [1:37:10<2:43:11, 16.85s/it]
33%|███▎ | 282/862 [1:37:24<2:35:45, 16.11s/it]
33%|███▎ | 283/862 [1:37:43<2:41:38, 16.75s/it]
33%|███▎ | 284/862 [1:38:01<2:46:17, 17.26s/it]
33%|███▎ | 285/862 [1:38:20<2:50:57, 17.78s/it]
33%|███▎ | 285/862 [1:38:20<2:50:57, 17.78s/it]
33%|███▎ | 286/862 [1:38:30<2:26:47, 15.29s/it]
33%|███▎ | 287/862 [1:38:41<2:15:07, 14.10s/it]
33%|███▎ | 288/862 [1:39:02<2:35:31, 16.26s/it]
34%|███▎ | 289/862 [1:39:20<2:40:13, 16.78s/it]
34%|███▎ | 290/862 [1:39:34<2:32:01, 15.95s/it]
34%|███▎ | 290/862 [1:39:34<2:32:01, 15.95s/it]
34%|███▍ | 291/862 [1:39:51<2:35:27, 16.33s/it]
34%|███▍ | 292/862 [1:40:03<2:21:13, 14.87s/it]
34%|███▍ | 293/862 [1:40:22<2:34:19, 16.27s/it]
34%|███▍ | 294/862 [1:40:32<2:16:24, 14.41s/it]
34%|███▍ | 295/862 [1:40:51<2:28:33, 15.72s/it]
34%|███▍ | 295/862 [1:40:51<2:28:33, 15.72s/it]
34%|███▍ | 296/862 [1:40:59<2:06:09, 13.37s/it]
34%|███▍ | 297/862 [1:41:17<2:18:22, 14.69s/it]
35%|███▍ | 298/862 [1:41:36<2:30:11, 15.98s/it]
35%|███▍ | 299/862 [1:41:53<2:34:21, 16.45s/it]
35%|███▍ | 300/862 [1:42:09<2:31:03, 16.13s/it]
35%|███▍ | 300/862 [1:42:09<2:31:03, 16.13s/it]{'eval_loss': 0.1410292237997055, 'eval_runtime': 57.9112, 'eval_samples_per_second': 6.113, 'eval_steps_per_second': 3.056, 'epoch': 0.29}
{'loss': 0.1774, 'grad_norm': 0.41748952865600586, 'learning_rate': 0.00016920138038595216, 'epoch': 0.3}
{'loss': 0.207, 'grad_norm': 0.4169933795928955, 'learning_rate': 0.0001678024805832284, 'epoch': 0.3}
{'loss': 0.1492, 'grad_norm': 0.3983752727508545, 'learning_rate': 0.0001663785793197323, 'epoch': 0.31}
{'loss': 0.1154, 'grad_norm': 0.3772595524787903, 'learning_rate': 0.00016493020164427152, 'epoch': 0.31}
{'loss': 0.177, 'grad_norm': 0.3978261947631836, 'learning_rate': 0.00016345788163107646, 'epoch': 0.32}
{'loss': 0.2136, 'grad_norm': 0.41389644145965576, 'learning_rate': 0.0001619621621828659, 'epoch': 0.33}
{'loss': 0.1915, 'grad_norm': 0.42459338903427124, 'learning_rate': 0.0001604435948306575, 'epoch': 0.33}
{'loss': 0.1747, 'grad_norm': 0.41835540533065796, 'learning_rate': 0.0001589027395303962, 'epoch': 0.34}
{'loss': 0.1698, 'grad_norm': 0.5012316703796387, 'learning_rate': 0.0001573401644564764, 'epoch': 0.34}
{'loss': 0.1693, 'grad_norm': 0.43907007575035095, 'learning_rate': 0.00015575644579223362, 'epoch': 0.35}
0%| | 0/177 [00:00<?, ?it/s][A
1%| | 2/177 [00:00<00:49, 3.52it/s][A
2%|▏ | 3/177 [00:01<01:20, 2.17it/s][A
2%|▏ | 4/177 [00:01<01:08, 2.53it/s][A
3%|▎ | 5/177 [00:01<00:50, 3.41it/s][A
3%|▎ | 6/177 [00:01<00:50, 3.36it/s][A
4%|▍ | 7/177 [00:02<00:52, 3.24it/s][A
5%|▍ | 8/177 [00:02<00:52, 3.19it/s][A
5%|▌ | 9/177 [00:02<00:55, 3.03it/s][A
6%|▌ | 10/177 [00:03<00:59, 2.79it/s][A
6%|▌ | 11/177 [00:03<01:05, 2.53it/s][A
7%|▋ | 12/177 [00:04<00:57, 2.88it/s][A
7%|▋ | 13/177 [00:04<00:58, 2.78it/s][A
8%|▊ | 14/177 [00:04<00:56, 2.90it/s][A
8%|▊ | 15/177 [00:04<00:44, 3.67it/s][A
9%|▉ | 16/177 [00:05<00:43, 3.72it/s][A
10%|▉ | 17/177 [00:05<00:44, 3.61it/s][A
11%|█ | 19/177 [00:05<00:37, 4.22it/s][A
11%|█▏ | 20/177 [00:06<00:38, 4.04it/s][A
12%|█▏ | 21/177 [00:06<00:46, 3.38it/s][A
12%|█▏ | 22/177 [00:07<01:01, 2.53it/s][A
13%|█▎ | 23/177 [00:07<01:14, 2.06it/s][A
14%|█▎ | 24/177 [00:08<01:14, 2.07it/s][A
14%|█▍ | 25/177 [00:08<01:05, 2.33it/s][A
15%|█▍ | 26/177 [00:09<00:57, 2.62it/s][A
15%|█▌ | 27/177 [00:09<00:53, 2.80it/s][A
16%|█▌ | 28/177 [00:09<00:48, 3.06it/s][A
16%|█▋ | 29/177 [00:10<00:58, 2.53it/s][A
17%|█▋ | 30/177 [00:10<01:05, 2.24it/s][A
18%|█▊ | 31/177 [00:11<01:14, 1.96it/s][A
18%|█▊ | 32/177 [00:11<01:05, 2.21it/s][A
19%|█▊ | 33/177 [00:11<00:51, 2.81it/s][A
20%|█▉ | 35/177 [00:12<00:38, 3.66it/s][A
20%|██ | 36/177 [00:12<00:37, 3.79it/s][A
21%|██ | 37/177 [00:12<00:32, 4.31it/s][A
21%|██▏ | 38/177 [00:12<00:32, 4.21it/s][A
22%|██▏ | 39/177 [00:13<00:47, 2.91it/s][A
23%|██▎ | 40/177 [00:13<00:49, 2.77it/s][A
23%|██▎ | 41/177 [00:14<00:49, 2.72it/s][A
24%|██▎ | 42/177 [00:14<00:55, 2.43it/s][A
24%|██▍ | 43/177 [00:15<01:02, 2.13it/s][A
25%|██▍ | 44/177 [00:15<01:00, 2.19it/s][A
25%|██▌ | 45/177 [00:16<01:01, 2.14it/s][A
26%|██▌ | 46/177 [00:16<01:02, 2.11it/s][A
27%|██▋ | 47/177 [00:17<01:05, 1.98it/s][A
27%|██▋ | 48/177 [00:17<01:05, 1.98it/s][A
28%|██▊ | 49/177 [00:18<01:06, 1.94it/s][A
28%|██▊ | 50/177 [00:18<01:04, 1.98it/s][A
29%|██▉ | 51/177 [00:19<01:08, 1.83it/s][A
29%|██▉ | 52/177 [00:20<01:09, 1.79it/s][A
30%|██▉ | 53/177 [00:20<01:10, 1.76it/s][A
31%|███ | 54/177 [00:20<00:58, 2.10it/s][A
31%|███ | 55/177 [00:21<00:54, 2.25it/s][A
32%|███▏ | 56/177 [00:21<00:48, 2.47it/s][A
32%|███▏ | 57/177 [00:21<00:42, 2.85it/s][A
33%|███▎ | 58/177 [00:21<00:36, 3.29it/s][A
33%|███▎ | 59/177 [00:22<00:28, 4.10it/s][A
34%|███▍ | 61/177 [00:22<00:18, 6.22it/s][A
35%|███▌ | 62/177 [00:22<00:20, 5.55it/s][A
36%|███▌ | 63/177 [00:23<00:32, 3.54it/s][A
36%|███▌ | 64/177 [00:23<00:39, 2.83it/s][A
37%|███▋ | 65/177 [00:23<00:39, 2.86it/s][A
38%|███▊ | 67/177 [00:24<00:25, 4.32it/s][A
38%|███▊ | 68/177 [00:24<00:25, 4.30it/s][A
39%|███▉ | 69/177 [00:24<00:26, 4.15it/s][A
40%|████ | 71/177 [00:25<00:24, 4.35it/s][A
41%|████ | 72/177 [00:25<00:25, 4.13it/s][A
41%|████ | 73/177 [00:25<00:22, 4.64it/s][A
42%|████▏ | 74/177 [00:25<00:24, 4.28it/s][A
42%|████▏ | 75/177 [00:26<00:26, 3.91it/s][A
44%|████▎ | 77/177 [00:26<00:24, 4.02it/s][A
44%|████▍ | 78/177 [00:27<00:31, 3.15it/s][A
45%|████▍ | 79/177 [00:27<00:32, 3.05it/s][A
45%|████▌ | 80/177 [00:27<00:30, 3.18it/s][A
46%|████▌ | 81/177 [00:28<00:30, 3.19it/s][A
46%|████▋ | 82/177 [00:28<00:30, 3.12it/s][A
47%|████▋ | 83/177 [00:28<00:38, 2.44it/s][A
47%|████▋ | 84/177 [00:29<00:35, 2.59it/s][A
48%|████▊ | 85/177 [00:29<00:27, 3.31it/s][A
49%|████▉ | 87/177 [00:29<00:18, 4.91it/s][A
50%|█████ | 89/177 [00:29<00:14, 6.16it/s][A
51%|█████▏ | 91/177 [00:29<00:10, 7.96it/s][A
53%|█████▎ | 93/177 [00:30<00:13, 6.36it/s][A
53%|█████▎ | 94/177 [00:30<00:17, 4.79it/s][A
54%|█████▎ | 95/177 [00:31<00:23, 3.54it/s][A
54%|█████▍ | 96/177 [00:31<00:25, 3.21it/s][A
55%|█████▍ | 97/177 [00:32<00:27, 2.86it/s][A
55%|█████▌ | 98/177 [00:32<00:30, 2.58it/s][A
56%|█████▌ | 99/177 [00:33<00:35, 2.22it/s][A
56%|█████▋ | 100/177 [00:33<00:36, 2.10it/s][A
57%|█████▋ | 101/177 [00:34<00:37, 2.03it/s][A
58%|█████▊ | 102/177 [00:34<00:35, 2.12it/s][A
58%|█████▊ | 103/177 [00:35<00:31, 2.35it/s][A
59%|█████▉ | 104/177 [00:35<00:26, 2.74it/s][A
59%|█████▉ | 105/177 [00:35<00:27, 2.66it/s][A
60%|█████▉ | 106/177 [00:36<00:26, 2.70it/s][A
60%|██████ | 107/177 [00:36<00:23, 2.94it/s][A
62%|██████▏ | 109/177 [00:36<00:14, 4.59it/s][A
63%|██████▎ | 111/177 [00:36<00:14, 4.70it/s][A
63%|██████▎ | 112/177 [00:37<00:14, 4.36it/s][A
64%|██████▍ | 113/177 [00:37<00:17, 3.60it/s][A
64%|██████▍ | 114/177 [00:37<00:18, 3.39it/s][A
65%|██████▍ | 115/177 [00:38<00:18, 3.43it/s][A
66%|██████▌ | 116/177 [00:38<00:16, 3.66it/s][A
66%|██████▌ | 117/177 [00:38<00:13, 4.40it/s][A
67%|██████▋ | 118/177 [00:38<00:14, 4.10it/s][A
67%|██████▋ | 119/177 [00:39<00:19, 2.98it/s][A
68%|██████▊ | 120/177 [00:39<00:18, 3.10it/s][A
68%|██████▊ | 121/177 [00:39<00:14, 3.81it/s][A
69%|██████▉ | 123/177 [00:40<00:11, 4.74it/s][A
70%|███████ | 124/177 [00:40<00:12, 4.09it/s][A
71%|███████ | 125/177 [00:41<00:17, 3.00it/s][A
71%|███████ | 126/177 [00:41<00:17, 2.99it/s][A
72%|███████▏ | 127/177 [00:41<00:18, 2.78it/s][A
72%|███████▏ | 128/177 [00:42<00:17, 2.87it/s][A
73%|███████▎ | 129/177 [00:42<00:16, 2.86it/s][A
73%|███████▎ | 130/177 [00:42<00:18, 2.58it/s][A
74%|███████▍ | 131/177 [00:43<00:19, 2.42it/s][A
75%|███████▍ | 132/177 [00:43<00:15, 2.85it/s][A
75%|███████▌ | 133/177 [00:43<00:14, 2.94it/s][A
76%|███████▌ | 134/177 [00:44<00:16, 2.63it/s][A
76%|███████▋ | 135/177 [00:44<00:15, 2.73it/s][A
77%|███████▋ | 137/177 [00:44<00:09, 4.31it/s][A
79%|███████▊ | 139/177 [00:45<00:06, 5.69it/s][A
79%|███████▉ | 140/177 [00:45<00:07, 5.24it/s][A
80%|███████▉ | 141/177 [00:45<00:09, 3.73it/s][A
80%|████████ | 142/177 [00:46<00:11, 3.17it/s][A
81%|████████ | 143/177 [00:46<00:12, 2.66it/s][A
81%|████████▏ | 144/177 [00:47<00:13, 2.51it/s][A
82%|████████▏ | 145/177 [00:47<00:14, 2.20it/s][A
82%|████████▏ | 146/177 [00:48<00:14, 2.15it/s][A
83%|████████▎ | 147/177 [00:49<00:15, 1.98it/s][A
84%|████████▎ | 148/177 [00:49<00:15, 1.93it/s][A
84%|████████▍ | 149/177 [00:49<00:13, 2.02it/s][A
85%|████████▌ | 151/177 [00:50<00:09, 2.61it/s][A
86%|████████▌ | 152/177 [00:50<00:09, 2.65it/s][A
86%|████████▋ | 153/177 [00:51<00:08, 2.88it/s][A
87%|████████▋ | 154/177 [00:51<00:07, 3.25it/s][A
88%|████████▊ | 155/177 [00:51<00:05, 3.87it/s][A
89%|████████▊ | 157/177 [00:51<00:03, 5.42it/s][A
89%|████████▉ | 158/177 [00:51<00:03, 4.81it/s][A
90%|████████▉ | 159/177 [00:52<00:04, 4.33it/s][A
91%|█████████ | 161/177 [00:52<00:02, 5.97it/s][A
92%|█████████▏| 163/177 [00:52<00:02, 6.41it/s][A
93%|█████████▎| 164/177 [00:53<00:02, 4.89it/s][A
93%|█████████▎| 165/177 [00:53<00:03, 3.44it/s][A
94%|█████████▍| 166/177 [00:54<00:03, 2.92it/s][A
94%|█████████▍| 167/177 [00:54<00:03, 2.92it/s][A
95%|█████████▍| 168/177 [00:54<00:03, 2.71it/s][A
95%|█████████▌| 169/177 [00:55<00:03, 2.62it/s][A
96%|█████████▌| 170/177 [00:55<00:02, 2.78it/s][A
97%|█████████▋| 171/177 [00:55<00:02, 2.85it/s][A
97%|█████████▋| 172/177 [00:56<00:01, 2.78it/s][A
98%|█████████▊| 173/177 [00:56<00:01, 2.74it/s][A
99%|█████████▉| 175/177 [00:56<00:00, 4.21it/s][A
99%|█████████▉| 176/177 [00:57<00:00, 3.90it/s][A
100%|██████████| 177/177 [00:57<00:00, 3.65it/s][A
[A
35%|███▍ | 300/862 [1:43:07<2:31:03, 16.13s/it]
100%|██████████| 177/177 [00:57<00:00, 3.65it/s][A
[Awandb: WARNING URL not available in offline run
`rope_scaling`'s beta_fast field must be a float, got 32
`rope_scaling`'s beta_slow field must be a float, got 1
35%|███▍ | 301/862 [1:43:27<5:24:50, 34.74s/it]
35%|███▌ | 302/862 [1:43:40<4:24:37, 28.35s/it]
35%|███▌ | 303/862 [1:43:58<3:53:09, 25.03s/it]
35%|███▌ | 304/862 [1:44:19<3:43:30, 24.03s/it]
35%|███▌ | 305/862 [1:44:35<3:20:08, 21.56s/it]
35%|███▌ | 305/862 [1:44:35<3:20:08, 21.56s/it]
35%|███▌ | 306/862 [1:45:12<4:01:19, 26.04s/it]
36%|███▌ | 307/862 [1:45:45<4:21:35, 28.28s/it]
36%|███▌ | 308/862 [1:46:11<4:14:09, 27.53s/it]
36%|███▌ | 309/862 [1:46:30<3:49:33, 24.91s/it]
36%|███▌ | 310/862 [1:46:43<3:16:37, 21.37s/it]
36%|███▌ | 310/862 [1:46:43<3:16:37, 21.37s/it]
36%|███▌ | 311/862 [1:47:05<3:17:32, 21.51s/it]
36%|███▌ | 312/862 [1:47:22<3:05:00, 20.18s/it]
36%|███▋ | 313/862 [1:47:46<3:17:01, 21.53s/it]
36%|███▋ | 314/862 [1:48:01<2:56:22, 19.31s/it]
37%|███▋ | 315/862 [1:48:23<3:04:38, 20.25s/it]
37%|███▋ | 315/862 [1:48:23<3:04:38, 20.25s/it]
37%|███▋ | 316/862 [1:48:58<3:44:05, 24.62s/it]
37%|███▋ | 317/862 [1:49:16<3:27:16, 22.82s/it]
37%|███▋ | 318/862 [1:49:39<3:27:24, 22.88s/it]
37%|███▋ | 319/862 [1:49:55<3:05:59, 20.55s/it]
37%|███▋ | 320/862 [1:50:17<3:11:31, 21.20s/it]
37%|███▋ | 320/862 [1:50:17<3:11:31, 21.20s/it]
37%|███▋ | 321/862 [1:50:42<3:19:39, 22.14s/it]
37%|███▋ | 322/862 [1:51:01<3:11:48, 21.31s/it]
37%|███▋ | 323/862 [1:51:27<3:24:06, 22.72s/it]Terminated
|