| tokens 640007879 | |
| shards 7 | |
| batches 4879 | |
| window 131072 | |
| grad_accum 4 | |
| usable_steps 1219 | |
| mix b6_gte_131072 target 0.1100 actual 0.0893 tokens 57133001 | |
| mix b5_65536_131072 target 0.3900 actual 0.3953 tokens 252966753 | |
| mix b4_32768_65536 target 0.2500 actual 0.2551 tokens 163293012 | |
| mix b3_16384_32768 target 0.1500 actual 0.1552 tokens 99311922 | |
| mix b1_4096_8192 target 0.1000 actual 0.1052 tokens 67303191 | |