| tokens 640046276 | |
| shards 7 | |
| batches 9762 | |
| window 65536 | |
| grad_accum 8 | |
| usable_steps 1220 | |
| mix b5_65536_131072 target 0.5000 actual 0.5000 tokens 320054446 | |
| mix b4_32768_65536 target 0.2500 actual 0.2500 tokens 160008905 | |
| mix b3_16384_32768 target 0.1500 actual 0.1500 tokens 95989992 | |
| mix b1_4096_8192 target 0.1000 actual 0.1000 tokens 63992933 | |