File size: 336 Bytes
0429f3c | 1 2 3 4 5 6 7 8 9 10 11 | tokens 640049232 shards 7 batches 9764 window 65536 grad_accum 8 usable_steps 1220 mix b5_65536_131072 target 0.5000 actual 0.5000 tokens 320054446 mix b4_32768_65536 target 0.2500 actual 0.2500 tokens 160001134 mix b3_16384_32768 target 0.1500 actual 0.1500 tokens 95998599 mix b1_4096_8192 target 0.1000 actual 0.1000 tokens 63995053 |