Maggio33's picture
Add run records (args, metrics, logs, val, environment) for all 400M runs, fala-1 finals K22/R1-R3, resume test, code
c28a890 verified
Raw History Blame Contribute Delete
612 Bytes
--data-dir /home/ubuntu/kolektyw/dane_dowody/out_v1_fw2 --out-dir /home/ubuntu/kolektyw/dowody400m/wyniki/VR0 --run-id dowody_VR0 --vocab 32768 --dtype uint16 --block 4096 --n-layer 16 --n-embd 896 --n-head 14 --batch 128 --grad-accum 8 --norm rmsnorm --pos rope --rope-theta 500000 --ffn swiglu --ffn-mult 2.667 --qk-norm --attn-bias 0 --optimizer muon --muon-lr 0.02 --lr 6e-4 --min-lr 6e-05 --wd 0.1 --schedule wsd --decay-frac 0.2 --warmup 500 --act-ckpt --loss-chunk 1024 --sampler random --seed 1 --data-seed 20261022 --compile --log-every 20 --eval-every 4769 --eval-iters 1 --steps 4768 --ckpt-every 500