agu18dec's picture
Add RL variants (co-trained AR, advantage normalisation), bullet readouts, gradient target, bare-model log reward
a1987eb verified
Raw History Blame Contribute Delete
725 Bytes
{
"name": "olens_ablations/gradient_target/ar",
"ckpt": "abl.ar.grad.k1.decay6945/milestone_step_8334",
"run": "abl.ar.grad.k1.decay6945",
"n_tensors": 994,
"n_params": 142947328,
"overrides": "kind=ar name=abl.ar.grad.k1 ar.space=raw training.lr=0.0001 data.train_dir=/data/abl/pairs_grad data.val_dir=/data/abl/rulers_grad grad_ckpt=False compile=True training.num_workers=16 training.val_batch_size=64 data.basis_path=/data/basis.safetensors training.train_batch_size=72 training.gradient_accumulation_steps=2 effective_batch=576 training.max_steps=6945 data.target_col=target_grad name=abl.ar.grad.k1.decay6945 wandb_id=abl.ar.grad.k1.decay6945 branch_from=/runs/abl.ar.grad.k1/milestone_step_6945 decay_frac=0.2"
}