Maggio33 commited on
Commit
d020584
·
verified ·
1 Parent(s): 5e9303d

Card: Track links on the owner's account

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -27,11 +27,11 @@ the weights is pending the owners' decision. Everything up to `ckpt_6750.pt` is
27
  | Hardware | steps 5721–6750: 4 × NVIDIA RTX 4090 (24 GB); steps 6751–7150: 4 × NVIDIA L40S (48 GB), resumed from `ckpt_6750.pt` (sha256 `970518a1…`) with the same trainer, recipe, data order and global batch. The data order of the resume was verified against the original run (batch hash of zero-based iterations 6751 and 7149, i.e. native steps 6752 and 7150, identical). DDP; global batch 256 × 4096 tokens as in 1.00–1.02 (effective: 4 ranks × micro-batch 2 × gradient accumulation 32) |
28
  | Checkpoints | every 150 steps from 5850 and the final step 7150; all are constant-lr states |
29
  | Trainer | `train_gpt_ref_r24_1.py`: the 1.02 trainer, releasing the checkpoint tensors from GPU memory after resuming (needed on 24 GB cards); the model definition is unchanged |
30
- | Training curve | steps up to 6850: https://track.fabryka.ai/run/7ca4f84f-3b07-4525-8cde-f3a936dc3425 · resume 6750–7150: https://track.fabryka.ai/run/5e362bba-53e0-5e5e-91b1-938ca2e3795a |
31
- | Benchmarks 1.00–1.03 (lm-eval 0.4.13, 0-shot) | https://track.fabryka.ai/run/4c729e9c-2e5a-56f0-a164-d4b5c2e2c627 |
32
 
33
  Training continues from `ckpt_7150.pt` on the rest of the prepared corpus (track:
34
- https://track.fabryka.ai/run/5c53b0cc-c947-5cd5-806a-0ace4b360ded); those checkpoints belong to the next versions.
35
 
36
  ## Benchmark caveat
37
 
 
27
  | Hardware | steps 5721–6750: 4 × NVIDIA RTX 4090 (24 GB); steps 6751–7150: 4 × NVIDIA L40S (48 GB), resumed from `ckpt_6750.pt` (sha256 `970518a1…`) with the same trainer, recipe, data order and global batch. The data order of the resume was verified against the original run (batch hash of zero-based iterations 6751 and 7149, i.e. native steps 6752 and 7150, identical). DDP; global batch 256 × 4096 tokens as in 1.00–1.02 (effective: 4 ranks × micro-batch 2 × gradient accumulation 32) |
28
  | Checkpoints | every 150 steps from 5850 and the final step 7150; all are constant-lr states |
29
  | Trainer | `train_gpt_ref_r24_1.py`: the 1.02 trainer, releasing the checkpoint tensors from GPU memory after resuming (needed on 24 GB cards); the model definition is unchanged |
30
+ | Training curve | steps up to 7150 (6860–7150 from the L40S resume): https://track.fabryka.ai/run/7ca4f84f-3b07-4525-8cde-f3a936dc3425 |
31
+ | Benchmarks 1.00–1.03 (lm-eval 0.4.13, 0-shot) | https://track.fabryka.ai/run/1426c0a8-2f07-5a03-80fc-92f7e922aa4b |
32
 
33
  Training continues from `ckpt_7150.pt` on the rest of the prepared corpus (track:
34
+ https://track.fabryka.ai/run/f056bb5c-f1da-56d5-9776-cc33063f10a5); those checkpoints belong to the next versions.
35
 
36
  ## Benchmark caveat
37