Update README.md
Browse files
README.md
CHANGED
|
@@ -36,6 +36,8 @@ Training
|
|
| 36 |
Hardware: GTX 1660 SUPER (6GB, Turing) — ~95k tok/s, ~5.5h/epoch; dataset stays on CPU memmap (measured: moving it to GPU = +0.08%).
|
| 37 |
Checkpoints every 1,000 steps: step, train/val loss, lr, sweep position, weights, plus 2 sampled stories — resumable.
|
| 38 |
|
|
|
|
|
|
|
| 39 |
As of the first checkpoint:
|
| 40 |
epoch 0 step 1000: loss 0.910 | 104,813 tok/s | lr 4.00e-04
|
| 41 |
val loss @ step 1000: 0.930
|
|
|
|
| 36 |
Hardware: GTX 1660 SUPER (6GB, Turing) — ~95k tok/s, ~5.5h/epoch; dataset stays on CPU memmap (measured: moving it to GPU = +0.08%).
|
| 37 |
Checkpoints every 1,000 steps: step, train/val loss, lr, sweep position, weights, plus 2 sampled stories — resumable.
|
| 38 |
|
| 39 |
+
used complete.py to sample completions of a choosen checkpoint.
|
| 40 |
+
|
| 41 |
As of the first checkpoint:
|
| 42 |
epoch 0 step 1000: loss 0.910 | 104,813 tok/s | lr 4.00e-04
|
| 43 |
val loss @ step 1000: 0.930
|