Card: resuming with the released code (--ignore-data-fingerprints; sparse optimizer state refused)
Browse files
README.md
CHANGED
|
@@ -71,9 +71,10 @@ torchrun --standalone --nproc-per-node=1 olmo-core/src/scripts/train/ladder/coun
|
|
| 71 |
--count-k 8 --count-max-order 6 --count-num-orders 6
|
| 72 |
```
|
| 73 |
|
| 74 |
-
Evaluation loads model weights only. Resuming training also restores the
|
| 75 |
-
a fingerprint of the
|
| 76 |
-
|
|
|
|
| 77 |
|
| 78 |
## Related
|
| 79 |
|
|
|
|
| 71 |
--count-k 8 --count-max-order 6 --count-num-orders 6
|
| 72 |
```
|
| 73 |
|
| 74 |
+
Evaluation loads model weights only. Resuming training also restores the optimizer and trainer state.
|
| 75 |
+
The trainer state records a fingerprint of the data paths the model was trained on, and the code
|
| 76 |
+
refuses to resume on data whose fingerprint differs; when your tokenized data is the same corpus at
|
| 77 |
+
different paths, pass `--ignore-data-fingerprints`.
|
| 78 |
|
| 79 |
## Related
|
| 80 |
|