English
n-gram
olmo-core
language-modeling
IanMagnusson commited on
Commit
3640e86
·
verified ·
1 Parent(s): e95cd16

Card: resuming with the released code (--ignore-data-fingerprints; sparse optimizer state refused)

Browse files
Files changed (1) hide show
  1. README.md +4 -3
README.md CHANGED
@@ -71,9 +71,10 @@ torchrun --standalone --nproc-per-node=1 olmo-core/src/scripts/train/ladder/coun
71
  --count-k 8 --count-max-order 6 --count-num-orders 6
72
  ```
73
 
74
- Evaluation loads model weights only. Resuming training also restores the trainer state, which records
75
- a fingerprint of the original data paths; the released code rejects a resume when your data paths
76
- differ.
 
77
 
78
  ## Related
79
 
 
71
  --count-k 8 --count-max-order 6 --count-num-orders 6
72
  ```
73
 
74
+ Evaluation loads model weights only. Resuming training also restores the optimizer and trainer state.
75
+ The trainer state records a fingerprint of the data paths the model was trained on, and the code
76
+ refuses to resume on data whose fingerprint differs; when your tokenized data is the same corpus at
77
+ different paths, pass `--ignore-data-fingerprints`.
78
 
79
  ## Related
80