Index all 10k optimizer-step checkpoints; retain final main weights
Browse files
README.md
CHANGED
|
@@ -94,3 +94,44 @@ Model release: [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0)
|
|
| 94 |
Attribution and MIT copyright/permission notices for adapted Parcae/ARLoop code
|
| 95 |
and the FLA routing reference are retained in `NOTICE`.
|
| 96 |
The tokenizer files are copied from the training run's SmolLM2 tokenizer assets.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 94 |
Attribution and MIT copyright/permission notices for adapted Parcae/ARLoop code
|
| 95 |
and the FLA routing reference are retained in `NOTICE`.
|
| 96 |
The tokenizer files are copied from the training run's SmolLM2 tokenizer assets.
|
| 97 |
+
|
| 98 |
+
## Training checkpoints
|
| 99 |
+
|
| 100 |
+
`main` contains the final checkpoint. Each revision below is independently loadable
|
| 101 |
+
with `revision="step-010000"` (use the same revision for the model and tokenizer).
|
| 102 |
+
|
| 103 |
+
Step labels use the **optimizer-step counter saved inside each checkpoint**.
|
| 104 |
+
Original filenames use microbatch steps; gradient accumulation changed during
|
| 105 |
+
training, so their numeric suffixes do not use a single conversion factor.
|
| 106 |
+
Training tokens below are calculated from the saved optimizer-step counter and
|
| 107 |
+
the constant global batch of 512 sequences × 4096 tokens (2,097,152 tokens/step).
|
| 108 |
+
|
| 109 |
+
| Optimizer step | Training tokens | Revision | Original checkpoint |
|
| 110 |
+
| ---: | ---: | --- | --- |
|
| 111 |
+
| 10,000 | 20,971,520,000 | [step-010000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-010000) | `step-00080000-arloop-1.3b-500b` |
|
| 112 |
+
| 20,000 | 41,943,040,000 | [step-020000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-020000) | `step-00160000-arloop-1.3b-500b-continue` |
|
| 113 |
+
| 30,000 | 62,914,560,000 | [step-030000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-030000) | `step-00240000-arloop-1.3b-500b-continue` |
|
| 114 |
+
| 40,000 | 83,886,080,000 | [step-040000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-040000) | `step-00320000-arloop-1.3b-500b-continue` |
|
| 115 |
+
| 50,000 | 104,857,600,000 | [step-050000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-050000) | `step-00400000-arloop-1.3b-500b-continue` |
|
| 116 |
+
| 60,000 | 125,829,120,000 | [step-060000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-060000) | `step-00480000-arloop-1.3b-500b-continue` |
|
| 117 |
+
| 70,000 | 146,800,640,000 | [step-070000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-070000) | `step-00560000-arloop-1.3b-500b-continue` |
|
| 118 |
+
| 80,000 | 167,772,160,000 | [step-080000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-080000) | `step-00640000-arloop-1.3b-500b-continue` |
|
| 119 |
+
| 90,000 | 188,743,680,000 | [step-090000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-090000) | `step-00720000-arloop-1.3b-500b-continue` |
|
| 120 |
+
| 100,000 | 209,715,200,000 | [step-100000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-100000) | `step-00800000-arloop-1.3b-500b-continue` |
|
| 121 |
+
| 110,000 | 230,686,720,000 | [step-110000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-110000) | `step-00880000-arloop-1.3b-500b-continue` |
|
| 122 |
+
| 120,000 | 251,658,240,000 | [step-120000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-120000) | `step-00960000-arloop-1.3b-500b-continue` |
|
| 123 |
+
| 130,000 | 272,629,760,000 | [step-130000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-130000) | `step-01040000-arloop-1.3b-500b-continue` |
|
| 124 |
+
| 140,000 | 293,601,280,000 | [step-140000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-140000) | `step-01120000-arloop-1.3b-500b-continue` |
|
| 125 |
+
| 150,000 | 314,572,800,000 | [step-150000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-150000) | `step-01200000-arloop-1.3b-500b-continue` |
|
| 126 |
+
| 160,000 | 335,544,320,000 | [step-160000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-160000) | `step-01280000-arloop-1.3b-500b-continue` |
|
| 127 |
+
| 170,000 | 356,515,840,000 | [step-170000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-170000) | `step-01360000-arloop-1.3b-500b-continue` |
|
| 128 |
+
| 180,000 | 377,487,360,000 | [step-180000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-180000) | `step-01440000-arloop-1.3b-500b-continue` |
|
| 129 |
+
| 190,000 | 398,458,880,000 | [step-190000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-190000) | `step-01520000-arloop-1.3b-500b-continue` |
|
| 130 |
+
| 200,000 | 419,430,400,000 | [step-200000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-200000) | `step-01600000-arloop-1.3b-500b-continue` |
|
| 131 |
+
| 210,000 | 440,401,920,000 | [step-210000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-210000) | `step-01680000-arloop-1.3b-500b-continue` |
|
| 132 |
+
| 220,000 | 461,373,440,000 | [step-220000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-220000) | `step-01760000-arloop-1.3b-500b-continue` |
|
| 133 |
+
| 230,000 | 482,344,960,000 | [step-230000](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-230000) | `step-01840000-arloop-1.3b-500b-continue` |
|
| 134 |
+
| 238,417 (final) | 499,996,688,384 | [main](https://huggingface.co/osunlp/ARLoop-1.3B/tree/main) / [step-238417](https://huggingface.co/osunlp/ARLoop-1.3B/tree/step-238417) | `step-03814672-arloop-1.3b-500b-continue` |
|
| 135 |
+
|
| 136 |
+
All 23 scheduled intermediate checkpoints from 10k through 230k are available.
|
| 137 |
+
For reproducible experiments, pin the selected revision's commit SHA.
|