| license: mit | |
| library_name: transformers | |
| # MidTrainingCheckpoint | |
| <!-- markdownlint-disable first-line-h1 --> | |
| <!-- markdownlint-disable html --> | |
| <!-- markdownlint-disable no-duplicate-header --> | |
| <div align="center"> | |
| <img src="figures/fig1.png" width="60%" alt="MidTrainingCheckpoint" /> | |
| </div> | |
| <hr> | |
| ## 1. Introduction | |
| MidTrainingCheckpoint is a snapshot taken at the midpoint of our training run. It captures the model state at step 500, providing a useful reference for studying training dynamics. | |
| <p align="center"> | |
| <img width="80%" src="figures/fig3.png"> | |
| </p> | |
| This checkpoint is particularly useful for: | |
| - Comparing with earlier and later checkpoints | |
| - Understanding the training trajectory | |
| - Performing intermediate model analysis | |
| ## 2. Model Details | |
| | Property | Value | | |
| |---|---| | |
| | Architecture | BERT | | |
| | Training Steps | 500 | | |
| | Checkpoint Name | step_500 | | |
| | Purpose | Mid-training reference | | |
| ## 3. Usage | |
| ```python | |
| from transformers import AutoModel | |
| model = AutoModel.from_pretrained("MidTraining-Checkpoint") | |
| ``` | |
| ## 4. License | |
| [MIT License](LICENSE) | |
| ## 5. Contact | |
| Open an issue on GitHub. | |