README.md

metadata

title: ShellElastic
emoji: 🐚
colorFrom: red
colorTo: yellow
sdk: static
pinned: true
short_description: 'Scalable elastic pretraining: checkpoints and data'
tags:
  - elastic-models
  - foundation-models
  - hyperparameter-transfer
  - maximal-update-parameterization
  - pretraining

The landing page of ShellElastic: Scalable Elastic Pretraining with Simultaneous Maximal Updates and Hyperparameter Transfer. It links the checkpoints and the data of the paper.