|
Download LOADING.md from zimplex/helm-c4-2b-lowp-checkpoints: direct link, hf CLI and curl.
- Browser
- Download file 2.27 kB
-
https://huggingface.co/zimplex/helm-c4-2b-lowp-checkpoints/resolve/main/LOADING.md
- Command line
-
hf download hf://zimplex/helm-c4-2b-lowp-checkpoints/LOADING.md
-
curl -L -o LOADING.md https://huggingface.co/zimplex/helm-c4-2b-lowp-checkpoints/resolve/main/LOADING.md
2.27 kB
| # Loading the native model-only checkpoints | |
| Use the pinned HELM/model configuration and evaluator supplied with PR2 | |
| `5edb20302017712ac97bf39374f66db80f94bc51` (HELM `7501deca8f413848bfef804be64ce874b72a3cd7`). Checkpoint state is FP32 and has exactly two keys: | |
| `key` and `state_dict`. The training environment used Python 3.12.3, | |
| PyTorch 2.11.0+cu128 and CUDA 12.8. | |
| ```python | |
| import torch | |
| payload = torch.load(checkpoint_path, map_location="cpu", weights_only=True) | |
| assert set(payload) == {"key", "state_dict"} | |
| ``` | |
| These are custom HELM native archives. Load them through the supplied HELM | |
| evaluator; they are not transformers save_pretrained/AutoModel repositories. | |
| Construct the matching width-910, 16-layer HELM with curvature1 and the | |
| configuration used by the supplied evaluator, then load state_dict strictly. | |
| Training used world8; preserve the corresponding model configuration when | |
| adapting the evaluator. Evaluate with the native BF16 autocast context while | |
| retaining FP32 parameter storage. | |
| Follow the original init-dependent construction/radius recipe before loading | |
| the learned state. Scaled runs recorded R_init=u_init=1; published native BF16 | |
| seeds7/99 recorded R_init=u_init=30.1496. Each profiles/*.json and unchanged | |
| native result retains the measured normalizer radii. Do not reapply initializer | |
| scaling to learned weights after loading them. | |
| For Scaled4, install the supplied PR2 hooks once in a fresh evaluation process: | |
| ```python | |
| from helm.hypercore.manifolds import Lorentz | |
| import lowp_emul | |
| lowp_emul.install(Lorentz, fmt="e2m1", block=32, ops="both") | |
| ``` | |
| For Scaled8, use fmt="e4m3". Install the hooks before model construction. | |
| Native BF16 seeds7/99 use neither lowp hooks nor the FP32 geometry policy. | |
| The bit-width suffix identifies operand emulation, not quantized weight storage. | |
| Run each profile in a fresh process so different wrappers do not stack. | |
| CHECKPOINTS.json distinguishes the seed99 full-budget final checkpoint from | |
| the three exports of earlier finite saved states. The latter are NOT FINAL | |
| and must be reported with their actual saved step. Optimizer/RNG resumes are | |
| not included here. The native scalar final PPL is NaN for failed runs; the | |
| last finite evaluation in RESULTS.json is a separate earlier observation. | |