Ctrl+K
- claim-1-flashoptim-combines-improved-master-weight-splitting-with-companded-8-bit-optimizer-state-quantization-to-reduce-parameter-associated-optimizer-memory-section-3
- claim-2-flashoptim-reduces-adamw-training-memory-from-16-to-7-bytes-per-parameter-or-5-bytes-with-gradient-release-table-1
- claim-3-for-llama-3-1-8b-finetuning-flashoptim-reduces-peak-memory-from-175-gib-to-113-gib-by-compressing-parameters-and-optimizer-states-figure-1-table-4
- claim-4-flashoptim-variants-match-reference-optimizer-training-loss-trajectories-in-gpt-2-pretraining-and-resnet-50-image-classification-figure-2
- claim-5-flashoptim-matches-reference-scores-on-resnet-50-validation-accuracy-llama-3-1-8b-gsm8k-finetuning-and-gpt-2-in-context-learning-benchmarks-table-2-table-3
- claim-6-ulp-based-weight-splitting-lowers-fp32-reconstruction-error-and-companding-prevents-quantized-adamw-training-divergence-compared-with-linear-optimizer-state-quantization-figure-3-figure-5
- conclusion
- executive-summary
- 2.22 kB