Commit History
-
Add built binary [skip-build]
github-actions[bot]6ec5093
-
Add built binary [skip-build]
github-actions[bot]de5bead
-
Add built binary [skip-build]
github-actions[bot]aee4dc0
-
Add built binary [ci skip]
github-actions[bot]e93bd1e -
Add built binary [ci skip]
github-actions[bot]678578a
-
Add built binary [ci skip]
github-actions[bot]99af74f -
Add built binary [ci skip]
github-actions[bot]f88998f
-
feat(muon_clip) : add muon clip (#6)
Unverified signature* feat(muon_clip) : add muon clip * fix(muon_clip): delete comment * fix(muon_clip): delete comment * fix(muon_clip): considering when nkvheadgroup>1 * docs(muon_clip): refine __init__ docstring and add clip_info argument description * refactor(muon_clip): refactor clip info using dataclass * fix(muon_clip): change min -> new_scaling compare * test(muon): add qk_clip=False case to model comparison * test(muon): show results * fix(muon_clip): change default is muon func * Add built binary [ci skip] --------- Co-authored-by: dongseokmotif <dongseokkim@motiftech.io> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>d65066c -
chore: add nix build workflow (#8)
Unverified signature* chore: add nix build workflow * Add built binary [ci skip] --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>8997e30
-
feat(muon) : add tuned-abc-values & blfoat16 communication
f7faa93
-
feat: update muon to receive paramgroups, not model (#4)
Unverified signature* feat: update muon to receive paramgroups, not model * feat: update message formats * feat: remove boilerplate assertion * chore: run pre-commits * test: fix testscript * fix: fix misc bugs * feat: add get_default_muon_param_groups helper function * fix: fix readme * fix: raise error if parametergroup does not follow instructions * chore: upload binary --------- Co-authored-by: junhyeok.lee <junhyeok.lee@moreh.io> Co-authored-by: WyldeCat <skan1543@gmail.com>b0f46c7
-
fix(muon): add update_p stage and dealloc tensors properly
99e7c0c
-
Support HSDP (#4)
Verified signature- feat(muon): support HSDP (dd04b0152725e7b55970171706d5d16f59352cda)8447fd1
-
support torch 2.8 (#3)
Verified signature- feat: upload torch28 build (e60259752446c4918b7219341e5cdd2d68defa86)1e2b528
-
fix(muon): handle un-distributed env
1f13dae -
refactor(muon): change argument adam_wd to weight_decay and handle params' type
02ac540
-
fix(muon): free tensors that are no longer needed
64757cb -
chore(muon): update comment
036642a -
chore(muon): clean build and update doc
febdf5b
-
fix(muon): delete intermediate tensors immediately to lower peak mem usage
bdd2678 -
chore: initial commit
8535e80