Spaces:
Runtime error
Runtime error
Commit History
-
feat: vmap optimizer (#166)
Unverified signatureb993d27
-
feat: scan layers + gradient checkpointing (#161)
Unverified signature* scan layers for faster compilation * support gradient checkpointing07a6f9a
-
Merge branch 'main' of https://github.com/borisdayma/dalle-mini into main
bcd360f -
feat: better multi-node support (#158)
Unverified signature* reproducible data loader * custom sharding * model parallel across multiple nodes728a3c3
-
feat(text): support emojis (#154)
Unverified signature7ef7bd9 -
fix: smelu
7f2f8ed -
fix: sinkformer
2c583b3
-
fix: support smelu
a2dcee4 -
feat: allow relative position (#156)
Unverified signature769d20a
-
feat: sinkhorn in lse mode (#155)
Unverified signaturefixes #15300d4661
-
fix: sinkformer gradient
eed4896
-
feat(model): allow bias (#152)
Unverified signature361a994
-
feat: add sinkformer + custom final ln + pre-ln (#151)
Unverified signaturef139b0b
-
feat: placeholders for more config
69bcbeb -
feat: force final ln in encoder
32f4ba5 -
feat: allow more configurations
5bd4c20 -
fix: DeepNet doesn't scale weights of embedding/output layers (#150)
Shuming Ma +1: Shuming MaUnverified signatureCo-authored-by: Shuming Ma <shumma@microsoft.com>503d6b4
-
feat: remove unecessary LN
02824a7
-
feat: add cogview
472c4cc -
fix(textnormalizer): consider utf8 on windows (#148)
illtellyoulaterUnverified signature3b8d8cb
-
feat: implement transformer variants (#144)
Unverified signature* added DeepNet * added Swin v2 * added NormFormer * added RMSNorm * added GLU variants542378c
-
feat(data): super conditioning (#141)
Unverified signature* feat(data): online filtering * feat(generate): super conditioning * feat: add processor7939874
-
feat: support pod (#139)
Unverified signature803ccbf
-
feat: handle gradient checkpointing
5173ec7 -
feat: load from bucket
1c4e839 -
feat: reduce artifact space + offset step
34cf91c
-
feat: restore weights on CPU
5f954fc
-
fix: position embedding for generate method
ebac379
-
fix: typo
68cc185
-
fix: load from checkpoint
44b7c3e
-
feat(modeling): simplify abstract_init
fa72aa7
-
feat(train) - handle multiple nodes (#130)
Unverified signature0952927 -
feat: handle model parallel
1bb3269
-
fix: style
386f839
-
style(tokenizer): remove unused variables
605df32 -
feat: use fast tokenizer
767d78a
-
feat(train): improve pjit speed
f254058 -
fix(train): consider correct batch size
b7c7458
-
feat(train): distributed_shampoo with pjit
cc34d07
-
style: unsused import
7a176b9
-
feat(model): clean way to load on cpu
12f323d -
feat(train): no batch dimension with pjit
df1fe19 -
feat(train): progress on pjit
49597a2
-
feat: use_artifact if run existing
a5ed112 -
Load from wandb artifact (#121)
Unverified signatureLoad model & tokenizer from artifacts. Fixes #116f69b21b -
Style (isort).
Pedro Cuencaf9d51f7
-
Tokenizer, config, model can be loaded from wandb.
Pedro CuencaA mixin class is used to share the functionality.7e48337
-
feat(data): support accumulation in non-streaming
88c8e06 -
feat: custom gradient accumulation
2d07559
-
Change import order again.
Pedro CuencaMy version of `isort` has a different opinion than the one in the github action.2b2be9b