Commit History

  1. feat: vmap optimizer (#166)

    Unverified signature
    b993d27

  1. feat: scan layers + gradient checkpointing (#161)⁠

    Unverified signature
    07a6f9a

  1. Merge branch 'main' of https://github.com/borisdayma/dalle-mini into main

    bcd360f

  1. feat(text): support emojis (#154)

    Unverified signature
    7ef7bd9
  2. fix: smelu

    7f2f8ed
  3. fix: sinkformer

    2c583b3

  1. fix: support smelu

    a2dcee4
  2. feat: allow relative position (#156)

    Unverified signature
    769d20a

  1. feat: sinkhorn in lse mode (#155)⁠

    Unverified signature
    00d4661

  1. fix: sinkformer gradient

    eed4896

  1. feat(model): allow bias (#152)

    Unverified signature
    361a994

  1. feat: add sinkformer + custom final ln + pre-ln (#151)

    Unverified signature
    f139b0b

  1. feat: placeholders for more config

    69bcbeb
  2. feat: force final ln in encoder

    32f4ba5
  3. feat: allow more configurations

    5bd4c20
  4. fix: DeepNet doesn't scale weights of embedding/output layers (#150)⁠

    Shuming Ma +1: Shuming Ma
    Unverified signature
    503d6b4

  1. feat: remove unecessary LN

    02824a7

  1. feat: add cogview

    472c4cc
  2. fix(textnormalizer): consider utf8 on windows (#148)

    illtellyoulater
    Unverified signature
    3b8d8cb

  1. feat: implement transformer variants (#144)⁠

    Unverified signature
    542378c

  1. feat(data): super conditioning (#141)⁠

    Unverified signature
    7939874

  1. feat: support pod (#139)

    Unverified signature
    803ccbf

  1. feat: handle gradient checkpointing

    5173ec7
  2. feat: load from bucket

    1c4e839
  3. feat: reduce artifact space + offset step

    34cf91c

  1. feat: restore weights on CPU

    5f954fc

  1. fix: position embedding for generate method

    ebac379

  1. fix: typo

    68cc185

  1. fix: load from checkpoint

    44b7c3e

  1. feat(modeling): simplify abstract_init

    fa72aa7

  1. feat: handle model parallel

    1bb3269

  1. style(tokenizer): remove unused variables

    605df32
  2. feat: use fast tokenizer

    767d78a

  1. feat(train): distributed_shampoo with pjit

    cc34d07

  1. feat(model): clean way to load on cpu

    12f323d
  2. feat(train): no batch dimension with pjit

    df1fe19

  1. feat: use_artifact if run existing

    a5ed112
  2. Style (isort).

    Pedro Cuenca
    f9d51f7

  1. Tokenizer, config, model can be loaded from wandb.⁠

    Pedro Cuenca
    7e48337

  1. Change import order again.⁠

    Pedro Cuenca
    2b2be9b
  2. Fix import order to make isort happy.

    Pedro Cuenca
    64d99b2
  3. Accept changes suggested by linter.

    Pedro Cuenca
    9f522b8
  4. Never consider local dirs as remote wandb references.⁠

    Pedro Cuenca
    08dd098
  5. Store resolved path after loading model.⁠

    Pedro Cuenca
    55a631d
  6. Override from_pretrained to support wandb artifacts.

    Pedro Cuenca
    1023afa

  1. feat: support pypi

    f5dba1e