Commit History
-
feat-add-configs (#18)
Verified signature- Add model configs and tokenizer files (6c468c3dd03167c546da93a17669eccfe4c59450) - fix:remove tokenizer files (ee3d7db165fdec3402553ce7292622aef69703a3)b78d159
-
fix: handle window_size passed as list
71aae6d
-
fix-glu-mlp (#17)
Verified signature- fix: glu for non-flash-attn (c768124c3822ff864a2fc7477dbd2a175754fc5b)8771224
-
feat: for converting v2, added lines to save model weights and print config
ad76444
-
Porting v2 models to flash attention (#15)
Verified signature- Added GLUMLP, changed config accordingly, added code to convert state_dict (0211324e8c38d72ef847d46db9a9f389c864a5de) - fixed GLU implementation, added conversion of layer norms (9587227ceebcbf4e7335c0938838e9a2eb0b5d6b) Co-authored-by: Markus Krimmel <Markus28@users.noreply.huggingface.co>e55e319
-
feat: updated activation checkpointing (#14)
Verified signature- wrap every layer in a checkpoint (e0da4c55e7a599407614621df650326c11cafd2f)4ed66c0 -
fix: use staticmethod istead of classmethod
6f8e8c5 -
feat: added comment
d9a681b -
feat: added docstrings
dae5c58 -
feat: Allow LoRA to be merged into weights (#12)
Verified signature- feat: added method to merge LoRA weights (a2b49b2c72b661d680e6fde184d5ef05a5ebc974)702e6c9
-
feat: added README
284316d -
removed num_tasks from config
6546b2c -
feat: removed tokenizer
559b3ae -
feat: removed task type embeddings
8e3d0b8
-
fix: remove cleaving (#13)
Verified signature- Revert "feat: cleave off layers from encoder (#11)" (99b812dc5c29ac777aa7b5164d3ec7399d520e06)139b4a5 -
fix: added trust_remote_code to tokenizer
9689b77 -
fix: fixed from_bert method
151f328 -
fix: fix LoRA implementation
20706dd -
feat: cleave off layers from encoder (#11)
Verified signature- feat: added functionality to cleave off layers from BERT encoder (86b04389ab236e732a06f363ffccaa4de1ca361d) - feat: added cleaved_layers property (4441ce60ec99a4666004458a91be03c8bca69729)b641603
-
feat: only apply select_task_for_layer if task has changed
462e28d -
feat: make num of loras part of the config
a416a9d -
feat: make main parameters trainable
cdf5490
-
fix BertForMaskedLM
c0b46cc -
feat: added separate BertForMaskedLM class
3cb3930
-
Update tokenizer.py
Verified signature9072f7f -
feat-add-constant-for-task-type-ids (#10)
Verified signature- feat: add enum for task type ids (db57d383793c47c7e6f6487d68c20311be3bf20d)11b09c9
-
feat: added return_dict
59c0808 -
fix: fixed syntax error in LoRA
e93b0fd -
feat: add current_task to forward
9410275
-
feat: use property in LoRA parametrization
0ff7c3d -
feat: added LoRA copyright notice
faa9951 -
feat: use property instead of setter
6aad619 -
clean up embeddings.py (#7)
Verified signature- clean up embeddings.py (6d5377580042f614ac525febd19c641c4d456d9b) Co-authored-by: Bo Wang <bwang0911@users.noreply.huggingface.co>7771ce3
-
feat: return from_bert for from_pretrained
5549314 -
support-fast-tokenizer (#6)
Verified signature- feat: support fast tokenizer (9b5c14890f1788f110229da96fbe6f60ac25cd69) - refactor: batch encoding (72221a70b0f9c93095e18d4b2321d00c2439c038)ed1b276
-
feat: made from_bert work
851184a -
feat: choose flash attention heuristically if not set explicitly
2e2b8d0 -
feat: select first LoRA upon initialization
fabeb13 -
feat: formatting and type hints
617fe56 -
support-multiple-task-ids (#5)
Verified signature- support multiple task ids (6170b43f054d6e22f48dec300374eeaec5b70a75) - add assertions and docs (4b6651909fb268865ca95db85a7ca214631cd5bf) Co-authored-by: Michael Günther <michael-guenther@users.noreply.huggingface.co>e151a8f -
fix: use proper initilization for embedding layer
850b9a2 -
fix: fixed typo
5c4e4bf -
feat: added LoRA
8561a1f
-
feat: assert return_dict
326b1c4 -
fix: same assertions in other models
c1d92c9 -
fix: assert is None for other kwargs too
3f5615c -
feat: added head_mask
599c64e -
added classifier dropout
767b681 -
fix: formatting
ae4c28c -
fix: formatting
f115a1d