Instructions to use jinaai/jina-bert-implementation with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use jinaai/jina-bert-implementation with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("jinaai/jina-bert-implementation", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
Use attention dropout during training (#10)
Verified signature- fix: use attention dropout with torch SDPA implementation (5bc298762e07eccbf5cd43fa33904d6dfdc1c601) - feat: set self.dropout_p in constructor (02ebe529e85a23d23c837d4e7a1041a8c955af77)f3ec4cf
-
Allow device auto map (#8)
- feat: no splt modules for device auto map (f4624e02311a18676ed10090705c09efc0f698c2)c41d17d
-
Truncate to 8k by default (#5)
- feat: truncate to 8k by default (43899f3b9f615f94ba6d882feb32198db7c288b0)43f3955 -
Set max length to 2B
619ca8d
-
Update README.md
a9db862 -
Update README.md
f838124
-
Allow pytorch<2 to use without passing attn_implementation flag (#4)
- Allow pytorch<2 to use without passing attn_implementation flag (969de83296b491de451557789e9770b9335612bb)b5794c5
-
chore: update from afe81ca705ca1a5bd6b7d90548fcac068850b2af
Team Finetuner344bcbc -
Remove triton flash implementation
5ee2c37 -
Delete flash_attn_triton.py
4fa2261 -
chore: update from 896c12d73073854c513200fb74a4887cf25b2b97
Team Finetuner96e9a75
-
chore: update from f36c08c8a58c21b5aaab523fa03fb4a24b475612
Team Finetuner3e3ced0
-
chore: update from 07ce15d58b77559fce77ea89e92d398f28663bd9
Team Finetuner0f4070e
-
feat: allow changing flash implementation
Jackmin80143b8513
-
allow math kernel
Jackmin801bc43a5e -
Flash attention! (#2)
- feat: flash attn with torch implementation (da0980eef341d791ed5587597e91006b94e94ee4)df1a7f6
-
Create README.md
eb9e889
-
Allow flash attn (#1)
- old flash attn (4f24e0fef7d21fdb772aa2981062937a55ab26ab) - set flash attn as option in config (6f3de154e580bce3f90396153114f28ddc5d672b) - use tri daos version with backprop (c3b584d86f66bedb3802fc3309fff5d1f04246ca)622abd4
-
rename to jina bert in configuration file
33026dc -
rename to jina bert
b4f2b16 -
add basic configuration and model file
e36c994 -
initial commit
de9348b