BeanovaLi commited on
Commit
d0cfa79
·
verified ·
1 Parent(s): e61f819

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -33
README.md CHANGED
@@ -1,33 +0,0 @@
1
- ---
2
- license: bsd-3-clause
3
- ---
4
- Mirror of the base ProGen2-small model (with slightly modified configuration and forward pass) by [Nijkamp, et al.](https://arxiv.org/abs/2206.13517).
5
-
6
- See also my github [repo](https://github.com/hugohrban/ProGen2-finetuning/tree/main) for an example of finetuning this model.
7
-
8
- Example usage:
9
-
10
- ```python
11
- from transformers import AutoModelForCausalLM
12
- from tokenizers import Tokenizer
13
- import torch
14
- import torch.nn.functional as F
15
-
16
- # load model and tokenizer
17
- model = AutoModelForCausalLM.from_pretrained("hugohrban/progen2-small", trust_remote_code=True)
18
- tokenizer = Tokenizer.from_pretrained("hugohrban/progen2-small")
19
- tokenizer.no_padding()
20
-
21
- # prepare input
22
- prompt = "1MEVVIVTGMSGAGK"
23
- input_ids = torch.tensor(tokenizer.encode(prompt).ids).to(model.device)
24
-
25
- # forward pass
26
- logits = model(input_ids).logits
27
-
28
- # print output probabilities
29
- next_token_logits = logits[-1, :]
30
- next_token_probs = F.softmax(next_token_logits, dim=-1)
31
- for i in range(tokenizer.get_vocab_size(with_added_tokens=False)):
32
- print(f"{tokenizer.id_to_token(i)}: {100 * next_token_probs[i].item():.2f} %")
33
- ```