File size: 1,151 Bytes
d3166c0 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | ---
license: mit
language:
- en
library_name: pytorch
tags:
- text-generation
- chatbot
---
# Mini
A small English chatbot trained from scratch. It is a decoder-only transformer with 6 layers, 4 attention heads, an embedding size of 192, and a context window of 512 tokens. The vocabulary is 507 words. It answers short questions it has seen in its training dialogues. It is not a general-purpose assistant.
## Load
`model.py`, `tokenizer.py`, and `pretrained.py` from this repo need to be on the Python path.
```python
from model import TinyGPT
from tokenizer import Tokenizer
model = TinyGPT.from_pretrained("StrongDev2024/mini", trust_remote_code=True)
tokenizer = Tokenizer.from_pretrained("StrongDev2024/mini")
ids = tokenizer.encode("<user> what is 2 + 2 <bot>")
import torch
out = model.generate(
torch.tensor([ids]),
max_new_tokens=40,
temperature=0.0,
stop_ids={tokenizer.token_to_id["<end>"], tokenizer.token_to_id["<user>"]},
)
print(tokenizer.decode(out[0, len(ids) :].tolist()))
```
The prompt is `<user> your question <bot>`. Generation stops at `<end>`.
## License
MIT
|