| ---
|
| license: mit
|
| language:
|
| - en
|
| library_name: pytorch
|
| tags:
|
| - text-generation
|
| - chatbot
|
| ---
|
|
|
| # Mini
|
|
|
| A small English chatbot trained from scratch. It is a decoder-only transformer with 6 layers, 4 attention heads, an embedding size of 192, and a context window of 512 tokens. The vocabulary is 507 words. It answers short questions it has seen in its training dialogues. It is not a general-purpose assistant.
|
|
|
| ## Load
|
|
|
| `model.py`, `tokenizer.py`, and `pretrained.py` from this repo need to be on the Python path.
|
|
|
| ```python
|
| from model import TinyGPT
|
| from tokenizer import Tokenizer
|
|
|
| model = TinyGPT.from_pretrained("StrongDev2024/mini", trust_remote_code=True)
|
| tokenizer = Tokenizer.from_pretrained("StrongDev2024/mini")
|
|
|
| ids = tokenizer.encode("<user> what is 2 + 2 <bot>")
|
| import torch
|
| out = model.generate(
|
| torch.tensor([ids]),
|
| max_new_tokens=40,
|
| temperature=0.0,
|
| stop_ids={tokenizer.token_to_id["<end>"], tokenizer.token_to_id["<user>"]},
|
| )
|
| print(tokenizer.decode(out[0, len(ids) :].tolist()))
|
| ```
|
|
|
| The prompt is `<user> your question <bot>`. Generation stops at `<end>`.
|
|
|
| ## License
|
|
|
| MIT
|
|
|
