| # Shakespeare GPT Transformer | |
| A character-level GPT model trained on Shakespeare plays. | |
| ## Model Architecture | |
| - Character-level tokenization | |
| - Multi-head self-attention with causal masking | |
| - 6 transformer blocks | |
| - 6 attention heads | |
| - Embedding dimension: 384 | |
| - Block size (context length): 512 | |
| - Dropout: 0.2 | |
| ## Files | |
| - `model.pt` - Model state dictionary | |
| - `config.json` - Model hyperparameters | |
| - `vocab.json` - Character to index and index to character mappings | |
| - `generated_output.txt` - Sample generated text | |
| ## Usage | |
| ```python | |
| import torch | |
| import json | |
| # Load config | |
| with open("config.json") as f: | |
| config = json.load(f) | |
| # Load vocab | |
| with open("vocab.json") as f: | |
| vocab = json.load(f) | |
| stoi = vocab["stoi"] | |
| itos = {int(k): v for k, v in vocab["itos"].items()} | |
| decoder = lambda s: "".join([itos[c] for c in s]) | |
| encoder = lambda s: [stoi[c] for c in s] | |
| # Rebuild model (use same class definitions) | |
| model = gpt(config["vocab_size"]) | |
| model.load_state_dict(torch.load("model.pt")) | |
| model.eval() | |
| # Generate text | |
| context = torch.zeros((1, 1), dtype=torch.long) | |
| print(decoder(model.generate(context, max_new_token=500)[0].tolist())) | |
| ``` | |