File size: 1,295 Bytes
9664d44 dc12ed4 9664d44 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | # Shakespeare GPT Transformer
A character-level GPT model trained on Shakespeare plays.
## Model Architecture
- Character-level tokenization
- Multi-head self-attention with causal masking
- 6 transformer blocks
- 6 attention heads
- Embedding dimension: 384
- Block size (context length): 512
- Dropout: 0.2
## Files
- `model.pt` - Model state dictionary
- `config.json` - Model hyperparameters
- `vocab.json` - Character to index and index to character mappings
- `generated_output.txt` - Sample generated text
## Usage
```python
import torch
import json
# Load config
with open("config.json") as f:
config = json.load(f)
# Load vocab
with open("vocab.json") as f:
vocab = json.load(f)
stoi = vocab["stoi"]
itos = {int(k): v for k, v in vocab["itos"].items()}
decoder = lambda s: "".join([itos[c] for c in s])
encoder = lambda s: [stoi[c] for c in s]
# Rebuild model (use same class definitions)
model = gpt(config["vocab_size"])
model.load_state_dict(torch.load("model.pt"))
model.eval()
# Generate text
context = torch.zeros((1, 1), dtype=torch.long)
print(decoder(model.generate(context, max_new_token=500)[0].tolist()))
```
|