ShakeGPT
**ShakeGPT** is a lightweight, decoder-only Transformer language model trained on the Tiny Shakespeare dataset. It is designed to capture the stylistic patterns, vocabulary, and structure of Shakespearean English at a character level.
## Model Description
* **Architecture:** Transformer Decoder
* **Parameters:** ~0.6M
* **Training Data:** Tiny Shakespeare (1.6MB of raw text)
* **Tokenization:** Character-level
* **Context Window:** 128 characters
## Technical Specifications
| Feature | Value |
| :--- | :--- |
| `n_embd` (Embedding Dimension) | 128 |
| `n_layer` (Transformer Blocks) | 3 |
| `n_head` (Attention Heads) | 4 |
| `block_size` (Context Length) | 128 |
| `dropout` | 0.1 |
---
## Inference Script
This script initializes the **ShakeGPT** architecture and loads your saved weights to generate new text.
```python
import torch
import torch.nn as nn
from torch.nn import functional as F
import os
# ==========================================
# HYPERPARAMETERS (Matched…
