Standard Vanilla Transformer Model
Pure standard autoregressive language model trained from scratch without architecture efficiency tricks.
Model Configuration
- Layers: 16
- Embedding Dim: 512
- Attention Heads: 4
- Context Length: 4096
- Max Tokens per Sample: 1024