| license: mit | |
| tags: | |
| - from-scratch | |
| - transformer | |
| - tiny-llm | |
| # Tiny LLM (Character-Level) | |
| This is a minimal, from-scratch implementation of a GPT-style Transformer model, trained on a tiny text dataset. | |
| - **Architecture:** Transformer (Self-Attention, Multi-Head, FeedForward) | |
| - **Parameters:** ~100K | |
| - **Tokenizer:** Character-level | |
| - **Framework:** PyTorch | |
| It was built as a learning exercise to understand the core mechanics of LLMs. | |