metadata
license: mit
tags:
- from-scratch
- transformer
- tiny-llm
Tiny LLM (Character-Level)
This is a minimal, from-scratch implementation of a GPT-style Transformer model, trained on a tiny text dataset.
- Architecture: Transformer (Self-Attention, Multi-Head, FeedForward)
- Parameters: ~100K
- Tokenizer: Character-level
- Framework: PyTorch
It was built as a learning exercise to understand the core mechanics of LLMs.