---
license: mit
tags:
- from-scratch
- transformer
- tiny-llm
---

# Tiny LLM (Character-Level)

This is a minimal, from-scratch implementation of a GPT-style Transformer model, trained on a tiny text dataset.

- **Architecture:** Transformer (Self-Attention, Multi-Head, FeedForward)
- **Parameters:** ~100K
- **Tokenizer:** Character-level
- **Framework:** PyTorch

It was built as a learning exercise to understand the core mechanics of LLMs.