---
license: mit
language:
  - en
library_name: pytorch
pipeline_tag: text-generation
tags:
  - tiny
  - character-level
  - gpt
  - from-scratch
  - educational
---

# Tiny LLM

A very small GPT-style language model, trained from scratch in about 15 minutes
on a laptop CPU. It writes one character at a time and only knows simple
greetings: hello, how are you, what is your name, thanks, goodbye.

It is a learning project, not a useful assistant. Anything that is not a
greeting gets an unrelated greeting-style reply.

**Try it in your browser:** [Tiny LLM demo Space](https://huggingface.co/spaces/Neural-Labs/NerualLabs)

## Model details

| | |
| --- | --- |
| Type | Decoder-only transformer (GPT-style), character level |
| Parameters | 824,120 |
| Layers | 4 |
| Attention heads | 4 |
| Embedding width | 128 |
| Context length | 128 characters |
| Vocabulary | 56 characters (the distinct characters in the training text) |
| Language | English |
| License | MIT |

Each block uses pre-layer-norm, causal self-attention, and an MLP
(128 → 512 → 128) with GELU. Token and position embeddings are both learned.

## Files

| File | What it is |
| --- | --- |
| `model.pt` | Trained weights: a dict with `state_dict`, `chars` (the vocabulary) and `config` |
| `model.py` | The model code in PyTorch, about 60 lines |

## How to use

This is a custom PyTorch model, so it does not load with `transformers`.
Download `model.py` and `model.pt`, then:

```bash
pip install torch huggingface_hub
```

```python
import torch
from huggingface_hub import hf_hub_download

repo = "Neural-Labs/tiny-llm"
hf_hub_download(repo, "model.py", local_dir=".")
weights = hf_hub_download(repo, "model.pt")

from model import TinyGPT

ckpt = torch.load(weights, map_location="cpu", weights_only=True)
chars = ckpt["chars"]
stoi = {c: i for i, c in enumerate(chars)}
model = TinyGPT(len(chars), **ckpt["config"])
model.load_state_dict(ckpt["state_dict"])
model.eval()


def reply(message):
    # Characters the model never saw are dropped.
    clean = "".join(c for c in message.replace("\n", " ") if c in stoi).strip() or "hi"
    prompt = f"User: {clean[:80]}\nBot:"
    idx = torch.tensor([[stoi[c] for c in prompt]])
    out = ""
    for tok in model.generate(idx, max_new_tokens=100, temperature=0.5, top_k=5):
        if chars[tok] == "\n":  # end of the bot's line
            break
        out += chars[tok]
    return out.strip()


print(reply("Hello!"))
print(reply("How are you?"))
```

The prompt format must match the training data: `User: <message>\nBot:`.
Generation stops at the first newline.

## Training data

12,000 generated exchanges in the form

```
User: <message>
Bot: <reply>

```

built from a short hand-written list of greeting phrases and replies
(hello, how are you, what is your name, thanks, goodbye, and similar).
No external dataset was used.

## Training

| | |
| --- | --- |
| Steps | 4,000 |
| Batch size | 32 |
| Optimizer | AdamW |
| Learning rate | 1e-3 with cosine decay |
| Hardware | 4-core CPU |
| Time | 14.5 minutes |
| Final loss | train 0.165, validation 0.170 |

## Limitations

- It only knows greetings. Any other message gets a greeting-style reply that
  has nothing to do with the question.
- It works on characters, and drops any character that was not in its training
  text (for example most non-English letters and emoji).
- It has no memory of earlier messages: each reply sees only the latest message.
- It is far too small to know facts, follow instructions, or reason.