README.md

metadata

license: mit
language:
  - en
library_name: pytorch
pipeline_tag: text-generation
tags:
  - tiny
  - character-level
  - gpt
  - from-scratch
  - educational

Tiny LLM

A very small GPT-style language model, trained from scratch in about 15 minutes on a laptop CPU. It writes one character at a time and only knows simple greetings: hello, how are you, what is your name, thanks, goodbye.

It is a learning project, not a useful assistant. Anything that is not a greeting gets an unrelated greeting-style reply.

Try it in your browser: Tiny LLM demo Space

Model details

Type	Decoder-only transformer (GPT-style), character level
Parameters	824,120
Layers	4
Attention heads	4
Embedding width	128
Context length	128 characters
Vocabulary	56 characters (the distinct characters in the training text)
Language	English
License	MIT

Each block uses pre-layer-norm, causal self-attention, and an MLP (128 → 512 → 128) with GELU. Token and position embeddings are both learned.

Files

File	What it is
model.pt	Trained weights: a dict with state_dict, chars (the vocabulary) and config
model.py	The model code in PyTorch, about 60 lines

How to use

This is a custom PyTorch model, so it does not load with transformers. Download model.py and model.pt, then:

pip install torch huggingface_hub

import torch
from huggingface_hub import hf_hub_download

repo = "Neural-Labs/tiny-llm"
hf_hub_download(repo, "model.py", local_dir=".")
weights = hf_hub_download(repo, "model.pt")

from model import TinyGPT

ckpt = torch.load(weights, map_location="cpu", weights_only=True)
chars = ckpt["chars"]
stoi = {c: i for i, c in enumerate(chars)}
model = TinyGPT(len(chars), **ckpt["config"])
model.load_state_dict(ckpt["state_dict"])
model.eval()


def reply(message):
    # Characters the model never saw are dropped.
    clean = "".join(c for c in message.replace("\n", " ") if c in stoi).strip() or "hi"
    prompt = f"User: {clean[:80]}\nBot:"
    idx = torch.tensor([[stoi[c] for c in prompt]])
    out = ""
    for tok in model.generate(idx, max_new_tokens=100, temperature=0.5, top_k=5):
        if chars[tok] == "\n":  # end of the bot's line
            break
        out += chars[tok]
    return out.strip()


print(reply("Hello!"))
print(reply("How are you?"))

The prompt format must match the training data: User: <message>\nBot:. Generation stops at the first newline.

Training data

12,000 generated exchanges in the form

User: <message>
Bot: <reply>

built from a short hand-written list of greeting phrases and replies (hello, how are you, what is your name, thanks, goodbye, and similar). No external dataset was used.

Training

Steps	4,000
Batch size	32
Optimizer	AdamW
Learning rate	1e-3 with cosine decay
Hardware	4-core CPU
Time	14.5 minutes
Final loss	train 0.165, validation 0.170

Limitations

• It only knows greetings. Any other message gets a greeting-style reply that has nothing to do with the question.
• It works on characters, and drops any character that was not in its training text (for example most non-English letters and emoji).
• It has no memory of earlier messages: each reply sees only the latest message.
• It is far too small to know facts, follow instructions, or reason.