Shakespeare GPT (10.75M)

A small decoder-only transformer trained character by character on Tiny Shakespeare.

Parameters 10.75M (6 layers, 6 heads, 384-dim, context 256)
Vocabulary 65 characters
Best validation loss 1.4642 (step 2000)
Training 5,000 steps, ~14 min on a single T4, best-validation checkpoint kept

Trained with a custom optimizer: โœจ details coming soon, surprise! โœจ

Use it

# pip install torch
# files in this repo: best_ckpt.pt, config.json, model.py, inference.py
from inference import Generator
g = Generator("best_ckpt.pt")
print(g.generate("ROMEO:", max_new_tokens=300, temperature=0.8))

Live demo + API: https://huggingface.co/spaces/hariomlohardev/shakespeare-gpt-demo

Limitations

It imitates the style of Shakespeare, not the meaning: expect plausible-looking nonsense. Trained on ~1 MB of text, so it overfits quickly after step ~2000. English only, characters outside its 65-symbol vocabulary are ignored.

Downloads last month
39
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Dataset used to train hariomlohardev/shakespeare-gpt