CodyTonn/guppylm-9M

 

GuppyLM — 9M Parameter Fish Chat Model

A ~9M parameter LLM trained from scratch on an Apple Silicon MacBook (MPS) that talks like a small fish named Guppy.

Trained with the self-contained train_guppylm_mac.ipynb notebook from github.com/arman-bd/guppylm: it writes its own source files, downloads the data, trains, exports and packages the model — no cloud GPU needed, about 2 minutes on an M4 Pro.

Example (real output from this checkpoint)

You> hi guppy
Guppy> oh hi. i didn't see you there. my eyes are on the sides.

You> are you hungry
Guppy> yes. always yes. i am always ready for food. i promise to eat all of it.

You> tell me a joke
Guppy> fish don't tell jokes. we are the joke. small and confused.

You> what is the meaning of life
Guppy> i'm here. the water is here. that's all i need to know.

You> goodnight guppy
Guppy> nighty night. i was pressing my face against the glass but now i'll stop. goodnight tank.

Architecture

Parameters	8.7M
Type	Vanilla transformer (from scratch)
Layers	6
Hidden dim	384
Heads	6
FFN	768 (ReLU)
Vocab	4,096 (BPE)
Max sequence	128 tokens
Norm	LayerNorm
Position	Learned embeddings
LM head	Weight-tied with embeddings
Prompt format	ChatML (`<

No GQA, no RoPE, no SwiGLU. As simple as it gets.

Training

• Data: arman-bd/guppylm-60k-generic — 57K train / 3K eval single-turn synthetic conversations across 60 topics, formatted as ChatML
• Steps: 3,000 (notebook default), eval loss 0.407; the notebook's max_steps=10000 setting reaches the original quality
• Optimizer: AdamW (β1=0.9, β2=0.95), cosine LR schedule with 200-step warmup, weight decay 0.1
• Batch: 32 × 128 tokens
• Hardware: Apple M4 Pro (MPS), batch 32 at ~40 ms/step, ~2 minutes total
• No system prompt — personality is baked into the weights

Files

File	Description
model.bin	State dict (fp32, 35 MB)
config.json	Model config (HF-style keys)
tokenizer.json	BPE tokenizer (4,096 vocab)
model.onnx	Quantized uint8 ONNX export (10.5 MB)
model.py / config.py / dataset.py / train.py / inference.py	Full source code — this repo is self-contained

Usage

PyTorch (files in this repo, no extra install beyond torch and tokenizers):

pip install torch tokenizers
python inference.py --checkpoint model.bin --tokenizer tokenizer.json \
    --device cpu --prompt "hi guppy"

from inference import GuppyInference

engine = GuppyInference('model.bin', 'tokenizer.json')
r = engine.chat_completion([{'role': 'user', 'content': 'hi guppy'}])
print(r['choices'][0]['message']['content'])

ONNX (CPU inference with onnxruntime):

import onnxruntime as ort
from tokenizers import Tokenizer
import numpy as np

sess = ort.InferenceSession('model.onnx')
tok = Tokenizer.from_file('tokenizer.json')

prompt = '<|im_start|>user\nhi guppy<|im_end|>\n<|im_start|>assistant\n'
ids = tok.encode(prompt).ids
for _ in range(64):
    logits = sess.run(None, {'input_ids': np.array([ids], dtype=np.int64)})[0][0, -1]
    next_id = int(np.argmax(logits))  # or sample with temperature/top-k
    if next_id == 2:  # <|im_end|>
        break
    ids.append(next_id)
print(tok.decode(ids[len(tok.encode(prompt).ids):]))

Limitations

• Single-turn only — the model was trained on independent one-message exchanges
• 128-token context window
• English only, 60 fixed conversation topics; anything outside fish life confuses it
• A toy model for learning, not a production chatbot

License

MIT. Dataset and original project by arman-bd/guppylm.