stripe1flash mountain

A 0.000001B parameter (982 params) character-level language model, written from scratch β€” forward pass, backprop, and Adam hand-implemented, only numpy required.

It knows 24 characters and one page of English about a cat, a dog, a man and the sun. That is the whole world it can remember. And honestly? It does okay.

Specs

Parameters 982 (= 0.000001B)
Vocab 24 chars (lowercase + space + punctuation + newline)
Architecture embed(6) β†’ concat(2 ctx) β†’ tanh(22) β†’ softmax(24)
Corpus 2,118 chars of hand-written simple English (corpus.txt)
Training 8,000 steps Γ— batch 256, Adam, cosine LR, ~25 s on CPU
Loss 3.18 (uniform baseline) β†’ ~0.59

Weights

  • stripe1flash_mountain.json β€” the full 982-parameter checkpoint (~100 KB, JSON)
  • stripe1flash_mountain.gguf β€” the same weights in GGUF format (F32, ~5 KB)

⚠️ The GGUF file uses a custom architecture (stripe1flash-mountain), so llama.cpp cannot load it β€” it's provided for format completeness / fun. Use the JSON checkpoint with the numpy code above to actually run the model.

Quick start

import json, random
import numpy as np

ck = json.load(open("stripe1flash_mountain.json", encoding="utf-8"))
vocab = ck["vocab"]; c2i = {c: i for i, c in enumerate(vocab)}; i2c = {i: c for i, c in enumerate(vocab)}
W = {k: np.array(v) for k, v in ck["weights"].items()}

def generate(prompt="the ", n=160, temperature=0.7, seed=0):
    r = random.Random(seed)
    ids = [c2i[c] for c in (" " + prompt)[-2:]]
    out = prompt
    for _ in range(n):
        x = np.concatenate([W["emb"][ids[0]], W["emb"][ids[1]]])
        h = np.tanh(W["w1"] @ x + W["b1"])
        z = (W["w2"] @ h + W["b2"]) / temperature
        z -= z.max(); q = np.exp(z); q /= q.sum()
        u, acc, nxt = r.random(), 0.0, len(vocab) - 1
        for i, qi in enumerate(q):
            acc += qi
            if u <= acc: nxt = i; break
        out += i2c[nxt]; ids = [ids[1], nxt]
    return out

print(generate("the cat "))

Sample output:

the dog sat. the sun ran blue. the sat sky the set. the sat and. the mat of the man the dog sun.

Train it yourself

pip install numpy
python train.py     # ~25 s, re-saves the checkpoint + prints samples
python play.py      # interactive REPL, type a prompt and let it continue

Files

  • stripe1flash_mountain.json β€” trained weights
  • config.json β€” architecture config
  • train.py β€” from-scratch training (hand-written backprop + Adam)
  • play.py β€” interactive generation
  • ui.py β€” tkinter GUI (also shipped as a standalone Windows exe)
  • corpus.txt β€” the entire training corpus (one page!)

Limitations

  • 982 parameters. It invents words (mand, cate), forgets anything beyond a 2-character context, and only speaks its 24-character alphabet.
  • Trained for fun and education β€” not useful for anything else. That's the point.

License

MIT

Downloads last month
57
GGUF
Model size
982 params
Architecture
stripe1flash-mountain
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support