stripe1flash mountain
A 0.000001B parameter (982 params) character-level language model, written from scratch β forward pass, backprop, and Adam hand-implemented, only numpy required.
It knows 24 characters and one page of English about a cat, a dog, a man and the sun. That is the whole world it can remember. And honestly? It does okay.
Specs
| Parameters | 982 (= 0.000001B) |
| Vocab | 24 chars (lowercase + space + punctuation + newline) |
| Architecture | embed(6) β concat(2 ctx) β tanh(22) β softmax(24) |
| Corpus | 2,118 chars of hand-written simple English (corpus.txt) |
| Training | 8,000 steps Γ batch 256, Adam, cosine LR, ~25 s on CPU |
| Loss | 3.18 (uniform baseline) β ~0.59 |
Weights
stripe1flash_mountain.jsonβ the full 982-parameter checkpoint (~100 KB, JSON)stripe1flash_mountain.ggufβ the same weights in GGUF format (F32, ~5 KB)
β οΈ The GGUF file uses a custom architecture (stripe1flash-mountain), so llama.cpp cannot load it β it's provided for format completeness / fun. Use the JSON checkpoint with the numpy code above to actually run the model.
Quick start
import json, random
import numpy as np
ck = json.load(open("stripe1flash_mountain.json", encoding="utf-8"))
vocab = ck["vocab"]; c2i = {c: i for i, c in enumerate(vocab)}; i2c = {i: c for i, c in enumerate(vocab)}
W = {k: np.array(v) for k, v in ck["weights"].items()}
def generate(prompt="the ", n=160, temperature=0.7, seed=0):
r = random.Random(seed)
ids = [c2i[c] for c in (" " + prompt)[-2:]]
out = prompt
for _ in range(n):
x = np.concatenate([W["emb"][ids[0]], W["emb"][ids[1]]])
h = np.tanh(W["w1"] @ x + W["b1"])
z = (W["w2"] @ h + W["b2"]) / temperature
z -= z.max(); q = np.exp(z); q /= q.sum()
u, acc, nxt = r.random(), 0.0, len(vocab) - 1
for i, qi in enumerate(q):
acc += qi
if u <= acc: nxt = i; break
out += i2c[nxt]; ids = [ids[1], nxt]
return out
print(generate("the cat "))
Sample output:
the dog sat. the sun ran blue. the sat sky the set. the sat and. the mat of the man the dog sun.
Train it yourself
pip install numpy
python train.py # ~25 s, re-saves the checkpoint + prints samples
python play.py # interactive REPL, type a prompt and let it continue
Files
stripe1flash_mountain.jsonβ trained weightsconfig.jsonβ architecture configtrain.pyβ from-scratch training (hand-written backprop + Adam)play.pyβ interactive generationui.pyβ tkinter GUI (also shipped as a standalone Windows exe)corpus.txtβ the entire training corpus (one page!)
Limitations
- 982 parameters. It invents words (
mand,cate), forgets anything beyond a 2-character context, and only speaks its 24-character alphabet. - Trained for fun and education β not useful for anything else. That's the point.
License
MIT
- Downloads last month
- 57
Hardware compatibility
Log In to add your hardware
We're not able to determine the quantization variants.
Inference Providers NEW
This model isn't deployed by any Inference Provider. π Ask for provider support