Update README.md

README.md

@@ -8,12 +8,30 @@ pipeline_tag: text-generation
 tags:
 - ASCII
 - BINARY
+- RNN
 ---
 
 >:), keep an eye out or don't, who knows - im insane muhehehe.
 
-ill clean this page up at some point.
+A tiny char-level language model that learns only the binary u8 id stream; all "english/symbols" rendering happens at runtime via a decoder that maps tags (<spc>, <nwl>, <bos>…) to their effects and drops them from display.
+  Status: As of this being writen, a full run has begun.
 
-109 token IDS for vocab.txt/.bin, line number is implied token id, token value e.g. line 99 == id 99, "token", coverted into binary(.bin).
+Architecture (modular, 12-layer naming, 4 building blocks)
+layer	component	                       config
+0       binary input (memmap, chunk/tapes) B=32, T=1024
+1	    embedding wte + wpe	               vocab=109, d=512
+2–3	    GRU blocks (pre-LN + residual) ×2  d=512
+11	    head (ln_f + lm_head, tied to wte) 109-way
 
-datasets included; cleaned to fit the training and token format, along with %95/%5 train/val split.
+3,735,040 params, fp32 ≈ 15MB weights.
+Causal by construction; hidden state carried across windows.
+   
+Data
+  TinyStoriesV2-GPT4: 2,118,989 stories → u8 id stream, 1,799,636,685 train ids / 95,696,324 val (95/5, story-boundary split, byte-exact verified).
+  Vocab: 109 tokens — 8 specials, 94 ASCII, 7 smart punct; vocab.txt == vocab.bin asserted at every train launch.
+
+Training
+  Stateful truncated BPTT (k1 = k2 = 1024): gradients truncated per window, state flows forward, reset only at sweep end — truncation + statefulness + tape continuity formally proven in --check.
+  AdamW, lr 4e-4, warmup 500, cosine → 4e-5, grad-clip 1.0.
+  Hardware: GTX 1660 SUPER (6GB, Turing) — ~95k tok/s, ~5.5h/epoch; dataset stays on CPU memmap (measured: moving it to GPU = +0.08%).
+  Checkpoints every 1,000 steps: step, train/val loss, lr, sweep position, weights, plus 2 sampled stories — resumable.