ContextReq commited on
Commit
252a84b
·
verified ·
1 Parent(s): 487f2a9

MARKDOWN STOP EATING MY TAGS haha

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -22,7 +22,7 @@ Brought to you via @ContextReq - Built, tested and trained within 24 hours! (exc
22
 
23
  A deliberately minimal character-level language model (3.7M params, 2-layer GRU) trained on TinyStories, using a curated 109-token vocabulary: 8 boundary/whitespace tags, 94 printable ASCII characters, and 7 smart punctuation marks.
24
 
25
- Notio's core idea: the model sees only token ids — a u8 stream, one byte per id — and all human-readable "english/symbols" rendering happens at runtime through a decoder that maps special tags (<spc>, <nwl>, <bos>, <eos>…) to their effects and drops them from display. The stream is byte-sized, not byte-level: it models the 109 curated characters above, not all 256 raw byte values.
26
 
27
  > **Status:** v1 Found the floor at 24k steps @ 0.679 val.
28
 
 
22
 
23
  A deliberately minimal character-level language model (3.7M params, 2-layer GRU) trained on TinyStories, using a curated 109-token vocabulary: 8 boundary/whitespace tags, 94 printable ASCII characters, and 7 smart punctuation marks.
24
 
25
+ Notio's core idea: the model sees only token ids — a u8 stream, one byte per id — and all human-readable "english/symbols" rendering happens at runtime through a decoder that maps special tags (`<spc>`, `<nwl>`, `<bos>`, `<eos>`…) to their effects and drops them from display. The stream is byte-sized, not byte-level: it models the 109 curated characters above, not all 256 raw byte values.
26
 
27
  > **Status:** v1 Found the floor at 24k steps @ 0.679 val.
28