MARKDOWN STOP EATING MY TAGS haha
Browse files
README.md
CHANGED
|
@@ -22,7 +22,7 @@ Brought to you via @ContextReq - Built, tested and trained within 24 hours! (exc
|
|
| 22 |
|
| 23 |
A deliberately minimal character-level language model (3.7M params, 2-layer GRU) trained on TinyStories, using a curated 109-token vocabulary: 8 boundary/whitespace tags, 94 printable ASCII characters, and 7 smart punctuation marks.
|
| 24 |
|
| 25 |
-
Notio's core idea: the model sees only token ids — a u8 stream, one byte per id — and all human-readable "english/symbols" rendering happens at runtime through a decoder that maps special tags (<spc>, <nwl>, <bos>, <eos>…) to their effects and drops them from display. The stream is byte-sized, not byte-level: it models the 109 curated characters above, not all 256 raw byte values.
|
| 26 |
|
| 27 |
> **Status:** v1 Found the floor at 24k steps @ 0.679 val.
|
| 28 |
|
|
|
|
| 22 |
|
| 23 |
A deliberately minimal character-level language model (3.7M params, 2-layer GRU) trained on TinyStories, using a curated 109-token vocabulary: 8 boundary/whitespace tags, 94 printable ASCII characters, and 7 smart punctuation marks.
|
| 24 |
|
| 25 |
+
Notio's core idea: the model sees only token ids — a u8 stream, one byte per id — and all human-readable "english/symbols" rendering happens at runtime through a decoder that maps special tags (`<spc>`, `<nwl>`, `<bos>`, `<eos>`…) to their effects and drops them from display. The stream is byte-sized, not byte-level: it models the 109 curated characters above, not all 256 raw byte values.
|
| 26 |
|
| 27 |
> **Status:** v1 Found the floor at 24k steps @ 0.679 val.
|
| 28 |
|