vynly/mica-ember-0.3.7

MICA Ember v0.3.7

A one-of-a-kind language model. Not a transformer, not a neural network.

MICA models language with learned integer rewrite rules on a ring of cells, a cellular automaton. It is an original architecture, designed and trained from scratch. Ember is the byte-level MICA model: it reads text one UTF-8 byte at a time, suggests the next word, and finishes the word you are typing.

Its sibling MICA Flame-W 0.3.5 predicts whole words and continues sentences. Try both in the live playground.

Release notes: RELEASE_NOTES.md, the same text as the GitHub release.

What changed from v0.3.5

1. Word rules over the previous words. Before each suggestion, 16 extra MICA rule ticks run. Each previous word gets a fixed 16-value integer code (a hash of the word); a tick picks a page by comparing pairs of values, scores 256 candidate rules with six ±1 terms plus an integer bias, and the winner writes six learned integers that the readout also reads. The terms read the codes of the previous three words and the automaton's probes, so a rule can combine "weather" + "today" + "is".
2. A word memory. Each word readout reads the automaton's 240 probes at the current position and at the starts of the previous 2 words.
3. A bigger vocabulary. The word readouts know 32,768 words (v0.3.5: 8,192), all learned from training text.
4. Search after any number of letters. Completion works after one, two, three or more typed letters, with prefix indexes and exact float32 products of the integer weights.
5. More training. The word readouts and the rules' values were fitted together on the exact integer probe states of 697,070 training texts (v0.3.5: 119,823). The automaton, its byte readout and its 14 KB byte memory are unchanged.

Results

Clean word benchmark (400 prompts)	Ember v0.3.5	Ember v0.3.7	Paired change (95% CI)
Next word, top-1	15.25%	21.50%	+6.25 [+2.50, +10.00] points
Next word, in top 5	34.75%	43.25%	+8.50 [+4.50, +12.75] points
Finish a word after 1 letter, top-1	43.50%	50.75%	+7.25 [+3.25, +11.25] points
Finish a word after 2 letters, top-1	54.50%	63.75%	+9.25 [+5.25, +13.00] points
Finish a word after 3 letters, top-1	73.00%	77.25%	+4.25 [+0.25, +8.00] points

On 1,920 older development prompts that no choice used: next word 13.92% → 22.65%, two-letter completion 53.06% → 66.03% (mean of the two +10.86 [+9.48, +12.22] points).

Tried and not kept: rules over the automaton's states at earlier word starts, one model with both rule kinds (not clearly better), more text without the rules, and a bigger byte memory. Ember v0.3.6 was an internal step and was not released. The configuration was chosen on development data only; clean sets were used only for reporting; the sealed test set was not accessed. Details are in README.md of the package on GitHub and in results/.

Why MICA is different

• No transformer inside. No attention and no stack of neural layers. Each byte is written onto a ring of integer cells, learned local rules rewrite the ring, and probes read the result. The word rules are the same kind of rule.
• Integers only in the cellular updates, the word rules, the probes and the word scores.
• Deterministic. The same text always gives the same suggestions.
• Small and CPU-only. Runs on any CPU with numpy alone.

Files

File	What it is
model.mica	The automaton and byte readout, SHA-256 c2bef375c8dcd6f944e1a44f4b92869645ddf97eb386382fa322cf9519eda506
memory.npz	Integer byte memory (unchanged from v0.3.5), SHA-256 3827c2a38981347bff17f9666543c333178c5a89fdd089d216e5c6f8ab8847c8
word_heads.npz	Integer word readouts (32,768 words, word memory) and 16 word-rule ticks per readout, SHA-256 40f3bfa6f9692f4424d05cc3666afdca7ecbd648244df059ce2d181f66561e67
word_model.py, mica_r1/	Exact integer runtime (numpy only)
test_word_model.py, test_memory.py	Conformance tests
results/	Evaluation summaries and the pass marks fixed before each run
RELEASE_NOTES.md	Release notes for v0.3.7 (the same text as the GitHub release)
CHECKSUMS.json	SHA-256 of every file

pip install numpy
python test_word_model.py
python word_model.py "Thank you for "
python word_model.py --complete "Thank you for b"

Limitations

A research model with a training-only vocabulary of 32,768 words of four or more letters. Personal and research use only; commercial use is forbidden (see LICENSE).

Code and history: github.com/Vovala14/Mica-Ai.