# m3 app files This folder holds the files that the Keylime apps download for the m3 model. The algorithm that uses them is in `m3dict_cpt/PORT_SPEC.md` (the m3dict decoder). The reference code is `m3dict_cpt/onnx_backend.py` (`M3Dict`). The test vectors are in `m3dict_cpt/test_vectors.json`. All binary files are little-endian. All offsets are in bytes. ## Files | File | Content | |---|---| | `m3.encoder.int4.onnx` | Encoder. Inputs: `src_ids` int64 [1,S], `taps` float32 [1,S,2] (zeros). Outputs: `memory`, `cross_k0..2`, `cross_v0..2` float32 [1,2,S,64]. | | `m3.decoder.int4.onnx` | Decoder with KV cache. See "Decoder" below. | | `m3.manifest.json` | Export metadata: config, style ids (casual 256, standard 257, formal 258), EOS 2, `max_src` 384, `max_tgt` 112. | | `vocab_m3.json` | SentencePiece BPE vocabulary (32,768 pieces, byte fallback) in the `web/export_vocab.py` format. Byte-identical to `vocab_m2.json`. | | `m3_lexicon.tsv` | The 100,000-word shipping lexicon. One `wordcount` per line, sorted by count (high first). Line number = lexicon id. | | `m3_symspell.bin` | Delete index for candidate lookup. | | `m3_ngram.bin` | Pruned and quantized KN word trigram (the `ng` feature). | The apps derive the other tables from `m3_lexicon.tsv` when they load it: - **Log prior**: `log(count / sum of counts)` (float64). - **Letter bag**: 26 letter counts per word (apostrophes not counted), and the letter count. - **Prefix table**: for each prefix of a lexicon word, the shortest letter count (apostrophes not counted) of a word with that prefix. A sorted word list plus a range-minimum query gives the same values as a full prefix map. - **Apostrophe expansions**: the word without apostrophes -> the lexicon words with apostrophes, in lexicon order. ## Accuracy level (set at load) Both graphs are stored with `accuracy_level` 0 on every MatMulNBits node (fp32 dequantization: slow). The apps run them with `accuracy_level` 4 (int8 activations against the same int4 weights: about 9 times faster per decoder step). The change is made in memory before the session is created: in the graph bytes, replace every occurrence of ``` 0a 0e "accuracy_level" 18 00 (attribute name, then the int value 0) ``` with the same bytes but a last byte of 04. The protobuf lengths do not change. The encoder has 65 such nodes and the decoder has 28. ONNX Runtime has no session option for this (`session.qdq_matmulnbits_accuracy_level` only applies to QDQ fusion). The test vectors are made with level 4 (`onnx_backend.with_accuracy_level`). Keyboard word-end path on Typing37-Clean held-out (30,177 messages): level 0 fix 64.487 / FC 0.989, level 4 fix 64.426 / FC 0.992. ## Input to the encoder `src_ids` = `[style id]` + the UTF-8 bytes of the text, cut to 384 ids in total. `taps` = zeros [1,S,2]. ## Decoder Inputs: `ids` int64 [1,N], `next_ids` int64 [1,N] (zeros when not needed), `pos` int64 [N] (absolute positions), `past_mask` bool [P], `src_mask` bool [S] (all true), `past_k0`/`past_v0` float32 [1,2,P,64], `past_c0`/`past_c1` float32 [1,6,512], `cross_*` (from the encoder). Outputs: `last_lp` float32 [1,32768] (log-probs after the last id), `tok_lp` float32 [1,N] (log-prob of `next_ids[i]` after `ids[..i]`), `present_k0`, `present_v0`, `present_c0`, `present_c1`. The start state: `past_k0`/`past_v0` zeros [1,2,1,64], `past_c*` zeros [1,6,512], `past_mask` = `[false]`, position 0. The first id is EOS (2). After a call, the new mask is the old mask + N times `true`, and the position moves by N. ## m3_symspell.bin ``` offset 0 char[4] "KLSS" offset 4 u32 version (1) offset 8 u32 n_words (100000) offset 12 u32 n_entries offset 16 u32 maxd (2) offset 20 u32 short (2) offset 24 u32 key[n_entries] u32 word_id[n_entries] ``` For each lexicon word w, let s = w without apostrophes. The index has one entry for each string in deletes(s, d) (s itself and all strings made by deleting up to d letters), with d = maxd if len(s) > short, else d = 1. `key` = the high 32 bits of FNV-1a 64 of the UTF-8 bytes of the delete string. Entries are sorted by (key, word_id). Lookup for a typed word t (lowercase, apostrophes removed, length n): make deletes(t, dq) with dq = 3 if n >= 7, dq = 2 if n > 2, else dq = 1. For each delete q, find the entries with key = hash32(q). Accept a word id only if q is a subsequence of the word's s and 0 <= len(s) - len(q) <= d (d of that word, as above). This check removes all 32-bit key collisions, so the result is exactly the set of words that share a delete with t. Return the sorted unique ids. ## m3_ngram.bin The KN word trigram of the m2 data (`data/stats/ngram_kn.npz`, 508 MB), pruned to trigram count >= 12 and bigram continuation count >= 12 (`m3dict_cpt/qkn.py`). The backoff weights are computed again from the full totals, so the pruned model is still an interpolated KN model. It is stored in backoff form: ``` log P(w | u v) = T(u,v,w) if the trigram (u,v,w) is stored = G3(u,v) + B(v,w) else, if the history (u,v) is stored = B(v,w) else B(v,w) = Q2(v,w) if the bigram (v,w) is stored = G2(v) + P1(w) else ``` Each value x is stored as q = round(-x * 1024), and the value is read back as -q / 1024. All sums of these values are exact in float64, so all ports get bit-identical results. ``` offset 0 char[4] "KLQN" offset 4 u32 version (1) u32 V (vocabulary size) u32 n_bi, n_hist, n_tri u32 vocab_bytes u8 vocab[vocab_bytes] UTF-8, words joined by '\n'; id 0 = , 1 = , 2 = pad to a multiple of 4 u16 P1[V] q of log P1(w) u16 G2[V] q of log G2(v) (0 when v has no bigram statistics) u32 bi_off[V+1] bigrams of v: entries bi_off[v] .. bi_off[v+1]-1 u32 bi[n_bi] (w << 15) | q(Q2(v,w)), w ascending in each v u32 hist_off[V+1] histories (u, *): entries hist_off[u] .. hist_off[u+1]-1 u32 hist[n_hist] (v << 15) | q(G3(u,v)), v ascending in each u u32 tri_off[n_hist+1] trigrams of history h: entries tri_off[h] .. tri_off[h+1]-1 u32 tri[n_tri] (w << 15) | q(T(u,v,w)), w ascending in each history ``` Find an entry with a binary search on `entry >> 15` in its range. The 15-bit fields hold q up to 32767 (log value down to -32). The u16 fields hold q up to 65535. Sentence log-prob (`sent_lp`): words = the lowercase SEG words (`[A-Za-z]+(?:'[A-Za-z]+)*`) of the text. The id of a word is its vocabulary index, or 0 if it is not in the vocabulary (a word that starts with a digit gets 2). History starts as (1, 1). For each word: add -16.0 if its id is 0, else log P(w | u v). Then shift the history (an OOV word enters the history as id 0). Add the values from left to right in float64. `ng` of an alternative = sent_lp(the greedy text with the slot replaced) - sent_lp(the greedy text). ## Decoder constants Shipping parameters: `m3dict_cpt/params_ship.json` (also in the manifest entry). Constants that are not in that file: | Name | Value | |---|---| | top-K tokens per greedy step | 8 (ties: log-prob desc, then token id asc) | | tau_split | 4.0 | | greedy steps | min(112, len(src_ids) // 2 + 24) | | candidates per slot (slot plan, injection) | 8 | | RARE rank | 10000 (a slot plan uses min(10000, params.rare)) | | generator | prior weight 0.5, max hits 4000, qd3 7, short 2, bag top 300 for words of >= 6 letters (ties: score desc, then lexicon id asc) | | spatial model | keyboard.SpatialModel defaults (sigma 0.3, key-centre touches), QWERTY eval geometry; splits use space bar x 3.5 .. 8.5 | | feature rounding | dm3, sp, pr, ng: Python round(x, 4) | | LP_OOV | -16.0 |