Download FORMAT.md from MostLime/keylime-m3-preview: direct link, hf CLI and curl.
- Browser
- Download file 8 kB
-
https://huggingface.co/MostLime/keylime-m3-preview/resolve/main/FORMAT.md
- Command line
-
hf download hf://MostLime/keylime-m3-preview/FORMAT.md
-
curl -L -o FORMAT.md https://huggingface.co/MostLime/keylime-m3-preview/resolve/main/FORMAT.md
m3 app files
This folder holds the files that the Keylime apps download for the m3 model. The algorithm that uses them is in
m3dict_cpt/PORT_SPEC.md (the m3dict decoder). The reference code is m3dict_cpt/onnx_backend.py (M3Dict).
The test vectors are in m3dict_cpt/test_vectors.json.
All binary files are little-endian. All offsets are in bytes.
Files
| File | Content |
|---|---|
m3.encoder.int4.onnx |
Encoder. Inputs: src_ids int64 [1,S], taps float32 [1,S,2] (zeros). Outputs: memory, cross_k0..2, cross_v0..2 float32 [1,2,S,64]. |
m3.decoder.int4.onnx |
Decoder with KV cache. See "Decoder" below. |
m3.manifest.json |
Export metadata: config, style ids (casual 256, standard 257, formal 258), EOS 2, max_src 384, max_tgt 112. |
vocab_m3.json |
SentencePiece BPE vocabulary (32,768 pieces, byte fallback) in the web/export_vocab.py format. Byte-identical to vocab_m2.json. |
m3_lexicon.tsv |
The 100,000-word shipping lexicon. One word<TAB>count per line, sorted by count (high first). Line number = lexicon id. |
m3_symspell.bin |
Delete index for candidate lookup. |
m3_ngram.bin |
Pruned and quantized KN word trigram (the ng feature). |
The apps derive the other tables from m3_lexicon.tsv when they load it:
- Log prior:
log(count / sum of counts)(float64). - Letter bag: 26 letter counts per word (apostrophes not counted), and the letter count.
- Prefix table: for each prefix of a lexicon word, the shortest letter count (apostrophes not counted) of a word with that prefix. A sorted word list plus a range-minimum query gives the same values as a full prefix map.
- Apostrophe expansions: the word without apostrophes -> the lexicon words with apostrophes, in lexicon order.
Accuracy level (set at load)
Both graphs are stored with accuracy_level 0 on every MatMulNBits node (fp32 dequantization: slow). The apps run
them with accuracy_level 4 (int8 activations against the same int4 weights: about 9 times faster per decoder step).
The change is made in memory before the session is created: in the graph bytes, replace every occurrence of
0a 0e "accuracy_level" 18 00 (attribute name, then the int value 0)
with the same bytes but a last byte of 04. The protobuf lengths do not change. The encoder has 65 such nodes and the
decoder has 28. ONNX Runtime has no session option for this (session.qdq_matmulnbits_accuracy_level only applies
to QDQ fusion). The test vectors are made with level 4 (onnx_backend.with_accuracy_level). Keyboard word-end path
on Typing37-Clean held-out (30,177 messages): level 0 fix 64.487 / FC 0.989, level 4 fix 64.426 / FC 0.992.
Input to the encoder
src_ids = [style id] + the UTF-8 bytes of the text, cut to 384 ids in total. taps = zeros [1,S,2].
Decoder
Inputs: ids int64 [1,N], next_ids int64 [1,N] (zeros when not needed), pos int64 [N] (absolute positions),
past_mask bool [P], src_mask bool [S] (all true), past_k0/past_v0 float32 [1,2,P,64], past_c0/past_c1
float32 [1,6,512], cross_* (from the encoder).
Outputs: last_lp float32 [1,32768] (log-probs after the last id), tok_lp float32 [1,N] (log-prob of next_ids[i]
after ids[..i]), present_k0, present_v0, present_c0, present_c1.
The start state: past_k0/past_v0 zeros [1,2,1,64], past_c* zeros [1,6,512], past_mask = [false], position 0.
The first id is EOS (2). After a call, the new mask is the old mask + N times true, and the position moves by N.
m3_symspell.bin
offset 0 char[4] "KLSS"
offset 4 u32 version (1)
offset 8 u32 n_words (100000)
offset 12 u32 n_entries
offset 16 u32 maxd (2)
offset 20 u32 short (2)
offset 24 u32 key[n_entries]
u32 word_id[n_entries]
For each lexicon word w, let s = w without apostrophes. The index has one entry for each string in deletes(s, d)
(s itself and all strings made by deleting up to d letters), with d = maxd if len(s) > short, else d = 1.
key = the high 32 bits of FNV-1a 64 of the UTF-8 bytes of the delete string. Entries are sorted by (key, word_id).
Lookup for a typed word t (lowercase, apostrophes removed, length n): make deletes(t, dq) with dq = 3 if n >= 7, dq = 2 if n > 2, else dq = 1. For each delete q, find the entries with key = hash32(q). Accept a word id only if q is a subsequence of the word's s and 0 <= len(s) - len(q) <= d (d of that word, as above). This check removes all 32-bit key collisions, so the result is exactly the set of words that share a delete with t. Return the sorted unique ids.
m3_ngram.bin
The KN word trigram of the m2 data (data/stats/ngram_kn.npz, 508 MB), pruned to trigram count >= 12 and bigram
continuation count >= 12 (m3dict_cpt/qkn.py). The backoff weights are computed again from the full totals, so the
pruned model is still an interpolated KN model. It is stored in backoff form:
log P(w | u v) = T(u,v,w) if the trigram (u,v,w) is stored
= G3(u,v) + B(v,w) else, if the history (u,v) is stored
= B(v,w) else
B(v,w) = Q2(v,w) if the bigram (v,w) is stored
= G2(v) + P1(w) else
Each value x is stored as q = round(-x * 1024), and the value is read back as -q / 1024. All sums of these values are exact in float64, so all ports get bit-identical results.
offset 0 char[4] "KLQN"
offset 4 u32 version (1)
u32 V (vocabulary size)
u32 n_bi, n_hist, n_tri
u32 vocab_bytes
u8 vocab[vocab_bytes] UTF-8, words joined by '\n'; id 0 = <unk>, 1 = <s>, 2 = <num>
pad to a multiple of 4
u16 P1[V] q of log P1(w)
u16 G2[V] q of log G2(v) (0 when v has no bigram statistics)
u32 bi_off[V+1] bigrams of v: entries bi_off[v] .. bi_off[v+1]-1
u32 bi[n_bi] (w << 15) | q(Q2(v,w)), w ascending in each v
u32 hist_off[V+1] histories (u, *): entries hist_off[u] .. hist_off[u+1]-1
u32 hist[n_hist] (v << 15) | q(G3(u,v)), v ascending in each u
u32 tri_off[n_hist+1] trigrams of history h: entries tri_off[h] .. tri_off[h+1]-1
u32 tri[n_tri] (w << 15) | q(T(u,v,w)), w ascending in each history
Find an entry with a binary search on entry >> 15 in its range. The 15-bit fields hold q up to 32767 (log value
down to -32). The u16 fields hold q up to 65535.
Sentence log-prob (sent_lp): words = the lowercase SEG words ([A-Za-z]+(?:'[A-Za-z]+)*) of the text. The id of a
word is its vocabulary index, or 0 if it is not in the vocabulary (a word that starts with a digit gets 2). History
starts as (1, 1). For each word: add -16.0 if its id is 0, else log P(w | u v). Then shift the history (an OOV word
enters the history as id 0). Add the values from left to right in float64.
ng of an alternative = sent_lp(the greedy text with the slot replaced) - sent_lp(the greedy text).
Decoder constants
Shipping parameters: m3dict_cpt/params_ship.json (also in the manifest entry). Constants that are not in that file:
| Name | Value |
|---|---|
| top-K tokens per greedy step | 8 (ties: log-prob desc, then token id asc) |
| tau_split | 4.0 |
| greedy steps | min(112, len(src_ids) // 2 + 24) |
| candidates per slot (slot plan, injection) | 8 |
| RARE rank | 10000 (a slot plan uses min(10000, params.rare)) |
| generator | prior weight 0.5, max hits 4000, qd3 7, short 2, bag top 300 for words of >= 6 letters (ties: score desc, then lexicon id asc) |
| spatial model | keyboard.SpatialModel defaults (sigma 0.3, key-centre touches), QWERTY eval geometry; splits use space bar x 3.5 .. 8.5 |
| feature rounding | dm3, sp, pr, ng: Python round(x, 4) |
| LP_OOV | -16.0 |