mindtype-tagger-v1.6
MindType's own grammar and spelling correction model, trained from scratch by Mindscend for an on-device keyboard (iOS and Android). Text never leaves the phone.
- 26.2M parameters, encoder-only transformer (8 layers, d=384), own 16k byte-level BPE tokenizer.
- Predicts one edit label per word (KEEP / DELETE / REPLACE_x / APPEND_x / case) in a single pass, so it is fast, tiny, and cannot invent words or rewrite correct text.
- Core ML package (8-bit weights) for iOS 18+, and ONNX (int8) for Android and servers.
What changed since v1.5: 1.2% of the training pairs are now past-tense web sentences with a present-tense error ("Yesterday I go"), and everything else is the v1.5 recipe (12M pairs, a fixed 30,000 steps). Tense errors went from almost never fixed (F0.5 0.12, 2.4% exact) to mostly fixed (F0.5 0.88, 64.9% exact), and it changes correct real-world messages less often (1.65% vs 1.95%). The cost: slightly more changes to correct sentences in the main suite (2.8% vs 2.4%).
Results (MindType evaluation suite, 4,356 held-out cases)
Settings: KEEP bias 2.0 and the plausibility guard used in the MindType apps.
| Set | n | F0.5 | Exact | False-positive rate | Lost protected spans |
|---|---|---|---|---|---|
| casual_clean | 118 | – | 0.975 | 0.025 | 0.000 |
| casual_noisy | 109 | 0.940 | 0.807 | 0.000 | 0.000 |
| coedit | 579 | 0.467 | 0.119 | 0.000 | 0.000 |
| formal_clean | 1000 | – | 0.957 | 0.043 | 0.000 |
| formal_noisy | 1000 | 0.870 | 0.663 | 0.000 | 0.000 |
| golden | 50 | 0.939 | 0.860 | 0.000 | 0.000 |
| preserve | 1000 | 0.852 | 0.670 | 0.000 | 0.000 |
| preserve_clean | 500 | – | 1.000 | 0.000 | 0.000 |
| overall | 4356 | 0.802 | 0.713 | 0.028 | 0.000 |
False-positive rate on 2,000 held-out casual messages (never used for training; used only to choose between two training seeds): 1.65% (v1.5: 1.95% on the same messages).
On the *_clean sets there is nothing to fix; the false-positive rate is what counts there.
Compared with fine-tuned small LLMs and earlier versions (same suite)
| Model | Size | F0.5 ↑ | Exact ↑ | False positives ↓ | Lost names/numbers ↓ |
|---|---|---|---|---|---|
| Gemma 3 270M instruct, zero-shot | 270M | 0.122 | 0.2% | 99.6% | 3.9% |
| Gemma 3 270M + LoRA r32 | 270M | 0.571 | 57.4% | 23.1% | 2.9% |
| SmolLM2 360M + LoRA r32 | 360M | 0.663 | 63.1% | 20.1% | 0.3% |
| mindtype-tagger-v1.1 (keep bias 2, guard) | 26M | 0.748 | 68.3% | 8.65% | 0.0% |
| mindtype-tagger-v1.2 (keep bias 2, guard) | 26M | 0.800 | 72.4% | 3.1% | 0.0% |
| mindtype-tagger-v1.5 (keep bias 2, guard) | 26M | 0.799 | 71.2% | 2.4% | 0.0% |
| mindtype-tagger-v1.6 (keep bias 2, guard) | 26M | 0.802 | 71.3% | 2.8% | 0.0% |
Error types (separate test sets, not used for training)
| Error type | n | v1.5 F0.5 | v1.6 F0.5 | v1.6 exact |
|---|---|---|---|---|
| Real misspellings ("recieve") | 500 | 0.963 | 0.957 | 92.2% |
| Dropped words ("I going home") | 500 | 0.698 | 0.683 | 40.4% |
| Present tense about the past ("Yesterday I go") | 248 | 0.119 | 0.885 | 64.9% |
Dropped words are now the weakest; they're next.
Files
| File | What it is |
|---|---|
model.safetensors |
PyTorch weights (fp32): encoder + per-token classification head |
config.json |
architecture (layers, width, heads, vocabulary and label counts) |
tokenizer.json |
16k byte-level BPE tokenizer (Hugging Face tokenizers format) |
labels.json |
the 15,000 edit labels, index → label |
plausible_pairs.json |
the word pairs the plausibility guard accepts |
coreml/MindTypeTagger.mlpackage |
Core ML model (8-bit weights, 26 MB), inputs padded to 16/32/64/96/160 tokens; 99.94% label agreement with fp32 |
onnx/mindtype_tagger.onnx |
ONNX model (opset 17, int8 dynamic quantization, 27 MB), inputs ids, mask (int64, [1, T]), output logits; 99.2% label agreement with fp32 |
metrics.json, log.jsonl, pretrain-log.jsonl, run.json |
evaluation results and training logs |
Training
- Pretraining: masked language modeling on FineWeb-Edu (ODC-BY): 1 hour on ~20M sentences, then 6.8 hours on 61.7M sentences.
- Correction: 12.02M (noisy, clean) pairs (30,000 steps at batch 256): synthetic phone-typing errors, CoEdIT GEC (Apache-2.0), and casual messages written by a local open-weight model (Qwen2.5-14B-Instruct) and filtered for correctness, plus real misspellings, dropped words and tense errors (1.2% of pairs: past-tense sentences with a tense error). 30% are already-correct text, so the model learns to leave good text alone.
Hardware: one RTX 3080 Ti laptop GPU, about 9 hours in total.
How it works
Text is split into words and punctuation; each word gets a label from labels.json: $KEEP, $DELETE,
$REPLACE_<word>, $APPEND_<word> (insert after), or a case change. The corrected sentence is the labels applied
to the words, and the model is run again (up to 3 passes) on its own output. One sentence takes about 4 ms on a
phone CPU (Core ML) and on a laptop CPU (ONNX Runtime).
Inference settings used for the results: KEEP bias 2.0 (added to the $KEEP logit; higher = fewer, surer fixes).
Names, numbers, emails and URLs are never changed.
Limitations
English only, sentence-level, experimental. Fixes what's there; it doesn't reword or change tone. About a third of tense errors are still missed. It can still change correct text (2.8% of correct sentences in our suite, 1.65% of real casual messages).
License
Apache-2.0 (weights, tokenizer, Core ML and ONNX models). Training data: FineWeb-Edu (ODC-BY), CoEdIT (Apache-2.0).
- Downloads last month
- -