tinyfix
Spell check can't tell that "form" was meant to be "from", since both are real words. This 2.1 MB model can.
It reads the words on both sides of a correctly spelled word and picks the best fit from nearby real words. It runs on the page with ONNX Runtime Web, so nothing you type is sent anywhere.
| Parameters | Model size | Gzipped | Time per check |
|---|---|---|---|
| 1.9M | 2.1 MB | 1.6 MB | About 3 ms in Chrome |
What it does
| You type | It becomes |
|---|---|
| I will send it form home tomorrow. | I will send it from home tomorrow. |
| Let me now if that works. | Let me know if that works. |
| We saw there new house. | We saw their new house. |
| Please fill out the form and return it. | No change |
How well it works
Measured on 2,460 real-word typos from GitHub commits and 45,000 correct words of email and documentation that it never trained on.
| Setting | Real-word typos fixed | Fixes that were right | Correct email words changed | Correct GitHub words changed |
|---|---|---|---|---|
| Careful | 19% | 94% | About 1 in 2,500 | About 1 in 1,700 |
| Eager | 31% | 94% | About 1 in 910 | About 1 in 430 |
Careful is the default. The demo runs Eager, so you see more fixes.
It also helps choose the right fix for misspelled words, raising that fix rate from 56% to 61.5% while 96.6% of fixes stay right.
What it misses
- It misses more real-word typos than it catches, including "don" typed for "do".
- It can only judge the 20,000 words it knows, so a rarer word like "ware" typed for "were" is left alone.
- It needs the words that come after, so these fixes land a couple of words later.
- English only.
Using the model
The model scores candidate words for one position in a sentence. Higher scores fit better, and comparing each candidate to the typed word tells you whether a swap is worth making.
| Name | Shape | What it holds |
|---|---|---|
left (input) |
[batch, 16] int64 |
The words before, left-padded with 0, starting with <s> when there are fewer than 16 |
right (input) |
[batch, 4] int64 |
The words after, nearest word last, left-padded with 0 |
candidates (input) |
[batch, 9] int64 |
The typed word first, then up to 8 nearby words, padded with 0 at the end |
scores (output) |
[batch, 9] float |
One score per candidate |
IDs are line numbers in wrong-word-vocab.txt, starting from 0. Words are lowercased, and capitalized words in the middle of a sentence become <name>.
Files
| File | What it is |
|---|---|
wrong-word.onnx |
The quantized model |
wrong-word-vocab.txt |
The 20,000 words it knows, one per line |
wrong-word-confusions.txt |
Mix-ups like "their" and "there" that are far apart on the keyboard |
train_realword.py |
The training script |
Training
A word-level GRU that reads 16 words before and up to 4 words after, trained for 25 minutes on an A10G. It learned from 118M tokens of FineWeb-Edu, SmolTalk and OpenAssistant, by swapping real words for nearby real words and learning to swap them back.