| --- |
| library_name: onnx |
| language: en |
| tags: |
| - autocorrect |
| - spelling-correction |
| - browser |
| - onnx |
| - offline-page |
| datasets: |
| - HuggingFaceFW/fineweb-edu |
| - HuggingFaceTB/smoltalk |
| - OpenAssistant/oasst1 |
| --- |
| |
| <p align="center"> |
| <img src="banner.png" alt="tinyfix: autocorrect that reads both sides of a word" width="880"> |
| </p> |
|
|
| <p align="center"> |
| <img style="display: inline-block; vertical-align: middle; margin: 2px;" alt="params: 1.9M" src="https://img.shields.io/badge/params-1.9M-7b1fe0?style=flat-square&labelColor=1d1d1f"> <img style="display: inline-block; vertical-align: middle; margin: 2px;" alt="size: 2.1 MB" src="https://img.shields.io/badge/size-2.1%20MB-7b1fe0?style=flat-square&labelColor=1d1d1f"> <img style="display: inline-block; vertical-align: middle; margin: 2px;" alt="speed: ~3 ms" src="https://img.shields.io/badge/speed-~3%20ms-7b1fe0?style=flat-square&labelColor=1d1d1f"> <img style="display: inline-block; vertical-align: middle; margin: 2px;" alt="runs: in browser" src="https://img.shields.io/badge/runs-in%20browser-7b1fe0?style=flat-square&labelColor=1d1d1f"> <img style="display: inline-block; vertical-align: middle; margin: 2px;" alt="data sent: none" src="https://img.shields.io/badge/data%20sent-none-7b1fe0?style=flat-square&labelColor=1d1d1f"> |
| </p> |
|
|
| <h3 align="center">Spell check can't tell that “form” was meant to be “from”,<br>since both are real words. This 2.1 MB model can.</h3> |
|
|
| <p align="center"> |
| <a href="https://huggingface.co/spaces/cfahlgren1/tinyfix"><b>Try the demo →</b></a> |
| · |
| <a href="https://github.com/cfahlgren1/offline-page"><b>Write with it in offline.page →</b></a> |
| </p> |
|
|
| <br> |
|
|
| tinyfix reads the words on both sides of a correctly spelled word and picks the best fit from nearby real words. It runs on the page with [ONNX Runtime Web](https://onnxruntime.ai/docs/tutorials/web/), so nothing you type is sent anywhere. |
|
|
| ## What it does |
|
|
| <p align="center"> |
| <img src="tinyfix.png" alt="The tinyfix demo correcting “I recieve the report tomorow and we saw there new house.” as it is typed" width="760"> |
| </p> |
|
|
| | You type | It becomes | |
| |---|---| |
| | I will send it <del>form</del> home tomorrow. | I will send it **from** home tomorrow. | |
| | Let me <del>now</del> if that works. | Let me **know** if that works. | |
| | We saw <del>there</del> new house. | We saw **their** new house. | |
| | Please fill out the **form** and return it. | *No change. “form” is right here.* | |
|
|
| ## How well it works |
|
|
| <table> |
| <tr> |
| <td align="center" width="33%"><h2>94%</h2>of its fixes are right</td> |
| <td align="center" width="33%"><h2>31%</h2>of real-word typos fixed<br><sub>in Eager mode</sub></td> |
| <td align="center" width="33%"><h2>~3 ms</h2>per check in Chrome</td> |
| </tr> |
| </table> |
| |
| Measured on 2,460 real-word typos from GitHub commits and 45,000 correct words of email and documentation that it never trained on. |
|
|
| | Setting | Real-word typos fixed | Fixes that were right | Correct email words changed | Correct GitHub words changed | |
| |---|---|---|---|---| |
| | Careful *(default)* | **19%** | **94%** | about 1 in 2,500 | about 1 in 1,700 | |
| | Eager *(demo and offline.page)* | **31%** | **94%** | about 1 in 910 | about 1 in 430 | |
|
|
| It also helps choose the right fix for misspelled words, raising that fix rate from 56% to 61.5% while 96.6% of fixes stay right. |
|
|
| ### What it misses |
|
|
| - It misses more real-word typos than it catches, including “don” typed for “do”. |
| - It can only judge the 20,000 words it knows, so a rarer word like “ware” typed for “were” is left alone. |
| - It needs the words that come after, so these fixes land a couple of words later. |
| - English only. |
|
|
| ## Using the model |
|
|
| The model scores candidate words for one position in a sentence. Higher scores fit better, and comparing each candidate to the typed word tells you whether a swap is worth making. |
|
|
| <details> |
| <summary><b>Inputs and outputs</b></summary> |
|
|
| | Name | Shape | What it holds | |
| |---|---|---| |
| | `left` (input) | `[batch, 16]` int64 | The words before, left-padded with 0, starting with `<s>` when there are fewer than 16 | |
| | `right` (input) | `[batch, 4]` int64 | The words after, nearest word last, left-padded with 0 | |
| | `candidates` (input) | `[batch, 9]` int64 | The typed word first, then up to 8 nearby words, padded with 0 at the end | |
| | `scores` (output) | `[batch, 9]` float | One score per candidate | |
|
|
| IDs are line numbers in `wrong-word-vocab.txt`, starting from 0. Words are lowercased, and capitalized words in the middle of a sentence become `<name>`. |
|
|
| </details> |
|
|
| <details> |
| <summary><b>Files</b></summary> |
|
|
| | File | What it is | |
| |---|---| |
| | `wrong-word.onnx` | The quantized model | |
| | `wrong-word-vocab.txt` | The 20,000 words it knows, one per line | |
| | `wrong-word-confusions.txt` | Mix-ups like “their” and “there” that are far apart on the keyboard | |
| | `train_realword.py` | The training script | |
|
|
| </details> |
|
|
| ## Training |
|
|
| A word-level GRU that reads 16 words before and up to 4 words after, trained for 25 minutes on an A10G. It learned from 118M tokens of [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu), [SmolTalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) and [OpenAssistant](https://huggingface.co/datasets/OpenAssistant/oasst1), by swapping real words for nearby real words and learning to swap them back. |
|
|
| <br> |
|
|
| <p align="center"> |
| <sub>tinyfix is one of two tiny models behind <a href="https://github.com/cfahlgren1/offline-page">offline.page</a>, a writing app that works in airplane mode.<br>Its partner, <a href="https://huggingface.co/cfahlgren1/tinynext">tinynext</a>, suggests the next few words.</sub> |
| </p> |
|
|
