Blame for README.md

---
library_name: onnx
language: en
tags:
  - autocorrect
  - spelling-correction
  - browser
  - onnx
  - offline-page
datasets:
  - HuggingFaceFW/fineweb-edu
  - HuggingFaceTB/smoltalk
  - OpenAssistant/oasst1
---

<p align="center">
  <img src="banner.png" alt="tinyfix: autocorrect that reads both sides of a word" width="880">
</p>

<p align="center">
  <img style="display: inline-block; vertical-align: middle; margin: 2px;" alt="params: 1.9M" src="https://img.shields.io/badge/params-1.9M-7b1fe0?style=flat-square&labelColor=1d1d1f"> <img style="display: inline-block; vertical-align: middle; margin: 2px;" alt="size: 2.1 MB" src="https://img.shields.io/badge/size-2.1%20MB-7b1fe0?style=flat-square&labelColor=1d1d1f"> <img style="display: inline-block; vertical-align: middle; margin: 2px;" alt="speed: ~3 ms" src="https://img.shields.io/badge/speed-~3%20ms-7b1fe0?style=flat-square&labelColor=1d1d1f"> <img style="display: inline-block; vertical-align: middle; margin: 2px;" alt="runs: in browser" src="https://img.shields.io/badge/runs-in%20browser-7b1fe0?style=flat-square&labelColor=1d1d1f"> <img style="display: inline-block; vertical-align: middle; margin: 2px;" alt="data sent: none" src="https://img.shields.io/badge/data%20sent-none-7b1fe0?style=flat-square&labelColor=1d1d1f">
</p>

<h3 align="center">Spell check can't tell that “form” was meant to be “from”,<br>since both are real words. This 2.1 MB model can.</h3>

<p align="center">
  <a href="https://huggingface.co/spaces/cfahlgren1/tinyfix"><b>Try the demo&nbsp;→</b></a>
  &nbsp;&nbsp;·&nbsp;&nbsp;
  <a href="https://github.com/cfahlgren1/offline-page"><b>Write with it in offline.page&nbsp;→</b></a>
</p>

<br>

tinyfix reads the words on both sides of a correctly spelled word and picks the best fit from nearby real words. It runs on the page with [ONNX Runtime Web](https://onnxruntime.ai/docs/tutorials/web/), so nothing you type is sent anywhere.

## What it does

<p align="center">
  <img src="tinyfix.png" alt="The tinyfix demo correcting “I recieve the report tomorow and we saw there new house.” as it is typed" width="760">
</p>

| You type | It becomes |
|---|---|
| I will send it <del>form</del> home tomorrow. | I will send it **from** home tomorrow. |
| Let me <del>now</del> if that works. | Let me **know** if that works. |
| We saw <del>there</del> new house. | We saw **their** new house. |
| Please fill out the **form** and return it. | *No change. “form” is right here.* |

## How well it works

<table>
  <tr>
    <td align="center" width="33%"><h2>94%</h2>of its fixes are right</td>
    <td align="center" width="33%"><h2>31%</h2>of real-word typos fixed<br><sub>in Eager mode</sub></td>
    <td align="center" width="33%"><h2>~3 ms</h2>per check in Chrome</td>
  </tr>
</table>

Measured on 2,460 real-word typos from GitHub commits and 45,000 correct words of email and documentation that it never trained on.

| Setting | Real-word typos fixed | Fixes that were right | Correct email words changed | Correct GitHub words changed |
|---|---|---|---|---|
| Careful *(default)* | **19%** | **94%** | about 1 in 2,500 | about 1 in 1,700 |
| Eager *(demo and offline.page)* | **31%** | **94%** | about 1 in 910 | about 1 in 430 |

It also helps choose the right fix for misspelled words, raising that fix rate from 56% to 61.5% while 96.6% of fixes stay right.

### What it misses

- It misses more real-word typos than it catches, including “don” typed for “do”.
- It can only judge the 20,000 words it knows, so a rarer word like “ware” typed for “were” is left alone.
- It needs the words that come after, so these fixes land a couple of words later.
- English only.

## Using the model

The model scores candidate words for one position in a sentence. Higher scores fit better, and comparing each candidate to the typed word tells you whether a swap is worth making.

<details>
<summary><b>Inputs and outputs</b></summary>

| Name | Shape | What it holds |
|---|---|---|
| `left` (input) | `[batch, 16]` int64 | The words before, left-padded with 0, starting with `<s>` when there are fewer than 16 |
| `right` (input) | `[batch, 4]` int64 | The words after, nearest word last, left-padded with 0 |
| `candidates` (input) | `[batch, 9]` int64 | The typed word first, then up to 8 nearby words, padded with 0 at the end |
| `scores` (output) | `[batch, 9]` float | One score per candidate |

IDs are line numbers in `wrong-word-vocab.txt`, starting from 0. Words are lowercased, and capitalized words in the middle of a sentence become `<name>`.

</details>

<details>
<summary><b>Files</b></summary>

| File | What it is |
|---|---|
| `wrong-word.onnx` | The quantized model |
| `wrong-word-vocab.txt` | The 20,000 words it knows, one per line |
| `wrong-word-confusions.txt` | Mix-ups like “their” and “there” that are far apart on the keyboard |
| `train_realword.py` | The training script |

</details>

## Training

A word-level GRU that reads 16 words before and up to 4 words after, trained for 25 minutes on an A10G. It learned from 118M tokens of [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu), [SmolTalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) and [OpenAssistant](https://huggingface.co/datasets/OpenAssistant/oasst1), by swapping real words for nearby real words and learning to swap them back.

<br>

<p align="center">
  <sub>tinyfix is one of two tiny models behind <a href="https://github.com/cfahlgren1/offline-page">offline.page</a>, a writing app that works in airplane mode.<br>Its partner, <a href="https://huggingface.co/cfahlgren1/tinynext">tinynext</a>, suggests the next few words.</sub>
</p>