cfahlgren1 HF Staff commited on
Commit
81ef439
·
verified ·
1 Parent(s): b4c1c55

add model card

Browse files
Files changed (3) hide show
  1. .gitattributes +1 -0
  2. README.md +82 -0
  3. tinyfix.png +3 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tinyfix.png filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: onnx
3
+ language: en
4
+ tags:
5
+ - autocorrect
6
+ - spelling-correction
7
+ - browser
8
+ - onnx
9
+ datasets:
10
+ - HuggingFaceFW/fineweb-edu
11
+ - HuggingFaceTB/smoltalk
12
+ - OpenAssistant/oasst1
13
+ ---
14
+
15
+ # tinyfix
16
+
17
+ **Spell check can't tell that "form" was meant to be "from", since both are real words. This 2.1 MB model can.**
18
+
19
+ ![The tinyfix demo correcting "I recieve the report tomorow and we saw there new house." as it is typed](tinyfix.png)
20
+
21
+ **[Try it in your browser](https://huggingface.co/spaces/cfahlgren1/tinyfix)**
22
+
23
+ It reads the words on both sides of a correctly spelled word and picks the best fit from nearby real words. It runs on the page with [ONNX Runtime Web](https://onnxruntime.ai/docs/tutorials/web/), so nothing you type is sent anywhere.
24
+
25
+ | Parameters | Model size | Gzipped | Time per check |
26
+ |---|---|---|---|
27
+ | **1.9M** | **2.1 MB** | **1.6 MB** | **About 3 ms** in Chrome |
28
+
29
+ ## What it does
30
+
31
+ | You type | It becomes |
32
+ |---|---|
33
+ | I will send it **form** home tomorrow. | I will send it **from** home tomorrow. |
34
+ | Let me **now** if that works. | Let me **know** if that works. |
35
+ | We saw **there** new house. | We saw **their** new house. |
36
+ | Please fill out the **form** and return it. | No change |
37
+
38
+ ## How well it works
39
+
40
+ Measured on 2,460 real-word typos from GitHub commits and 45,000 correct words of email and documentation that it never trained on.
41
+
42
+ | Setting | Real-word typos fixed | Fixes that were right | Correct email words changed | Correct GitHub words changed |
43
+ |---|---|---|---|---|
44
+ | Careful | **19%** | **94%** | About 1 in 2,500 | About 1 in 1,700 |
45
+ | Eager | **31%** | **94%** | About 1 in 910 | About 1 in 430 |
46
+
47
+ Careful is the default. The demo runs Eager, so you see more fixes.
48
+
49
+ It also helps choose the right fix for misspelled words, raising that fix rate from 56% to 61.5% while 96.6% of fixes stay right.
50
+
51
+ ### What it misses
52
+
53
+ - It misses more real-word typos than it catches, including "don" typed for "do".
54
+ - It can only judge the 20,000 words it knows, so a rarer word like "ware" typed for "were" is left alone.
55
+ - It needs the words that come after, so these fixes land a couple of words later.
56
+ - English only.
57
+
58
+ ## Using the model
59
+
60
+ The model scores candidate words for one position in a sentence. Higher scores fit better, and comparing each candidate to the typed word tells you whether a swap is worth making.
61
+
62
+ | Name | Shape | What it holds |
63
+ |---|---|---|
64
+ | `left` (input) | `[batch, 16]` int64 | The words before, left-padded with 0, starting with `<s>` when there are fewer than 16 |
65
+ | `right` (input) | `[batch, 4]` int64 | The words after, nearest word last, left-padded with 0 |
66
+ | `candidates` (input) | `[batch, 9]` int64 | The typed word first, then up to 8 nearby words, padded with 0 at the end |
67
+ | `scores` (output) | `[batch, 9]` float | One score per candidate |
68
+
69
+ IDs are line numbers in `wrong-word-vocab.txt`, starting from 0. Words are lowercased, and capitalized words in the middle of a sentence become `<name>`.
70
+
71
+ ## Files
72
+
73
+ | File | What it is |
74
+ |---|---|
75
+ | `wrong-word.onnx` | The quantized model |
76
+ | `wrong-word-vocab.txt` | The 20,000 words it knows, one per line |
77
+ | `wrong-word-confusions.txt` | Mix-ups like "their" and "there" that are far apart on the keyboard |
78
+ | `train_realword.py` | The training script |
79
+
80
+ ## Training
81
+
82
+ A word-level GRU that reads 16 words before and up to 4 words after, trained for 25 minutes on an A10G. It learned from 118M tokens of [FineWeb-Edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu), [SmolTalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) and [OpenAssistant](https://huggingface.co/datasets/OpenAssistant/oasst1), by swapping real words for nearby real words and learning to swap them back.
tinyfix.png ADDED

Git LFS Details

  • SHA256: 5b2e3d32fcb2d32cdc9fda987ef983503d03ce078fb6157727f7d4b8fe049d8c
  • Pointer size: 131 Bytes
  • Size of remote file: 180 kB