Imperatore commited on
Commit
8e3b50b
·
verified ·
1 Parent(s): 9008c4d

2.1.0: run 2026-11-09-0626 (gold 182/104/6/0, Tatoeba 0.43%) + guard: a noun followed by its own verb is not a doubled word

Browse files
Files changed (5) hide show
  1. README.md +15 -11
  2. config.json +0 -0
  3. gector-guards.json +2 -1
  4. gector-meta.json +5 -5
  5. model.onnx +1 -1
README.md CHANGED
@@ -22,7 +22,7 @@ datasets:
22
 
23
  **Try it in your browser:** type a sentence and see the corrections. The model runs on your own computer; nothing to install, and your text never leaves the page.
24
 
25
- **Version 2.0.0** · run `2026-11-08-1901` (1.x = run `2026-11-08-0749`) · tag [`v2.0.0`](https://huggingface.co/proofbird/gector-bert-de/tree/v2.0.0), the files the Proofbird app downloads. Pin it with `revision="v2.0.0"`. Versions: a new training run raises the first number, changed guards or rules the second, card or metadata fixes the third.
26
 
27
  A GECToR-style sequence tagger for German grammatical error correction, exported to ONNX (fp32) to run on the CPU.
28
  It is the on-device grammar model of [Proofbird](https://proofbird.app), a writing assistant for Windows, and it is
@@ -81,28 +81,32 @@ words, and capitalisation at the start of a sentence.
81
  - **Data: synthetic only.** 3 000 000 sentences from German Wikipedia
82
  ([`wikimedia/wikipedia`](https://huggingface.co/datasets/wikimedia/wikipedia), `20231101.de`) and 250 000 sentences
83
  from [Tatoeba](https://tatoeba.org) were corrupted by hand-written German error rules; about 10% stay clean, so the
84
- model learns to leave correct text alone. 45 000 additional template sentences teach patterns Wikipedia rarely
85
  contains (`du` and `ihr` with their verb forms, `das` / `dass` with the same pronouns on both sides, tag questions,
86
- errors in a later clause), with correct twins so the model also learns what to leave alone.
 
87
  - **No learner corpora** and no non-commercial data of any kind. No user text was used.
88
- - One epoch (95 685 steps, batch 64, learning rate 3e-5) on one NVIDIA L4. Run id `2026-11-08-1901`.
89
 
90
  ## How it was measured
91
 
92
  | Check | Result |
93
  |---|---|
94
- | Gold set (278 hand-written sentences: 182 with one error each, 96 correct) | **178 fixed, 96 left alone, 8 missed, 0 made worse** |
95
- | False alarms on 4 000 everyday Tatoeba sentences (held out from training) | 17 changed (0.43%); 13 of them correct an error or an old spelling in Tatoeba's own text |
96
- | Stability: the gold re-run with one extra token in front of each sentence | 3 sentences change result |
97
 
98
  The gold score is with the apply-side guards in `gector-guards.json` (below). The model was trained for precision: a
99
  missed error costs less than a wrong correction. The Tatoeba changes include real fixes of Tatoeba's text (`muß` →
100
- `muss`, `Du wird` → `wirst`, `Unsere Lehrer ist` → `sind`) and four false alarms, for example `den Talern` → `Tälern`
101
- and `Können Sie mir das buchstabieren?` → `Buchstabieren`. Known misses: `nicht war` and `schon` → `schön` at the end of a
102
- long sentence (short ones are fixed), and `hätte` → `hatte`, which the model leaves alone because both are often right.
 
 
103
  The guards keep `man` with the Konjunktiv I of instructions (`Man nehme zwei Eier`), `das das` after a form of `sein`
104
  (`Ist das das Haus von Anna?`), a relative pronoun and an article after a comma (`diejenige, die die Entscheidungen
105
- trifft`) and a plural after `die`, `meine` and similar words (`Ich stelle hier die Fragen`). A sentence written
 
106
  throughout with `ss` instead of `ß` (Swiss spelling) is left alone. It has not been measured on learner essays.
107
 
108
  ## Files
 
22
 
23
  **Try it in your browser:** type a sentence and see the corrections. The model runs on your own computer; nothing to install, and your text never leaves the page.
24
 
25
+ **Version 2.1.0** · run `2026-11-09-0626` (2.0.0 = run `2026-11-08-1901`, 1.x = run `2026-11-08-0749`) · tag [`v2.1.0`](https://huggingface.co/proofbird/gector-bert-de/tree/v2.1.0), the files the Proofbird app downloads. Pin it with `revision="v2.1.0"`. Versions: a change to the model's design (encoder, inputs, outputs) raises the first number, a new training run or changed guards and rules the second, card or metadata fixes the third.
26
 
27
  A GECToR-style sequence tagger for German grammatical error correction, exported to ONNX (fp32) to run on the CPU.
28
  It is the on-device grammar model of [Proofbird](https://proofbird.app), a writing assistant for Windows, and it is
 
81
  - **Data: synthetic only.** 3 000 000 sentences from German Wikipedia
82
  ([`wikimedia/wikipedia`](https://huggingface.co/datasets/wikimedia/wikipedia), `20231101.de`) and 250 000 sentences
83
  from [Tatoeba](https://tatoeba.org) were corrupted by hand-written German error rules; about 10% stay clean, so the
84
+ model learns to leave correct text alone. 50 000 additional template sentences teach patterns Wikipedia rarely
85
  contains (`du` and `ihr` with their verb forms, `das` / `dass` with the same pronouns on both sides, tag questions,
86
+ errors in a later clause, and the same errors inside long sentences), with correct twins so the model also learns
87
+ what to leave alone.
88
  - **No learner corpora** and no non-commercial data of any kind. No user text was used.
89
+ - One epoch (95 794 steps, batch 64, learning rate 3e-5) on one NVIDIA L4. Run id `2026-11-09-0626`.
90
 
91
  ## How it was measured
92
 
93
  | Check | Result |
94
  |---|---|
95
+ | Gold set (292 hand-written sentences: 188 with one error each, 104 correct) | **182 fixed, 104 left alone, 6 missed, 0 made worse** |
96
+ | False alarms on 4 000 everyday Tatoeba sentences (held out from training) | 17 changed (0.43%); 14 of them correct an error or an old spelling in Tatoeba's own text |
97
+ | Stability: the gold re-run with one extra token in front of each sentence | 1 sentence changes result |
98
 
99
  The gold score is with the apply-side guards in `gector-guards.json` (below). The model was trained for precision: a
100
  missed error costs less than a wrong correction. The Tatoeba changes include real fixes of Tatoeba's text (`muß` →
101
+ `muss`, `Du wird` → `wirst`, `Unsere Lehrer ist` → `sind`) and three false alarms, for example `den Talern` → `Tälern`
102
+ and `der jüngere der Brüder` → `Jüngere`. Since 2.0.0 it also fixes `nicht war` and `ß` inside long sentences
103
+ (`Er ist schon seit gestern krank, nicht war?`). Known misses: `schon` → `schön` without a word like `sehr` or `wirklich`
104
+ before it (`Die Blumen sind trotzdem schon.`; `Die Blumen sind wirklich schon.` is fixed), and `hätte` → `hatte`, which
105
+ the model leaves alone because both are often right.
106
  The guards keep `man` with the Konjunktiv I of instructions (`Man nehme zwei Eier`), `das das` after a form of `sein`
107
  (`Ist das das Haus von Anna?`), a relative pronoun and an article after a comma (`diejenige, die die Entscheidungen
108
+ trifft`), a noun followed by its own verb (`Wir müssen das Lernen lernen`) and a plural after `die`, `meine` and similar
109
+ words (`Ich stelle hier die Fragen`). A sentence written
110
  throughout with `ss` instead of `ß` (Swiss spelling) is left alone. It has not been measured on learner essays.
111
 
112
  ## Files
config.json CHANGED
The diff for this file is too large to render. See raw diff
 
gector-guards.json CHANGED
@@ -45,7 +45,8 @@
45
  "type": "delete_only_duplicates",
46
  "words": "das|die|der|den|dem",
47
  "whenPrevIs": "ist|war|wäre|sei|wird|bleibt|,",
48
- "note": "a $DELETE is honoured only on a repeated word ('ist ist'); never on a real word ('wirklich echt?'); 'das das' after a form of sein is a demonstrative + an article (Ist das das Haus?) and stays (DE 1.1.0); so does a relative pronoun + article after a comma (diejenige, die die Entscheidungen trifft, DE 2.0.0) - Mach bitte die die Tür zu still loses one"
 
49
  },
50
  {
51
  "type": "redirect_replace",
 
45
  "type": "delete_only_duplicates",
46
  "words": "das|die|der|den|dem",
47
  "whenPrevIs": "ist|war|wäre|sei|wird|bleibt|,",
48
+ "keepCaseDiffering": true,
49
+ "note": "a $DELETE is honoured only on a repeated word ('ist ist'); never on a real word ('wirklich echt?'); 'das das' after a form of sein is a demonstrative + an article (Ist das das Haus?) and stays (DE 1.1.0); so does a relative pronoun + article after a comma (diejenige, die die Entscheidungen trifft, DE 2.0.0) - Mach bitte die die Tür zu still loses one; a pair differing only in case mid-sentence is a noun + its verb (Wir müssen das Lernen lernen, DE 3.0.0)"
50
  },
51
  {
52
  "type": "redirect_replace",
gector-meta.json CHANGED
@@ -18,20 +18,20 @@
18
  "min_error_prob": 0.0,
19
  "n_iteration": 5
20
  },
21
- "run_id": "2026-11-08-1901",
22
- "exported_at": "2026-11-08T23:06:29+00:00",
23
  "checkpoint": "ckpt_best.pt",
24
  "git_commit": null,
25
  "recipe_sha256": {
26
  "build_dataset.py": "311ab76557eb",
27
  "preprocess.py": "384e6c617e5e",
28
  "train.py": "489238d21b2c",
29
- "export_onnx.py": "d43608f9b66c",
30
  "runrecord.py": "718118cd4cb0",
31
  "tatoeba_split.py": "4183ce16b93d",
32
- "corrupt_de.py": "109c0a7bc2fb",
33
  "morph_de.py": "a9361cce0af0",
34
  "language-rules.de.json": "901b350a9c75"
35
  },
36
- "version": "2.0.0"
37
  }
 
18
  "min_error_prob": 0.0,
19
  "n_iteration": 5
20
  },
21
+ "run_id": "2026-11-09-0626",
22
+ "exported_at": "2026-11-09T10:33:32+00:00",
23
  "checkpoint": "ckpt_best.pt",
24
  "git_commit": null,
25
  "recipe_sha256": {
26
  "build_dataset.py": "311ab76557eb",
27
  "preprocess.py": "384e6c617e5e",
28
  "train.py": "489238d21b2c",
29
+ "export_onnx.py": "51b98fd3d7a6",
30
  "runrecord.py": "718118cd4cb0",
31
  "tatoeba_split.py": "4183ce16b93d",
32
+ "corrupt_de.py": "9ab57aed5ce8",
33
  "morph_de.py": "a9361cce0af0",
34
  "language-rules.de.json": "901b350a9c75"
35
  },
36
+ "version": "2.1.0"
37
  }
model.onnx CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5fda9f98285ef18aba99ae639eb4f1a3a3b95a23720f43972d19d4f67b61b5a9
3
  size 452972247
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6d7b43b4bd95c7a00c02c8babc7569d28558df894afe4874d96c202e6ab9a464
3
  size 452972247