Titlehhhh commited on
Commit
37556a8
·
verified ·
1 Parent(s): 27eb6fe

model card: describe universal_v1, not the previous all20 run

Browse files
Files changed (1) hide show
  1. README.md +287 -93
README.md CHANGED
@@ -11,6 +11,7 @@ tags:
11
  - minecraft
12
  - pp-ocrv6
13
  - text-recognition
 
14
  datasets:
15
  - Titlehhhh/captchaocr-dataset
16
  base_model:
@@ -19,96 +20,205 @@ base_model:
19
 
20
  # CaptchaOCR PP-OCRv6 — a text recogniser for Minecraft anti-bot captchas
21
 
22
- A fine-tune of **PP-OCRv6_medium_rec** on 15 captcha types of the
23
- [`Titlehhhh/captchaocr-dataset`](https://huggingface.co/datasets/Titlehhhh/captchaocr-dataset) dataset.
24
- Feed it one captcha image, get the string drawn on it.
25
 
26
- ## What it reads
 
 
 
27
 
28
- Minecraft anti-bot plugins show a player a picture with a code and ask them to retype it.
29
- Every plugin draws differently: its own font, canvas size, noise, distortion — and its own bugs.
30
- The 20 types in the dataset are flat captcha images; this model was trained on the **15** of them
31
- whose alphabet fits the PP-OCRv6 dictionary.
32
 
33
- Trained on:
34
- `Archy-X/Krypton/map`, `InstantlyMoist/Captcha/textcaptcha`, `JSH32/Minecaptcha/map`,
35
- `LagBug/XProtect/code`, `LagBug/XProtect/map`, `haq/mapcha/map`, `irootdev/AnBot/captcha`,
36
- `jonesdevelopment/sonar/{standard,legacy,complex}`, `just-captcha/ProstoCaptcha/map`,
37
- `ringoame196-s-mcPlugin/ReproductionCAPTCHA/map`, `BotFilter/map`, `CaptchaVerifyZ/map`,
38
- `GalleryCaptcha/ru-web`.
 
 
 
 
 
39
 
40
- **Not supported:** `GalleryCaptcha/{ru-numberword, ru-numberword-caption, ru-symbols}` and
41
- `LimboFilter/{map, symbols}` — these are Cyrillic, and the PP-OCRv6 dictionary contains no
42
- Cyrillic letters at all.
 
 
 
 
 
 
 
 
43
 
44
- ## What this model is
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
 
46
  | | |
47
  |---|---|
48
- | base | `PaddlePaddle/PP-OCRv6_medium_rec_safetensors` (Apache-2.0) |
49
- | parameters | 19 176 118 |
50
- | classes | 18 710 (index 0 = blank), the pretrained head is kept, not replaced |
51
- | fine-tune | 15 types × 800 images, 5 epochs, 1 875 steps, batch 32, lr 1e-5, fp16 |
52
- | best checkpoint | epoch 2 |
53
- | weight interpolation | **none** (α = 1.0) — see "Measured results" |
 
 
 
 
54
 
55
- ## Measured results
56
 
57
- Measured on a **fresh generator run**: 20 types × 200 = 4 000 images that were never used for
58
- training or for choosing the epoch. 15 types were scored (the 5 Cyrillic ones are outside the
59
- dictionary), 3 000 images in total. Crop by the text band when the type has one, full frame
60
- otherwise; greedy CTC decode; exact match on `text` (letters/digits of any script plus `+-*/=`).
61
 
62
- | aggregate | value |
63
  |---|---|
64
- | **median exact match over 15 types** | **0.920** |
65
- | median CER | 0.021 |
66
- | micro exact match (3 000 images) | 0.745 |
67
- | micro CER | 0.098 |
68
-
69
- | type | n | exact | CER |
70
- |---|---|---|---|
71
- | `CaptchaVerifyZ/map` | 200 | **1.000** | 0.000 |
72
- | `ringoame196-s-mcPlugin/ReproductionCAPTCHA/map` | 200 | 0.995 | 0.001 |
73
- | `irootdev/AnBot/captcha` | 200 | 0.985 | 0.005 |
74
- | `Archy-X/Krypton/map` | 200 | 0.980 | 0.004 |
75
- | `GalleryCaptcha/ru-web` | 200 | 0.945 | 0.014 |
76
- | `haq/mapcha/map` | 200 | 0.945 | 0.014 |
77
- | `BotFilter/map` | 200 | 0.935 | 0.014 |
78
- | `JSH32/Minecaptcha/map` | 200 | 0.920 | 0.021 |
79
- | `jonesdevelopment/sonar/legacy` | 200 | 0.830 | 0.052 |
80
- | `jonesdevelopment/sonar/complex` | 200 | 0.805 | 0.060 |
81
- | `just-captcha/ProstoCaptcha/map` | 200 | 0.785 | 0.044 |
82
- | `jonesdevelopment/sonar/standard` | 200 | 0.695 | 0.101 |
83
- | `InstantlyMoist/Captcha/textcaptcha` | 200 | 0.220 | 0.438 |
84
- | `LagBug/XProtect/code` | 200 | 0.065 | 0.351 |
85
- | `LagBug/XProtect/map` | 200 | 0.065 | 0.343 |
86
-
87
- **Known weak types.** `LagBug/XProtect/code` and `/map` are barely read (6.5 %): they are the same
88
- 130×130 sheet with per-character cumulative translation and 51 interference lines. `InstantlyMoist`
89
- (22 %) is a 128×128 sheet with thin diagonal noise. Neither is a decoding problem — the model simply
90
- never learned these two drawing styles well enough at 800 images each.
91
-
92
- ### Why there is no weight interpolation here
93
-
94
- Weight interpolation towards the base (WiSE-FT) was what made the **previous** version of this
95
- repository useful: a fine-tune on a few types degraded unseen plugins, so α = 0.4 was published.
96
- This model is trained on most of the types it is evaluated on, so interpolation now only drags it
97
- back towards the base. Measured on 6 held-out types, 1 200 images (200 per type):
98
-
99
- | α | median exact | CER |
100
  |---|---|---|
101
- | 0.0 (base) | 0.600 | 0.164 |
102
- | 0.4 | 0.738 | 0.089 |
103
- | 0.8 | 0.858 | 0.046 |
104
- | **1.0 (published)** | **0.883** | 0.035 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
105
 
106
- ### What it is NOT good at
 
 
 
107
 
108
- A different plugin that draws in a style not present in the dataset will not be handled: this is a
109
- fine-tune for a **known set of captcha types with new images**, not a general captcha solver.
110
- Measured generalization to unseen plugins for a comparable fine-tune sits around 0.4–0.6 median
111
- exact match, and that is the honest expectation.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
112
 
113
  ## How to use it
114
 
@@ -117,34 +227,118 @@ import torch
117
  from PIL import Image
118
  from transformers import PPOCRV6SmallRecForTextRecognition, PPOCRV6SmallRecImageProcessor
119
 
120
- model = PPOCRV6SmallRecForTextRecognition.from_pretrained("Titlehhhh/captchaocr-ppocrv6")
121
- proc = PPOCRV6SmallRecImageProcessor.from_pretrained("Titlehhhh/captchaocr-ppocrv6")
 
 
 
 
 
122
 
123
- image = Image.open("001.png").convert("RGB")
124
- px = proc(images=[image], return_tensors="pt")["pixel_values"]
125
  with torch.no_grad():
126
  logits = model.head.head(model.head.encoder(
127
  model.model(px).last_hidden_state).last_hidden_state)
128
- # greedy CTC decode against character_list.txt
 
 
 
 
 
 
 
 
 
 
 
129
  ```
130
 
131
- The dictionary ships with the repository: `character_list.txt` (18 710 entries, index 0 = blank,
132
- the same list as in `preprocessor_config.json`). The images are normalised with mean = std = 0.5
133
- (PaddleOCR convention), which is already patched into `preprocessor_config.json`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
134
 
135
- ## Provenance and licence
136
 
137
- Base model: `PaddlePaddle/PP-OCRv6_medium_rec_safetensors`, Apache-2.0.
138
- The dataset is synthetic: the images are re-renders of captchas drawn by open-source Minecraft
139
- plugins, and each type carries the licence of its upstream (see the dataset card).
140
- This repository's weights are a derivative of the Apache-2.0 base.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
141
 
142
  ## Reproducing
143
 
144
  ```powershell
145
- .venv-gpu\Scripts\python.exe train\finetune_ppocr.py --out train\runs\ppocrv6\all20 ^
146
- --limit-per-type 1000 --holdout-per-type 200 --train-frames --val-types ^
147
- --epochs 5 --max-minutes 16 --eval-every 1 --batch-size 32 --lr 1e-5 --head-lr 1e-5 --workers 6
148
- .venv-gpu\Scripts\python.exe train\export_ppocr_hf.py ^
149
- --ckpt train\runs\ppocrv6\all20\best.pt --soup-alpha 1.0 --out train\runs\ppocrv6\all20\export_alpha1
 
 
 
 
 
 
 
 
 
 
150
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  - minecraft
12
  - pp-ocrv6
13
  - text-recognition
14
+ - cyrillic
15
  datasets:
16
  - Titlehhhh/captchaocr-dataset
17
  base_model:
 
20
 
21
  # CaptchaOCR PP-OCRv6 — a text recogniser for Minecraft anti-bot captchas
22
 
23
+ A fine-tune of **PP-OCRv6_medium_rec** on Minecraft anti-bot captchas whose CTC head was
24
+ **extended to 18 762 classes**, so that it reads Cyrillic as well as Latin letters and digits.
25
+ Feed it **one crop of the answer band**, get the string drawn on it.
26
 
27
+ > **This card describes the `universal_v1` checkpoint — the weights that are actually in this
28
+ > repository.** The card that was here before described an older run (`all20`) whose weights are
29
+ > **no longer** published: 18 710 classes, 15 types, median exact 0.920. Those numbers were true
30
+ > of that model and are **not** true of this one.
31
 
32
+ ## What is in this repository
 
 
 
33
 
34
+ | file | what it is |
35
+ |---|---|
36
+ | `model.safetensors` | the weights: 308 tensors, 19 186 154 float parameters |
37
+ | `config.json` | the base architecture, with `head_out_channels` patched to 18 762 |
38
+ | `preprocessor_config.json` | the base processor + the PaddleOCR normalisation + the 18 762-character dictionary |
39
+ | `processor_config.json` | `PPOCRV6SmallRecImageProcessor` |
40
+ | `character_list.txt` | the dictionary, one entry per line, index 0 = `blank` |
41
+ | `finetune_meta.json` | the provenance of the export (checkpoint, epoch, step, α) |
42
+ | `README.md` | this card |
43
+
44
+ Provenance, as shipped in `finetune_meta.json`:
45
 
46
+ | | |
47
+ |---|---|
48
+ | base model | `PaddlePaddle/PP-OCRv6_medium_rec_safetensors` (Apache-2.0) |
49
+ | fine-tuned checkpoint | `train/runs/ppocrv6/universal_v1/best.pt` |
50
+ | epoch / step | 4 (the last of 5) / 2 440 |
51
+ | classes | **18 762** (index 0 = `blank`) — the pretrained head is **kept and extended**, not replaced |
52
+ | float parameters | **19 186 154** |
53
+ | weight interpolation | **none** — α = 1.0, the checkpoint exactly (see below) |
54
+ | transformers class | `PPOCRV6SmallRecForTextRecognition` |
55
+ | normalisation | mean = std = 0.5 (the PaddleOCR convention) |
56
+ | exported by | `train/export_ppocr_hf.py` |
57
 
58
+ ### What changed in the head
59
+
60
+ The pretrained PP-OCRv6 dictionary has **18 710** classes and no Cyrillic letters at all. Rows
61
+ **0 … 18 709** of the published dictionary are the pretrained ones (index 18 709 is the space);
62
+ rows **18 710 … 18 761** are the **52 Cyrillic characters** that were appended:
63
+
64
+ ```
65
+ ЁАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯ + авдеикмнопрстцчшыья
66
+ ```
67
+
68
+ The pretrained rows were **kept**, not replaced (`--expand-charset` grows the head and copies the
69
+ old rows). The extension was done in an earlier run of the same recipe; in this run the flag
70
+ reported «0 characters missing», i.e. the grown head was inherited from the previous checkpoint
71
+ together with its weights.
72
+
73
+ ## The training run in numbers
74
 
75
  | | |
76
  |---|---|
77
+ | initialised from | `train/runs/ppocrv6/limbo_cyr3/best.pt` — the **«replay mix»** Cyrillic checkpoint (3 short Cyrillic types + digits + 5 Latin types, 4 epochs) |
78
+ | epochs × steps | 5 × 488 = **2 440** steps |
79
+ | batch size | 32 |
80
+ | learning rate | **8e-6** for the backbone **and** 8e-6 for the head |
81
+ | weight decay / grad clip | 0.01 / 1.0 |
82
+ | `--limit-per-type` | **1 500** |
83
+ | training examples | **15 594** |
84
+ | evaluation set | **11 400** images over 18 types |
85
+ | wall clock | **33.9 min** (2 032 s) at **108 img/s** (one GTX 1080 Ti, fp16/AMP) |
86
+ | seed | 42 · `--workers 0` (the sandbox this was trained in denies multiprocessing named pipes) |
87
 
88
+ ### What was trained on, and what was not
89
 
90
+ **12 types were trained on.** Eight of them are fed as the **answer band** (the `roi` rectangle
91
+ of the dataset); the four that have no band at all are trained on the **whole frame**, so the
92
+ model has to find the code by itself:
 
93
 
94
+ | type | input |
95
  |---|---|
96
+ | `out/Archy-X/Krypton/map` | band |
97
+ | `out/haq/mapcha/map` | band |
98
+ | `out/just-captcha/ProstoCaptcha/map` | band |
99
+ | `out/ringoame196-s-mcPlugin/ReproductionCAPTCHA/map` | band |
100
+ | `out_ru/GalleryCaptcha/ru-symbols` | band |
101
+ | `out_ru/GalleryCaptcha/ru-web` | band |
102
+ | `out_ru/LimboFilter/digits` | band |
103
+ | `out_ru/LimboFilter/symbols` | band |
104
+ | `out/InstantlyMoist/Captcha/textcaptcha` | **whole frame** |
105
+ | `out/LagBug/XProtect/code` | **whole frame** |
106
+ | `out/LagBug/XProtect/map` | **whole frame** |
107
+ | `out/irootdev/AnBot/captcha` | **whole frame** |
108
+
109
+ (The names above are the dataset's own `type_id` values.)
110
+
111
+ **6 types were never seen in training.** The unit of generalisation is the **plugin**, never the
112
+ picture: these six whole plugins were held out of the training set:
113
+
114
+ `out/JSH32/Minecaptcha/map`, `out/jonesdevelopment/sonar/standard`,
115
+ `out/jonesdevelopment/sonar/legacy`, `out/jonesdevelopment/sonar/complex`,
116
+ `out_ru/BotFilter/map`, `out_ru/CaptchaVerifyZ/map`.
117
+
118
+ **3 types were excluded on purpose:** `out_ru/GalleryCaptcha/ru-numberword`,
119
+ `out_ru/GalleryCaptcha/ru-numberword-caption` and `out_ru/LimboFilter/map`. In all three the picture shows a
120
+ Russian number **written out in words** while the player types **digits** (`text != answer` in
121
+ all 10 000 rows of each). Converting a written number into a typed one is a **solver step**,
122
+ not an OCR task, so the model was not asked to do it. That is a limitation of the *task*, not a
123
+ defect of the model — but it does mean these three types are **not supported**.
124
+
125
+ ## Measured results
126
+
127
+ Measured inside the same run (`train/runs/ppocrv6/universal_v1/result.json`): 11 400 images over
128
+ 18 types, greedy CTC decoding, exact match against the drawn string.
129
+
130
+ | aggregate | before this run¹ | **after** |
 
131
  |---|---|---|
132
+ | median exact match over 18 types | 0.8750 | **0.8875** |
133
+ | median CER | 0.0394 | **0.0344** |
134
+ | micro exact match (11 400 images) | 0.6499 | **0.7797** |
135
+ | micro CER | 0.1795 | **0.0755** |
136
+
137
+ ¹ «before this run» is the **zero step of this very run** — the checkpoint it was initialised
138
+ from (`limbo_cyr3/best.pt`), **not** the pretrained PP-OCRv6 base. The fine-tune moved the
139
+ median only a little (0.875 → 0.8875) but the **micro** average a lot (0.6499 → 0.7797): the
140
+ median is dominated by the types that were already solved, while the micro average counts every
141
+ image — and the frame-trained types that were unreadable rose sharply
142
+ (`out/LagBug/XProtect/code` 0.00 → 0.36, `out/LagBug/XProtect/map` 0.00 → 0.27,
143
+ `out/InstantlyMoist/Captcha/textcaptcha` 0.12 → 0.24).
144
+
145
+ ### Types the model was trained on
146
+
147
+ These numbers are a **sample-level hold-out**: the first 200 rows of every type were taken out
148
+ of training, the rest were trained on. They are *not* unseen-plugin numbers.
149
+
150
+ | type | n | exact (zero step) | **exact** | CER |
151
+ |---|---:|---:|---:|---:|
152
+ | `out/Archy-X/Krypton/map` | 200 | 1.0000 | **1.0000** | 0.0000 |
153
+ | `out/ringoame196-s-mcPlugin/ReproductionCAPTCHA/map` | 200 | 1.0000 | **1.0000** | 0.0000 |
154
+ | `out/irootdev/AnBot/captcha` | 200 | 0.9350 | **0.9950** | 0.0017 |
155
+ | `out/haq/mapcha/map` | 200 | 0.9650 | 0.9550 | 0.0112 |
156
+ | `out_ru/GalleryCaptcha/ru-web` | 200 | 0.9300 | 0.9250 | 0.0150 |
157
+ | `out/just-captcha/ProstoCaptcha/map` | 200 | 0.9250 | 0.9150 | 0.0150 |
158
+ | `out_ru/LimboFilter/digits` | 200 | 0.8900 | 0.8900 | 0.0288 |
159
+ | `out_ru/GalleryCaptcha/ru-symbols` | 200 | 0.8600 | **0.8850** | 0.0400 |
160
+ | `out_ru/LimboFilter/symbols` | 200 | 0.7650 | **0.8000** | 0.0783 |
161
+ | `out/LagBug/XProtect/code` | 200 | 0.0000 | **0.3600** | 0.1542 |
162
+ | `out/LagBug/XProtect/map` | 200 | 0.0000 | **0.2700** | 0.1850 |
163
+ | `out/InstantlyMoist/Captcha/textcaptcha` | 200 | 0.1200 | **0.2400** | 0.4220 |
164
+
165
+ ### Types the model never saw
166
+
167
+ Whole plugins that were held out of training — these are the honest generalisation numbers.
168
+
169
+ | type | n | exact (zero step) | **exact** | CER |
170
+ |---|---:|---:|---:|---:|
171
+ | `out_ru/CaptchaVerifyZ/map` | 1 500 | 0.9647 | **0.9780** | 0.0045 |
172
+ | `out/JSH32/Minecaptcha/map` | 1 500 | 0.9340 | **0.9373** | 0.0162 |
173
+ | `out_ru/BotFilter/map` | 1 500 | 0.7113 | **0.8140** | 0.0508 |
174
+ | `out/jonesdevelopment/sonar/legacy` | 1 500 | 0.5973 | **0.7693** | 0.0793 |
175
+ | `out/jonesdevelopment/sonar/complex` | 1 500 | 0.3520 | **0.6253** | 0.1293 |
176
+ | `out/jonesdevelopment/sonar/standard` | 1 500 | 0.2613 | **0.5707** | 0.1671 |
177
 
178
+ **One caveat about the protocol.** The evaluation runs on held-out rows of the **same generator
179
+ render** the model was trained on (the generators are not seeded, so a re-render is a different
180
+ sample set). For the twelve trained types the hold-out is by *sample*; only the six held-out
181
+ plugins above are genuinely unseen styles.
182
 
183
+ ## No weight interpolation — and why the previous card said α = 0.4
184
+
185
+ The previous version of this repository published a WiSE-FT soup, `θ = 0.4·θ_ft + 0.6·θ_pretrained`.
186
+ That was the right call **for that model**: a small fine-tune on a few types degraded the unseen
187
+ plugins, and interpolating back towards the base restored them (measured there: base median
188
+ 0.588 → α = 0.4 median 0.638 → α = 1.0 median 0.480).
189
+
190
+ This model is different in two ways: it is trained on **most** of the types it is evaluated on,
191
+ and its head has **grown**. On an expanded head the interpolation has nothing to interpolate
192
+ *towards* for the 52 appended rows — the base model has no such rows — while the encoder is
193
+ dragged back to the pretrained solution. Measured on the α-sweep of the `limbo_cyr3`
194
+ predecessor: the sweep's best α = 0.6 lifted the Latin held-out types from 0.644 to 0.772 and
195
+ at the same time **collapsed the Cyrillic ones**:
196
+
197
+ | type | α = 1.0 | after interpolation (α = 0.6) |
198
+ |---|---:|---:|
199
+ | `out_ru/LimboFilter/symbols` | 0.765 | **0.016** |
200
+ | `out_ru/GalleryCaptcha/ru-symbols` | 0.860 | **0.002** |
201
+ | `out_ru/LimboFilter/digits` | 0.890 | **0.666** |
202
+
203
+ So WiSE-FT is **destructive** on an expanded head. That soup was kept as an exhibit
204
+ (`train/runs/ppocrv6/limbo_cyr3/soup_best.pt`) and is **not** used here. What works for a
205
+ vocabulary expansion on this model is **replay** — keeping the old data in the training mix —
206
+ and this checkpoint is therefore published with **α = 1.0**: the fine-tuned weights, unchanged.
207
+
208
+ ## What this model is NOT good at
209
+
210
+ * **It is not a universal captcha solver.** It is a fine-tune for a **known set of captcha
211
+ styles**: a plugin that draws in a style absent from the training set will not be handled.
212
+ For the six plugins that were held out, the honest expectation is the numbers in the table
213
+ above — between 0.57 and 0.98 exact match, not 1.0.
214
+ * **The three number-word types are not supported at all** (`out_ru/GalleryCaptcha/ru-numberword`,
215
+ `out_ru/GalleryCaptcha/ru-numberword-caption`, `out_ru/LimboFilter/map`): the picture shows a number
216
+ written in Russian words while the answer is digits. Use a solver step for those.
217
+ * **The weakest trained types are `out/LagBug/XProtect/code` (0.36) and `out/LagBug/XProtect/map`
218
+ (0.27)**: both are the same 130×130 sheet with per-character cumulative translation and 51
219
+ interference lines. `out/InstantlyMoist/Captcha/textcaptcha` (0.24) is a 128×128 sheet with
220
+ thin diagonal noise.
221
+ * **It reads a band, not a picture.** On a whole game screenshot it falls apart — see below.
222
 
223
  ## How to use it
224
 
 
227
  from PIL import Image
228
  from transformers import PPOCRV6SmallRecForTextRecognition, PPOCRV6SmallRecImageProcessor
229
 
230
+ REPO = "Titlehhhh/captchaocr-ppocrv6"
231
+
232
+ model = PPOCRV6SmallRecForTextRecognition.from_pretrained(REPO)
233
+ proc = PPOCRV6SmallRecImageProcessor.from_pretrained(REPO)
234
+
235
+ image = Image.open("001.png").convert("RGB") # a crop of the answer band
236
+ px = proc(images=[image], return_tensors="pt")["pixel_values"] # 48 x 320, mean = std = 0.5
237
 
 
 
238
  with torch.no_grad():
239
  logits = model.head.head(model.head.encoder(
240
  model.model(px).last_hidden_state).last_hidden_state)
241
+
242
+ # greedy CTC decoding against character_list.txt (index 0 = "blank")
243
+ with open("character_list.txt", encoding="utf-8") as f:
244
+ dictionary = [line.rstrip("\r\n") for line in f] # rstrip, not strip: " " is a class
245
+
246
+ ids = logits.argmax(-1)[0].tolist()
247
+ text, prev = [], -1
248
+ for i in ids:
249
+ if i != prev and i != 0: # collapse repeats, drop the blank
250
+ text.append(dictionary[i])
251
+ prev = i
252
+ print("".join(text))
253
  ```
254
 
255
+ The same 18 762-character dictionary is also inside `preprocessor_config.json`, so the processor
256
+ can decode without the text file. The input geometry is 48 × 320 (resize + pad), and the
257
+ normalisation is `mean = std = 0.5` — the PaddleOCR convention the fine-tune was trained with.
258
+ **Do not let the processor fall back to the ImageNet statistics**: `preprocessor_config.json`
259
+ already carries the patched values.
260
+
261
+ ### The important practical part: give it a band, not a screenshot
262
+
263
+ The model is trained on the **band** for 8 of its 12 types, so it expects a tight crop of the
264
+ code. On a **whole game screenshot** it collapses: measured on the six real screenshots of the
265
+ project owner, feeding the whole frame gives **2/6** correct readings.
266
+
267
+ The fix is a **cropper** — an «eye» in front of the recogniser. It proposes candidate text bands
268
+ (a per-type heuristic, an ink box, and a glyph detector), the model reads every candidate, and
269
+ the reading with the **highest confidence** wins (vote counts are only a tie-break). With the
270
+ cropper: **6/6** on those same six screenshots. The implementation ships with the project
271
+ (`webapp/cropper.py`, `webapp/app.py` — see
272
+ <https://github.com/Titlehhhh/CaptchaOCR> and `reports/CROPPER.md` for the measured numbers).
273
 
274
+ If you use the dataset: crop by `roi` whenever it is not `null`.
275
 
276
+ ## The dataset this was trained on
277
+
278
+ [`Titlehhhh/captchaocr-dataset`](https://huggingface.co/datasets/Titlehhhh/captchaocr-dataset)
279
+ — synthetic, every image is a re-render by this project's own Java generators of a captcha drawn
280
+ by an open-source Minecraft plugin. As published (commit
281
+ `dd4d7479a95f8e4c8ea19643dfe5889fe5dea113`):
282
+
283
+ | | |
284
+ |---|---|
285
+ | types | **20** (12 in `out`, 8 in `out_ru`), 13 plugins |
286
+ | samples | **200 000** (10 000 per type), 1.9 GiB of PNG bytes |
287
+ | files | 9 Parquet shards (`data/train-*-of-00009.parquet`), one row per sample, raw PNG in `image` |
288
+ | bands | 12 types carry an `roi` band; 119 984 rows have one, 80 016 do not |
289
+ | `text != answer` | 3 types (the Russian number words) |
290
+ | alphabet | 115 distinct characters over the whole set |
291
+
292
+ Two facts worth stating plainly:
293
+
294
+ * The model was trained on the **project's own render** of that dataset (10 000 images per
295
+ type), not on the published Parquet shards — the pixels are the same style, the sample set is
296
+ that of the local render.
297
+ * The type **`out_ru/LimboFilter/digits`** (four digits, real-server style) was trained on and
298
+ **is not part of the published dataset snapshot**: the generator gained that type after the
299
+ dataset was exported.
300
+
301
+ ### Provenance and licence
302
+
303
+ * Base model: `PaddlePaddle/PP-OCRv6_medium_rec_safetensors`, **Apache-2.0**. These weights are
304
+ a derivative of it.
305
+ * The dataset is **synthetic** — re-renders, not screenshots of a live server; no picture, page
306
+ or answer of a third-party commercial service is used or published.
307
+ * The dataset is published under **the licence of the upstream plugin each row came from**
308
+ (`license_name: mixed-per-plugin-licences`; the per-plugin breakdown is in the dataset
309
+ repository's `LICENSE` and in its card). Some of those licences forbid commercial hosting or
310
+ reserve all rights — **read them before you use the data**.
311
+ * The 52 characters added to the head are the Cyrillic alphabet; the images that taught them
312
+ come from the `GalleryCaptcha` and `LimboFilter` types, whose licences are listed above.
313
 
314
  ## Reproducing
315
 
316
  ```powershell
317
+ $env:PYTHONIOENCODING="utf-8"
318
+ $env:HF_HOME="C:\Users\Title\Projects\CapthaModel\.hf-cache"
319
+
320
+ # the run that produced these weights
321
+ .venv-gpu\Scripts\python.exe train\finetune_ppocr.py --out train\runs\ppocrv6\universal_v1 `
322
+ --init-from train\runs\ppocrv6\limbo_cyr3\best.pt --expand-charset --train-frames `
323
+ --epochs 5 --batch-size 32 --lr 8e-6 --head-lr 8e-6 --workers 0 --seed 42 `
324
+ --limit-per-type 1500 --holdout-per-type 200 --eval-every 5 --save-every 400 --max-minutes 60 `
325
+ --val-types out/jonesdevelopment/sonar/standard out/jonesdevelopment/sonar/legacy out/jonesdevelopment/sonar/complex out_ru/BotFilter/map out_ru/CaptchaVerifyZ/map out/laforetbrut/are-you-sure/code out/JSH32/Minecaptcha/map `
326
+ --exclude-types ru-numberword ru-numberword-caption LimboFilter/map
327
+
328
+ # the export (alpha = 1.0 -> the checkpoint itself, no interpolation)
329
+ .venv-gpu\Scripts\python.exe train\export_ppocr_hf.py `
330
+ --ckpt train\runs\ppocrv6\universal_v1\best.pt --soup-alpha 1.0 `
331
+ --out train\runs\ppocrv6\universal_v1\export_hf
332
  ```
333
+
334
+ Notes on the command as it was actually run:
335
+
336
+ * `--init-from train\runs\ppocrv6\limbo_cyr3\best.pt` points at the **replay-mix** checkpoint of
337
+ the same recipe (the chain is `limbo_cyr` → charset expansion → `limbo_cyr3` → replay mix →
338
+ `universal_v1`); the whole chain is documented in `reports/TRAINING_LIMBO_DIGITS.md` §8.
339
+ * The filter `out/laforetbrut/are-you-sure/code` matched **no type** in the dataset and had no
340
+ effect; the other six `--val-types` are the six held-out plugins.
341
+ * `--expand-charset` was a **no-op in this run** («0 characters missing»): the head had already
342
+ been grown to 18 762 classes by the predecessor checkpoint.
343
+ * The dataset is **not seeded**: a re-render produces a different sample set, so the exact
344
+ per-type numbers above belong to the render this checkpoint was measured on.