arkhabbazan commited on
Commit
f34e5e9
·
verified ·
1 Parent(s): f38ece4

Update model card with representative captcha samples

Browse files
Files changed (1) hide show
  1. README.md +85 -183
README.md CHANGED
@@ -1,55 +1,55 @@
1
  ---
2
- library_name: transformers
3
- pipeline_tag: image-to-text
4
- base_model: microsoft/trocr-base-printed
5
- tags:
6
- - trocr
7
- - vision-encoder-decoder
8
- - image-to-text
9
- - ocr
10
- - captcha
11
- - math-captcha
12
- - synthetic-data
13
- ---
14
-
15
- # OCR Match Captcha
16
-
17
- A TrOCR model fine-tuned to recognize short mathematical captcha
18
- expressions such as `26+7=?` and `45-6=?`.
19
 
20
  <table border="0" cellspacing="24" cellpadding="12">
21
  <tbody>
22
  <tr>
23
- <td style="border: none; padding: 18px;">
24
  <img
25
  src="assets/captcha-handwritten-7-plus-12.png"
26
- alt="Handwritten mathematical captcha"
27
  width="290"
28
  height="90"
29
  >
30
  </td>
31
- <td style="border: none; padding: 18px;">
32
  <img
33
  src="assets/captcha-clean-10-plus-3.png"
34
- alt="Clean mathematical captcha"
35
  width="290"
36
  height="90"
37
  >
38
  </td>
39
  </tr>
40
  <tr>
41
- <td style="border: none; padding: 18px;">
42
  <img
43
  src="assets/captcha-distorted-18-plus-5.png"
44
- alt="Distorted mathematical captcha"
45
  width="290"
46
  height="90"
47
  >
48
  </td>
49
- <td style="border: none; padding: 18px;">
50
  <img
51
  src="assets/captcha-omex-37-plus-13.png"
52
- alt="OMEX mathematical captcha"
53
  width="290"
54
  height="90"
55
  >
@@ -58,161 +58,63 @@ expressions such as `26+7=?` and `45-6=?`.
58
  </tbody>
59
  </table>
60
 
61
- ## Model details
62
-
63
- - **Architecture:** Vision Encoder-Decoder / TrOCR
64
- - **Base model:** `microsoft/trocr-base-printed`
65
- - **Task:** Image-to-text OCR
66
- - **Target geometry:** 130x30 RGB images
67
- - **Model-card image size:** 290x90 pixels
68
- - **Output:** Mathematical expressions without spaces
69
- - **Supported operators:** `+` and `-`
70
- - **Expected format:** `number operator number = ?`
71
-
72
- ## Training data
73
-
74
- The model was fine-tuned on 10,000 target-matched synthetic
75
- mathematical captcha images.
76
-
77
- | Split | Samples |
78
- |---|---:|
79
- | Training | 8,500 |
80
- | Validation | 1,000 |
81
- | Test | 500 |
82
-
83
- The synthetic generator balances `+` and `-`, includes `=` and `?`,
84
- and simulates:
85
-
86
- - Thin and low-contrast arithmetic symbols
87
- - Pastel circle and ring overlap
88
- - Background noise
89
- - Blur and interpolation artifacts
90
- - JPEG compression
91
- - Small character displacement
92
- - Variable character spacing
93
- - Different foreground contrast levels
94
-
95
- Labels contain no spaces and use ASCII `-` for subtraction.
96
- Visual spacing is applied only while rendering the images.
97
-
98
- ## Training procedure
99
-
100
- The complete Vision Encoder-Decoder model was fine-tuned using an
101
- operator-weighted sequence loss.
102
-
103
- Tokens belonging to `+`, `-`, `=`, and `?` received additional
104
- weight so digit tokens did not dominate the training objective.
105
-
106
- The best checkpoint was selected using validation exact-expression
107
- accuracy. The held-out test split was evaluated only after the best
108
- validation checkpoint had been restored.
109
-
110
- ## Training configuration
111
-
112
- | Parameter | Value |
113
- |---|---:|
114
- | Epochs | 5 |
115
- | Learning rate | 5e-6 |
116
- | Batch size | 8 |
117
- | Gradient accumulation | 2 |
118
- | Operator-token weight | 3.0 |
119
- | Generation beams | 4 |
120
- | Maximum output length | 32 |
121
-
122
- ## Synthetic evaluation
123
-
124
- | Metric | Validation | Test |
125
- |---|---:|---:|
126
- | Exact-expression accuracy | 100.000% | 100.000% |
127
- | Arithmetic-operator accuracy | 100.000% | 100.000% |
128
-
129
- These results measure the synthetic rendering family used during
130
- fine-tuning. They do not guarantee identical performance on every
131
- real captcha provider, unseen font, or unseen visual style.
132
-
133
- ## Usage
134
-
135
- ```python
136
- import torch
137
- from PIL import Image
138
- from transformers import (
139
- TrOCRProcessor,
140
- VisionEncoderDecoderModel,
141
- )
142
-
143
- repo_id = "arkhabbazan/ocr-match-captcha"
144
- device = "cuda" if torch.cuda.is_available() else "cpu"
145
-
146
- processor = TrOCRProcessor.from_pretrained(
147
- repo_id,
148
- token=True,
149
- )
150
-
151
- model = VisionEncoderDecoderModel.from_pretrained(
152
- repo_id,
153
- token=True,
154
- ).to(device).eval()
155
-
156
- image = Image.open("captcha.jpeg").convert("RGB")
157
-
158
- # Keep the original image aspect ratio.
159
- pixel_values = processor(
160
- images=image,
161
- return_tensors="pt",
162
- ).pixel_values.to(device)
163
-
164
- with torch.inference_mode():
165
- generated_ids = model.generate(
166
- pixel_values,
167
- num_beams=4,
168
- max_length=32,
169
- )
170
-
171
- prediction = processor.batch_decode(
172
- generated_ids,
173
- skip_special_tokens=True,
174
- )[0]
175
-
176
- print(prediction.replace(" ", ""))
177
- ```
178
-
179
- Example output:
180
-
181
- ```text
182
- 26+7=?
183
- ```
184
-
185
- ## Input preprocessing
186
-
187
- Pass the original RGB captcha directly to `TrOCRProcessor`.
188
-
189
- Do not manually stretch a wide captcha to 130x130. The processor
190
- performs the model's required resizing and normalization.
191
-
192
- The `290x90` size used by the four model-card images is only for
193
- consistent presentation. It is not a required inference input size.
194
-
195
- ## Intended use
196
-
197
- This model is intended for:
198
-
199
- - Short mathematical captcha OCR
200
- - Authorized captcha-processing workflows
201
- - OCR experimentation
202
- - Synthetic-to-real domain adaptation research
203
- - Arithmetic-symbol recognition evaluation
204
-
205
- ## Limitations
206
-
207
- - Training data is primarily synthetic.
208
- - Unseen fonts and layouts may reduce accuracy.
209
- - Very faint or heavily occluded operators may be missed.
210
- - Visual noise may be confused with `+`, `-`, `=`, or `?`.
211
- - The model is intended for short expressions, not document OCR.
212
- - Synthetic evaluation may overestimate real-world performance.
213
-
214
- ## Responsible use
215
-
216
- Use this model only on systems and captcha images that you are
217
- authorized to process. Users are responsible for complying with
218
- applicable laws, service terms, and access-control policies.
 
1
  ---
2
+ library_name: transformers
3
+ pipeline_tag: image-to-text
4
+ base_model: microsoft/trocr-base-printed
5
+ tags:
6
+ - trocr
7
+ - vision-encoder-decoder
8
+ - image-to-text
9
+ - ocr
10
+ - captcha
11
+ - math-captcha
12
+ - synthetic-data
13
+ ---
14
+
15
+ # OCR Match Captcha
16
+
17
+ A TrOCR model fine-tuned to recognize short mathematical captcha
18
+ expressions such as `26+7=?` and `45-6=?`.
19
 
20
  <table border="0" cellspacing="24" cellpadding="12">
21
  <tbody>
22
  <tr>
23
+ <td>
24
  <img
25
  src="assets/captcha-handwritten-7-plus-12.png"
26
+ alt="Handwritten captcha"
27
  width="290"
28
  height="90"
29
  >
30
  </td>
31
+ <td>
32
  <img
33
  src="assets/captcha-clean-10-plus-3.png"
34
+ alt="Clean captcha"
35
  width="290"
36
  height="90"
37
  >
38
  </td>
39
  </tr>
40
  <tr>
41
+ <td>
42
  <img
43
  src="assets/captcha-distorted-18-plus-5.png"
44
+ alt="Distorted captcha"
45
  width="290"
46
  height="90"
47
  >
48
  </td>
49
+ <td>
50
  <img
51
  src="assets/captcha-omex-37-plus-13.png"
52
+ alt="OMEX captcha"
53
  width="290"
54
  height="90"
55
  >
 
58
  </tbody>
59
  </table>
60
 
61
+ ## Model details
62
+
63
+ - **Architecture:** Vision Encoder-Decoder / TrOCR
64
+ - **Base model:** `microsoft/trocr-base-printed`
65
+ - **Task:** Image-to-text OCR
66
+ - **Target geometry:** 130x30 RGB images
67
+ - **Output:** Mathematical expressions without spaces
68
+
69
+ ## Training data
70
+
71
+ The model was fine-tuned on 10,000 target-matched synthetic images:
72
+
73
+ | Split | Samples |
74
+ |---|---:|
75
+ | Training | 8,500 |
76
+ | Validation | 1,000 |
77
+ | Test | 500 |
78
+
79
+ The generator balances `+` and `-`, includes `=` and `?`, and simulates
80
+ thin low-contrast symbols, pastel circle overlap, blur, JPEG compression,
81
+ background noise, and small character displacement.
82
+
83
+ ## Training configuration
84
+
85
+ | Parameter | Value |
86
+ |---|---:|
87
+ | Epochs | 5 |
88
+ | Learning rate | 5e-6 |
89
+ | Batch size | 8 |
90
+ | Gradient accumulation | 2 |
91
+ | Operator-token weight | 3.0 |
92
+ | Generation beams | 4 |
93
+ | Maximum output length | 32 |
94
+
95
+ ## Usage
96
+
97
+ ```python
98
+ from PIL import Image
99
+ from transformers import TrOCRProcessor, VisionEncoderDecoderModel
100
+
101
+ repo_id = "arkhabbazan/ocr-match-captcha"
102
+ processor = TrOCRProcessor.from_pretrained(repo_id, token=True)
103
+ model = VisionEncoderDecoderModel.from_pretrained(repo_id, token=True)
104
+
105
+ image = Image.open("captcha.jpeg").convert("RGB")
106
+ pixel_values = processor(images=image, return_tensors="pt").pixel_values
107
+ generated_ids = model.generate(pixel_values, num_beams=4, max_length=32)
108
+ prediction = processor.batch_decode(
109
+ generated_ids, skip_special_tokens=True
110
+ )[0]
111
+ print(prediction.replace(" ", ""))
112
+ ```
113
+
114
+ ## Limitations
115
+
116
+ - Training data is primarily synthetic.
117
+ - Unseen fonts and layouts may reduce accuracy.
118
+ - Very faint or heavily occluded operators may be missed.
119
+ - The model is intended for short expressions, not document OCR.
120
+ - Usage must be authorized and compliant with applicable policies.