C7: honest card — 6 epochs/240 of 1230 steps, measured sources, no weights

README.md

@@ -1,39 +1,36 @@
 ---
 license: apache-2.0
-library_name: transformers
-base_model: PaddlePaddle/PP-OCRv6_medium_rec_safetensors
 tags:
 - captcha
 - ocr
 - minecraft
 - anti-bot
 - pp-ocr
-- image-to-text
-- pytorch
-pipeline_tag: image-to-text
+- base_model:PaddlePaddle/PP-OCRv6_medium_rec_safetensors
 ---
 
 # CaptchaOCR PP-OCRv6 — a text recogniser for Minecraft anti-bot captchas
 
 > ## Status: the weights are **not published yet**. This repository holds the model card only.
 >
-> The fine-tune on the [CaptchaOCR dataset](https://huggingface.co/datasets/Titlehhhh/captchaocr-dataset)
-> has **not been run to completion yet**, so there is **no checkpoint, no `config.json` and no
-> measured metric** in this repository. Nothing here is invented and nothing is extrapolated from a
-> probe run: the numbers quoted below are the **measured numbers of the base model before
-> fine-tuning**, and they are labelled as such.
+> The fine-tune on the [CaptchaOCR dataset](https://huggingface.co/datasets/Titlehhhh/captchaocr-dataset) has **not been run to completion**.
+> The main run reached **6 epochs of 30 planned
+> (240 of 1230 steps)** and was stopped by the author, so there is
+> **no checkpoint, no `config.json` and no weight file** in this repository. Nothing here is
+> invented: every number below is a measurement, and the file it was measured in is named
+> next to it.
 >
-> **RU:** репозиторий модели пока **пуст по весам** — здесь только карточка. Обучение на
-> датасете `Titlehhhh/captchaocr-dataset` ещё не завершено, поэтому **метрики появятся после
-> прогона** (плейслолдер ниже — это не выдуманные числа). Выкладка пробного чекпоинта (2 эпохи)
-> сознательно не сделана: он хуже базовой модели.
+> **RU:** репозиторий модели пока **пуст по весам** — здесь только карточка. Основной прогон
+> дообучения отработал **6 эпох из 30** (шаг
+> **240 из 1230**) и был остановлен автором; обучение продолжается.
+> Пилотный прогон на 2 эпохи (`train/run_ppocr/smoke/`) — это отдельный прогон, его числа
+> не относятся к основному прогону и в таблицах ниже не подмешаны.
 
 ## What it reads
 
-Frames from the [CaptchaOCR dataset](https://huggingface.co/datasets/Titlehhhh/captchaocr-dataset) — the kind of
-pictures the model is trained on. The caption gives the type and the **gold answer** from
-`labels.jsonl` (what the player has to type). The links are absolute: the pictures live in the
-dataset repository, not in this one.
+Frames from the [CaptchaOCR dataset](https://huggingface.co/datasets/Titlehhhh/captchaocr-dataset) — the kind of pictures the model is trained
+on. The caption gives the type and the **gold answer** from `labels.jsonl` (what the player has
+to type). The links are absolute: the pictures live in the dataset repository, not in this one.
 
 | | | |
 |---|---|---|
@@ -44,11 +41,11 @@ dataset repository, not in this one.
 
 A fine-tune of **PP-OCRv6 medium** (PaddleOCR's text recogniser) on the CaptchaOCR dataset: the
 captchas that open-source Minecraft anti-bot plugins draw, rendered by this project's own Java
-generators (46 types, 4600 labelled PNG).
+generators (46 captcha types, 4600 labelled PNG, 23 upstream plugins).
 
 * **Architecture:** PP-OCRv6 medium recogniser — PP-LCNetV4 backbone + CTC head,
   `transformers` class `PPOCRV6SmallRecForTextRecognition`
-  (`model_type: pp_ocrv6_small_rec`), 19.1 M parameters, hidden size 192.
+  (`model_type: pp_ocrv6_small_rec`), 19 134 326 parameters, hidden size 192.
 * **Dictionary:** the base model's own 18 710 classes (index 0 = CTC blank, space at the end).
 * **Input:** the captcha **text band** (the strip of pixels that carries the answer), not the whole
   frame: resized to height 48, width scaled proportionally and right-padded to 320, BGR channel
@@ -58,49 +55,74 @@ generators (46 types, 4600 labelled PNG).
   model has nothing to exploit.
 
 **The band is the whole game.** On the 29 dataset types that carry a band, the base model scores
-`acc_micro = 0.5517` on the band and `acc_micro = 0.0345` on the whole frame — about 16× worse on
-the frame. A solver must crop first.
+`acc_micro = 0.5517` on the band and `acc_micro = 0.0345` on the whole frame — about
+16× worse on the frame. A solver must crop first.
 
 ## Provenance and licence
 
-* **Base model:** [`PaddlePaddle/PP-OCRv6_medium_rec_safetensors`](https://huggingface.co/PaddlePaddle/PP-OCRv6_medium_rec_safetensors)
-  — Apache-2.0, by PaddlePaddle.
+* **Base model:** [`PaddlePaddle/PP-OCRv6_medium_rec_safetensors`](https://huggingface.co/PaddlePaddle/PP-OCRv6_medium_rec_safetensors) — Apache-2.0, by PaddlePaddle.
 * **This model:** that base, fine-tuned on the CaptchaOCR dataset published here:
   [`Titlehhhh/captchaocr-dataset`](https://huggingface.co/datasets/Titlehhhh/captchaocr-dataset)
   (46 captcha types · 4600 labelled PNG · 23 upstream plugins).
 * **Licence of the weights:** Apache-2.0, inherited from the base model. The fine-tuning code and
   this card are released under the same terms.
 * **Licence of the dataset:** *not* Apache-2.0 — the dataset is split by the licence of the
-  upstream plugin of each captcha type. The dataset card has the per-type table.
+  upstream plugin of each captcha type. The dataset card has the per-type table, and the file
+  `LICENSE` in the dataset repository spells the same table out in full.
 
 ## Measured results
 
 ### This repository's weights
 
-**No metrics yet — the training run has not been completed.** This section will hold exact match
-and CER per type and on the type-held-out split when the fine-tune is done. Until then, treat the
-numbers below as the **starting point**, not as the result of this repository.
+**No weights, no metrics — the training run has not been completed.** This section will hold the
+exact match and CER per type and on the type-held-out split when a fine-tune that is actually
+better than the base model is finished. Until then, treat every number below as the **starting
+point**, not as a result of this repository.
 
-### The base model, measured before fine-tuning (baseline)
+### The fine-tuning run that was measured (incomplete, stopped by the author)
 
-These are measured numbers of `PaddlePaddle/PP-OCRv6_medium_rec_safetensors` **as it is**, on the
-CaptchaOCR dataset — i.e. the point the fine-tune starts from:
+| | |
+|---|---|
+| command | `.venv-gpu\Scripts\python.exe train\finetune_ppocr.py --out train\run_ppocr\final --epochs 30 --batch-size 32 --lr 1e-4 --head-lr 1e-4 --workers 6 --eval-every 2 --save-every 100 --max-minutes 11` |
+| epochs completed | **6 of 30 planned** — stopped: by the author (the run was killed during epoch 5) |
+| steps completed | **240 of 1230** |
+| train loss | 1.0171 (epoch 0) → 0.0136 (epoch 5) |
+| speed | 101.3–119.2 img/s |
+| checkpoint (not published) | `train/run_ppocr/final/ckpt/step_000200.pt (model + optimiser + scheduler + GradScaler + step), best.pt (epoch 1)` |
+| source | `reports/TRAINING_PPOCR_V6.json` → `training_run` |
+
+### The base model, measured before fine-tuning (the zero step of that run)
+
+| held-out type | exact match | CER |
+|---|---|---|
+| `out/JSH32/Minecaptcha/map` | 0.93 | 0.02 |
+| `out/jonesdevelopment/sonar/complex` | 0.45 | 0.278 |
+| `out/jonesdevelopment/sonar/legacy` | 0.23 | 0.435 |
+| `out/jonesdevelopment/sonar/standard` | 0.6 | 0.158 |
+| `out/laforetbrut/are-you-sure/code` | 0.01 | 0.8 |
+| `out_ru/BotFilter/map` | 0.73 | 0.093 |
+| `out_ru/CaptchaVerifyZ/map` | 0.96 | 0.01 |
+| **median over 7 types** | **0.6** | **0.1575** |
+| **micro (700 images)** | **0.5586** | **0.2561** |
+
+### The base model on the whole dataset (archived benchmark)
 
 | measurement | value | source |
 |---|---|---|
-| exact match (micro) on the 7 type-held-out types, band crop | 0.5586 | `train/run_ppocr/` zero-step baseline |
-| exact match (median over those 7 types) | 0.60 | `train/run_ppocr/` zero-step baseline |
-| CER (micro) on the same split | 0.2561 | `train/run_ppocr/` zero-step baseline |
 | exact match (micro) on the 29 band-carrying types, band crop | 0.5517 | `reports/bench_models/ppocrv6-medium.json` |
 | exact match (micro) on the same 29 types, whole frame | 0.0345 | `reports/bench_models/ppocrv6-medium.json` |
+| exact match (micro) on all 46 types, band crop | 0.4043 | `reports/bench_models/ppocrv6-medium.json` |
 
-Both sources are files in the **CapthaModel** project repository; nothing above is a projection.
+Every number in this card comes from a file of the **CaptchaOCR** project repository — the
+measured results are in `reports/`, they are not typed by hand and nothing is extrapolated.
 
-**Why the weights are not here yet.** A short probe run (2 epochs) of the fine-tune was measured
-on the same held-out types and it came out **worse than the base model** — median exact match
-0.41 after epoch 1 and 0.24 after epoch 3, against 0.60 for the untouched base model. That
-checkpoint is deliberately **not** published: a two-epoch probe is not a model, and shipping it as
-one would misrepresent it.
+**Why the weights are not here yet.** The measured run did **not** improve on the base model:
+the median exact match over the 7 held-out types fell from 0.6 (untouched base)
+to 0.41 at epoch 1 and to 0.24 at epoch 3, and
+one type out of seven got better while six got worse. A checkpoint like that is deliberately
+**not** published: shipping it as "the model" would misrepresent it. Training continues with a
+changed hypothesis (a much smaller learning rate, a frozen backbone, a larger and more diverse
+training set) — the plan and the measurements are in `reports/TRAINING_PPOCR_V6.md` §8.
 
 ## How to use it
 
@@ -169,23 +191,36 @@ the whole frame makes the model read the wrong line.
 
 ## Reproducing
 
-The training code lives in the **CapthaModel** repository:
+The training code lives in the **CaptchaOCR** project repository:
 
 ```powershell
 .venv-gpu\Scripts\python.exe train\finetune_ppocr.py --epochs 30 --batch-size 32
 ```
 
-`train/finetune_ppocr.py` fine-tunes `PaddlePaddle/PP-OCRv6_medium_rec_safetensors` on the
-dataset, holds out whole **plugins** (never random splits — a random split leaks the exact gold
-answer of 34 % of the test samples), and reports exact match and CER per type plus the median over
-types. `train/ppocr_data.py` holds the dataset loader and the preprocessing shown above.
+`train/finetune_ppocr.py` fine-tunes `PaddlePaddle/PP-OCRv6_medium_rec_safetensors` on the dataset, holds out whole **plugins**
+(never a random image split: several captcha types have only a few distinct answers, so a random
+split lets the exact gold answer of a test image appear in the training half), and reports exact
+match and CER per type plus the median over types. `train/ppocr_data.py` holds the dataset loader
+and the preprocessing shown above.
+
+The measured report of the run quoted above is `reports/TRAINING_PPOCR_V6.json` (numbers) and
+`reports/TRAINING_PPOCR_V6.md` (the analysis), both in the project repository.
+
+## Repository metadata
+
+While the weights are absent, this repository declares **only** `license: Apache-2.0` and the
+tags above — there is deliberately **no** `library_name`, **no** `pipeline_tag` and **no**
+`base_model:` field, because those three fields make the Hub list the repository as a loadable
+`transformers` model. There is nothing to load yet. When the weights are published, the
+`library_name: transformers` and `pipeline_tag: image-to-text` fields will be added back together
+with the weight files, in the same commit.
 
 ## Limitations
 
 * **Only the text band is read.** This is a recogniser, not a solver: it does not reverse strings,
   does not turn Russian number words into digits and does not compute arithmetic.
-* **No Cyrillic in the dictionary.** The base model's 18 710 classes contain no Cyrillic at all, so
-  the Cyrillic captcha types of the dataset cannot be expressed in this model's alphabet.
+* **No Cyrillic in the dictionary.** The base model's 18 710 classes contain no Cyrillic at
+  all, so the Cyrillic captcha types of the dataset cannot be expressed in this model's alphabet.
 * **The data is synthetic.** Every image is a render of this project's own generators — faithful to
   the upstream algorithms, but not a screenshot of a live server.
 * **Generalisation is the hard part.** A recogniser can memorise a plugin's style instead of