| --- |
| license: apache-2.0 |
| library_name: torchocr |
| pipeline_tag: image-to-text |
| tags: |
| - ocr |
| - text-detection |
| - text-recognition |
| - dbnet |
| - crnn |
| - pytorch |
| datasets: |
| - icdar2015 |
| - total-text |
| - mlt2019 |
| - funsd |
| - mjsynth |
| --- |
| |
| # torchocr weights |
|
|
| Pretrained checkpoints for [torchocr](https://github.com/BryanBradfo/torchOCR), a PyTorch-native, |
| torchvision-style OCR library. You normally never download these files by hand: each one backs a |
| member of a `*_Weights` enum, and `torch.hub` fetches it on first use and checks the SHA-256 prefix |
| embedded in the file name. |
|
|
| ```python |
| from torchocr import DBPostProcessor |
| from torchocr.models import DBNet_ResNet18_VD_Weights, dbnet_resnet18_vd |
| |
| weights = DBNet_ResNet18_VD_Weights.ICDAR2015 |
| model = dbnet_resnet18_vd(weights=weights).eval() |
| preprocess = weights.transforms() |
| postprocess = DBPostProcessor(**weights.meta["postprocess"]) |
| ``` |
|
|
| ## Checkpoints |
|
|
| | Weights enum | File | Params | ICDAR-2015 test | |
| |---|---|---|---| |
| | `DBNet_ResNet18_VD_Weights.ICDAR2015` | `dbnet_resnet18_vd_ic15-d803cec9.pth` | 12.4 M | P 0.849 · R 0.774 · **hmean 0.809** | |
| | `DBNet_MobileNetV3_Large_05_Weights.ICDAR2015` | `dbnet_mobilenet_v3_large_05_ic15-d9d17ae7.pth` | 0.60 M | P 0.785 · R 0.689 · hmean 0.734 | |
| | `DBNet_MobileNetV3_Large_05_Weights.PPOCR_V3_EN` | `dbnet_mobilenet_v3_large_05_ppocr_v3_en-9bfd3b59.pth` | 0.60 M | hmean 0.441 ¹ | |
| | `DBNet_MobileNetV3_Large_05_Weights.PPOCR_V3_CH` | `dbnet_mobilenet_v3_large_05_ppocr_v3_ch-fc000d1e.pth` | 0.60 M | hmean 0.422 ¹ | |
| | `DBNet_ResNet18_VD_Weights.PPOCR_SERVER_V2` | `dbnet_resnet18_vd_ppocr_server_v2-59d99b11.pth` | 12.4 M | hmean 0.414 ¹ | |
| | `CRNN_ResNet34_VD_Weights.MJSYNTH_ICDAR2015_TOTALTEXT_MLT19` | `crnn_resnet34_vd_synth-513c0334.pth` (+ LM `charlm6_scene-883dd7cd.json`) | 27.9 M | **word acc. 0.799** · Total-Text **0.819** ³ | |
| | `CRNN_ResNet34_VD_Weights.ICDAR2015_TOTALTEXT_MLT19` | `crnn_resnet34_vd_scene-38393899.pth` (+ LM `charlm6_scene-883dd7cd.json`) | 27.9 M | word acc. 0.791 · Total-Text 0.802 ³ | |
| | `CRNN_ResNet34_VD_Weights.ICDAR2015` | `crnn_resnet34_vd_ic15-da0782b5.pth` | 27.9 M | word acc. 0.738 · CER 0.101 ² | |
| | `CRNN_ResNet34_VD_Weights.PPOCR_SERVER_V2` | `crnn_resnet34_vd_ppocr_server_v2-803ba4c9.pth` | 27.9 M | word acc. 0.665 ² | |
|
|
| ¹ Line-level detectors converted from PaddleOCR. ICDAR-2015 scores *word* boxes, so adjacent words |
| merged into one line count as misses: these numbers compare checkpoints, they are not the accuracy |
| you will see on documents. |
| ² 2077 alphanumeric care words of ICDAR-2015 test cropped from the full images with |
| `torchocr.ops.crop_quads` on their ground-truth quads, case-insensitive (not the official Task 4.3 |
| crops). Axis-aligned crops of the same words score 0.579. |
| ³ Decoded with its declared CTC beam search (width 10) and 6-gram character language model; 0.750 with greedy |
| decoding (0.757 for the MJSynth checkpoint). Same ground-truth-quad protocol; Total-Text curved words use their minimum-area rectangle. |
|
|
| End to end (`OCRPipeline.from_pretrained`, image → text; a word counts when it is detected *and* |
| read correctly, case-insensitive, generic vocabulary; `references/end2end/evaluate.py`), with |
| `DBNet_ResNet18_VD_Weights.ICDAR2015` and each scene recognizer, `min_text_score=0.8`: |
|
|
| | Test set | Detection hmean | `…ICDAR2015_TOTALTEXT_MLT19` (v0.3) | `…MJSYNTH_ICDAR2015_TOTALTEXT_MLT19` | |
| | --- | --- | --- | --- | |
| | ICDAR-2015 (street scenes) | 0.809 | 0.649 | **0.665** | |
| | Total-Text (curved text) | 0.696 | 0.585 | **0.585** | |
| | FUNSD (scanned forms) | 0.826 | 0.715 | **0.722** | |
|
|
| Without `min_text_score`, the v0.3 recognizer scores 0.621 / 0.566 / 0.711. |
|
|
| `min_text_score=0.8` drops regions read with low confidence; the value maximizes end-to-end hmean on the |
| ICDAR-2015 *training* split. The converted PaddleOCR weights reach 0.219 on ICDAR-2015 and 0.17–0.18 detection |
| hmean on FUNSD, whose word-level boxes do not match their line-level output. |
|
|
| ```python |
| from torchocr import OCRPipeline, load_image |
| |
| pipeline = OCRPipeline.from_pretrained( |
| "DBNet_ResNet18_VD_Weights.ICDAR2015", "CRNN_ResNet34_VD_Weights.MJSYNTH_ICDAR2015_TOTALTEXT_MLT19", device="cuda" |
| ) # the recognizer's beam search + language model is set up automatically |
| pipeline.min_text_score = 0.8 |
| doc = pipeline(load_image("photo.jpg")) # doc.text, doc.polygons |
| ``` |
|
|
| ## Evaluation protocol |
|
|
| Detection: official ICDAR-2015 IoU protocol (one-to-one greedy matching at IoU > 0.5, `###` |
| don't-care filtering) on rotated quads, implemented by `torchocr.metrics.DetectionHmean` and verified |
| identical to PaddleOCR's `DetectionIoUEvaluator`. Pre/post-processing is exactly |
| `weights.transforms()` and `weights.meta["postprocess"]`. Every number is reproducible with: |
|
|
| ```bash |
| python references/detection/evaluate.py --root <icdar2015> --weights <enum> |
| python references/recognition/evaluate.py --root <icdar2015> --weights CRNN_ResNet34_VD_Weights.PPOCR_SERVER_V2 |
| ``` |
|
|
| ### The `ICDAR2015` checkpoints |
|
|
| All were fine-tuned by torchocr on *training* splits only (ICDAR-2015, plus Total-Text and MLT-2019 for the |
| recognizer noted below); reported weights are the |
| **last epoch** and no checkpoint or hyper-parameter was selected on the test set. |
|
|
| - `DBNet_ResNet18_VD_Weights.ICDAR2015`: `PPOCR_SERVER_V2` fine-tuned 300 epochs with the recipe below; |
| `box_thresh = 0.5` maximizes train-split hmean (0.794 with PaddleOCR's 0.6). |
| - `CRNN_ResNet34_VD_Weights.ICDAR2015`: `PPOCR_SERVER_V2` fine-tuned 30 epochs on the 4467 legible |
| training words, cropped with `crop_quads` from jittered ground-truth quads |
| (`references/recognition/train.py`). It keeps PaddleOCR's Chinese + Latin charset but saw only Latin |
| scene text, so Chinese accuracy is no longer guaranteed. |
| - `CRNN_ResNet34_VD_Weights.ICDAR2015_TOTALTEXT_MLT19`: `PPOCR_SERVER_V2` fine-tuned 10 epochs (bf16) on |
| 66k Latin scene words (ICDAR-2015 train ×2, Total-Text train, MLT-2019 Latin crops). Its language model |
| (`charlm6_scene-883dd7cd.json`) is a case-insensitive 6-gram character model built from those *training* |
| transcriptions only; beam width and LM weight were chosen on Total-Text *training* words. |
| - `CRNN_ResNet34_VD_Weights.MJSYNTH_ICDAR2015_TOTALTEXT_MLT19`: the checkpoint above fine-tuned 6 more |
| epochs (bf16), each drawing 370k words at random: the 66k real words once on average plus 300k of the |
| 7.2M [MJSynth](https://www.robots.ox.ac.uk/~vgg/data/text/) synthetic words (14 undecodable images |
| skipped). Same decoder and language model. |
| - `DBNet_MobileNetV3_Large_05_Weights.ICDAR2015`, described next. |
|
|
| #### `DBNet_MobileNetV3_Large_05_Weights.ICDAR2015` |
|
|
| `PPOCR_V3_EN` fine-tuned for 300 epochs on the 1000 ICDAR-2015 training images |
| (`references/detection/train.py`, PaddleOCR's augmentation recipe, 640 px crops, AdamW 1e-3 with |
| warmup + cosine, ~80 min on one laptop GPU). The reported weights are the **last epoch**: no |
| checkpoint was selected on the test set. `box_thresh = 0.45` maximizes hmean on the **training** |
| split (`references/detection/calibrate.py`) and was applied once to the test split; with |
| PaddleOCR's default `box_thresh = 0.6` the same checkpoint scores 0.688. |
|
|
| Intended inputs: incidental scene text in street-level photos, Latin script, evaluated at |
| 736 × 1280. Expect lower recall on dense documents and on non-Latin scripts. |
|
|
| ## Provenance and licenses |
|
|
| - `PPOCR_*` checkpoints are converted from [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR) |
| releases (Apache-2.0) using the build-time converters in torchocr's `scripts/`, whose recipes are |
| adapted from [PaddleOCR2Pytorch](https://github.com/frotms/PaddleOCR2Pytorch) (Apache-2.0). |
| The PP-OCRv3 files hold the `Student` network that PaddleOCR exports for inference. |
| - The `ICDAR2015` checkpoint derives from `PPOCR_V3_EN` (Apache-2.0) and was fine-tuned on the |
| [ICDAR 2015 Incidental Scene Text](https://rrc.cvc.uab.es/?ch=4) training set, which its |
| organizers release for research use. Check that your use complies with those terms. |
| - `CRNN_ResNet34_VD_Weights.MJSYNTH_ICDAR2015_TOTALTEXT_MLT19` was additionally trained on MJSynth |
| (Jaderberg et al., 2014), released by the Oxford Visual Geometry Group for research use. |
|
|
