--- license: apache-2.0 library_name: torchocr pipeline_tag: image-to-text tags: - ocr - text-detection - text-recognition - dbnet - crnn - pytorch datasets: - icdar2015 - total-text - mlt2019 - funsd - mjsynth --- # torchocr weights Pretrained checkpoints for [torchocr](https://github.com/BryanBradfo/torchOCR), a PyTorch-native, torchvision-style OCR library. You normally never download these files by hand: each one backs a member of a `*_Weights` enum, and `torch.hub` fetches it on first use and checks the SHA-256 prefix embedded in the file name. ```python from torchocr import DBPostProcessor from torchocr.models import DBNet_ResNet18_VD_Weights, dbnet_resnet18_vd weights = DBNet_ResNet18_VD_Weights.ICDAR2015 model = dbnet_resnet18_vd(weights=weights).eval() preprocess = weights.transforms() postprocess = DBPostProcessor(**weights.meta["postprocess"]) ``` ## Checkpoints | Weights enum | File | Params | ICDAR-2015 test | |---|---|---|---| | `DBNet_ResNet18_VD_Weights.ICDAR2015` | `dbnet_resnet18_vd_ic15-d803cec9.pth` | 12.4 M | P 0.849 · R 0.774 · **hmean 0.809** | | `DBNet_MobileNetV3_Large_05_Weights.ICDAR2015` | `dbnet_mobilenet_v3_large_05_ic15-d9d17ae7.pth` | 0.60 M | P 0.785 · R 0.689 · hmean 0.734 | | `DBNet_MobileNetV3_Large_05_Weights.PPOCR_V3_EN` | `dbnet_mobilenet_v3_large_05_ppocr_v3_en-9bfd3b59.pth` | 0.60 M | hmean 0.441 ¹ | | `DBNet_MobileNetV3_Large_05_Weights.PPOCR_V3_CH` | `dbnet_mobilenet_v3_large_05_ppocr_v3_ch-fc000d1e.pth` | 0.60 M | hmean 0.422 ¹ | | `DBNet_ResNet18_VD_Weights.PPOCR_SERVER_V2` | `dbnet_resnet18_vd_ppocr_server_v2-59d99b11.pth` | 12.4 M | hmean 0.414 ¹ | | `CRNN_ResNet34_VD_Weights.MJSYNTH_ICDAR2015_TOTALTEXT_MLT19` | `crnn_resnet34_vd_synth-513c0334.pth` (+ LM `charlm6_scene-883dd7cd.json`) | 27.9 M | **word acc. 0.799** · Total-Text **0.819** ³ | | `CRNN_ResNet34_VD_Weights.ICDAR2015_TOTALTEXT_MLT19` | `crnn_resnet34_vd_scene-38393899.pth` (+ LM `charlm6_scene-883dd7cd.json`) | 27.9 M | word acc. 0.791 · Total-Text 0.802 ³ | | `CRNN_ResNet34_VD_Weights.ICDAR2015` | `crnn_resnet34_vd_ic15-da0782b5.pth` | 27.9 M | word acc. 0.738 · CER 0.101 ² | | `CRNN_ResNet34_VD_Weights.PPOCR_SERVER_V2` | `crnn_resnet34_vd_ppocr_server_v2-803ba4c9.pth` | 27.9 M | word acc. 0.665 ² | ¹ Line-level detectors converted from PaddleOCR. ICDAR-2015 scores *word* boxes, so adjacent words merged into one line count as misses: these numbers compare checkpoints, they are not the accuracy you will see on documents. ² 2077 alphanumeric care words of ICDAR-2015 test cropped from the full images with `torchocr.ops.crop_quads` on their ground-truth quads, case-insensitive (not the official Task 4.3 crops). Axis-aligned crops of the same words score 0.579. ³ Decoded with its declared CTC beam search (width 10) and 6-gram character language model; 0.750 with greedy decoding (0.757 for the MJSynth checkpoint). Same ground-truth-quad protocol; Total-Text curved words use their minimum-area rectangle. End to end (`OCRPipeline.from_pretrained`, image → text; a word counts when it is detected *and* read correctly, case-insensitive, generic vocabulary; `references/end2end/evaluate.py`), with `DBNet_ResNet18_VD_Weights.ICDAR2015` and each scene recognizer, `min_text_score=0.8`: | Test set | Detection hmean | `…ICDAR2015_TOTALTEXT_MLT19` (v0.3) | `…MJSYNTH_ICDAR2015_TOTALTEXT_MLT19` | | --- | --- | --- | --- | | ICDAR-2015 (street scenes) | 0.809 | 0.649 | **0.665** | | Total-Text (curved text) | 0.696 | 0.585 | **0.585** | | FUNSD (scanned forms) | 0.826 | 0.715 | **0.722** | Without `min_text_score`, the v0.3 recognizer scores 0.621 / 0.566 / 0.711. `min_text_score=0.8` drops regions read with low confidence; the value maximizes end-to-end hmean on the ICDAR-2015 *training* split. The converted PaddleOCR weights reach 0.219 on ICDAR-2015 and 0.17–0.18 detection hmean on FUNSD, whose word-level boxes do not match their line-level output. ```python from torchocr import OCRPipeline, load_image pipeline = OCRPipeline.from_pretrained( "DBNet_ResNet18_VD_Weights.ICDAR2015", "CRNN_ResNet34_VD_Weights.MJSYNTH_ICDAR2015_TOTALTEXT_MLT19", device="cuda" ) # the recognizer's beam search + language model is set up automatically pipeline.min_text_score = 0.8 doc = pipeline(load_image("photo.jpg")) # doc.text, doc.polygons ``` ## Evaluation protocol Detection: official ICDAR-2015 IoU protocol (one-to-one greedy matching at IoU > 0.5, `###` don't-care filtering) on rotated quads, implemented by `torchocr.metrics.DetectionHmean` and verified identical to PaddleOCR's `DetectionIoUEvaluator`. Pre/post-processing is exactly `weights.transforms()` and `weights.meta["postprocess"]`. Every number is reproducible with: ```bash python references/detection/evaluate.py --root --weights python references/recognition/evaluate.py --root --weights CRNN_ResNet34_VD_Weights.PPOCR_SERVER_V2 ``` ### The `ICDAR2015` checkpoints All were fine-tuned by torchocr on *training* splits only (ICDAR-2015, plus Total-Text and MLT-2019 for the recognizer noted below); reported weights are the **last epoch** and no checkpoint or hyper-parameter was selected on the test set. - `DBNet_ResNet18_VD_Weights.ICDAR2015`: `PPOCR_SERVER_V2` fine-tuned 300 epochs with the recipe below; `box_thresh = 0.5` maximizes train-split hmean (0.794 with PaddleOCR's 0.6). - `CRNN_ResNet34_VD_Weights.ICDAR2015`: `PPOCR_SERVER_V2` fine-tuned 30 epochs on the 4467 legible training words, cropped with `crop_quads` from jittered ground-truth quads (`references/recognition/train.py`). It keeps PaddleOCR's Chinese + Latin charset but saw only Latin scene text, so Chinese accuracy is no longer guaranteed. - `CRNN_ResNet34_VD_Weights.ICDAR2015_TOTALTEXT_MLT19`: `PPOCR_SERVER_V2` fine-tuned 10 epochs (bf16) on 66k Latin scene words (ICDAR-2015 train ×2, Total-Text train, MLT-2019 Latin crops). Its language model (`charlm6_scene-883dd7cd.json`) is a case-insensitive 6-gram character model built from those *training* transcriptions only; beam width and LM weight were chosen on Total-Text *training* words. - `CRNN_ResNet34_VD_Weights.MJSYNTH_ICDAR2015_TOTALTEXT_MLT19`: the checkpoint above fine-tuned 6 more epochs (bf16), each drawing 370k words at random: the 66k real words once on average plus 300k of the 7.2M [MJSynth](https://www.robots.ox.ac.uk/~vgg/data/text/) synthetic words (14 undecodable images skipped). Same decoder and language model. - `DBNet_MobileNetV3_Large_05_Weights.ICDAR2015`, described next. #### `DBNet_MobileNetV3_Large_05_Weights.ICDAR2015` `PPOCR_V3_EN` fine-tuned for 300 epochs on the 1000 ICDAR-2015 training images (`references/detection/train.py`, PaddleOCR's augmentation recipe, 640 px crops, AdamW 1e-3 with warmup + cosine, ~80 min on one laptop GPU). The reported weights are the **last epoch**: no checkpoint was selected on the test set. `box_thresh = 0.45` maximizes hmean on the **training** split (`references/detection/calibrate.py`) and was applied once to the test split; with PaddleOCR's default `box_thresh = 0.6` the same checkpoint scores 0.688. Intended inputs: incidental scene text in street-level photos, Latin script, evaluated at 736 × 1280. Expect lower recall on dense documents and on non-Latin scripts. ## Provenance and licenses - `PPOCR_*` checkpoints are converted from [PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR) releases (Apache-2.0) using the build-time converters in torchocr's `scripts/`, whose recipes are adapted from [PaddleOCR2Pytorch](https://github.com/frotms/PaddleOCR2Pytorch) (Apache-2.0). The PP-OCRv3 files hold the `Student` network that PaddleOCR exports for inference. - The `ICDAR2015` checkpoint derives from `PPOCR_V3_EN` (Apache-2.0) and was fine-tuned on the [ICDAR 2015 Incidental Scene Text](https://rrc.cvc.uab.es/?ch=4) training set, which its organizers release for research use. Check that your use complies with those terms. - `CRNN_ResNet34_VD_Weights.MJSYNTH_ICDAR2015_TOTALTEXT_MLT19` was additionally trained on MJSynth (Jaderberg et al., 2014), released by the Oxford Visual Geometry Group for research use.