---
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
tags:
  - ocr
  - text-detection
  - text-recognition
  - openocrv2
---

# OpenOCRv2

Text detection and recognition for scene/document images, deployable with plain
`torch` + `transformers`. Three models live in one repository, in three
subfolders — download only what you need with `subfolder=`.

| Subfolder | Model | Architecture | Params | Weights |
| --- | --- | --- | ---: | ---: |
| `det/` | **OpenOCRv2_Det** | HGNetv2-b4 + DCNv2 + polygon regression head | 9.36 M | 36 MB |
| `rec_ctc/` | **OpenOCRv2_Rec_CTC** | ConvNeXt + 2D-RoPE text encoder + linear CTC head | 21.97 M | 85 MB |
| `rec_ar/` | **OpenOCRv2_Rec_AR** | Same encoder + autoregressive decoder (RoPE, KV cache) | 36.66 M | 141 MB |

## Requirements

For Windows CPU inference, install these versions in a fresh Python 3.11
virtual environment:

```powershell
pip install torch==2.14.0 transformers==5.22.2 safetensors==0.8.0 numpy==2.2.6 pillow==12.3.0 opencv-python-headless==5.0.0.93
```

For NVIDIA GPU inference, install a CUDA-enabled build of `torch` using the
[PyTorch installation guide](https://pytorch.org/get-started/locally/). Use the
same versions of the other packages.

`opencv-python-headless` is only needed for the crop step (`det.ocr(...)`).
Polygon detection (`det.detect(...)`) and recognition (`rec.recognize(...)` on
already-cropped images) work without it.

> If you see `Run pip install cv2` — that is transformers' fallback hint and the
> package name does not exist. Install `opencv-python-headless` instead.

OpenOCRv2 does not require `torchvision`. If importing the model raises
`operator torchvision::nms does not exist`, use a clean environment with the
packages above, or install a `torchvision` build that matches your `torch`.

## Usage

- **CTC (`rec_ctc`)** is the faster recognizer. Use it when throughput or latency matters most.
- **AR (`rec_ar`)** is the accuracy-focused recognizer. It decodes one token at a time, so it is slower.

Each example runs on CUDA when `torch.cuda.is_available()` is true and uses CPU
otherwise. Pass NumPy images as shown. The models move their input tensors to the
selected device, while OpenCV cropping stays on the CPU.

`det.detect()` and `rec.recognize()` already disable gradient recording for the
model forward passes. `det.ocr()` uses both methods.

### Full-page OCR

Replace `page.jpg` with a page or scene image. Each example prints the text in
reading order and each region's recognition score, text, and quadrilateral.

#### CTC (speed-focused)

```python
import numpy as np
import torch
from PIL import Image
from transformers import AutoModel

REPO = "Yesianrohn/openocrv2"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

with Image.open("page.jpg") as source:
    image = np.asarray(source.convert("RGB"))

det = AutoModel.from_pretrained(REPO, subfolder="det", trust_remote_code=True).to(device).eval()

rec = AutoModel.from_pretrained(REPO, subfolder="rec_ctc", trust_remote_code=True).to(device).eval()
result = det.ocr(image, rec)

print("CTC — text in reading order")
print("\n".join(result["text"]))
print("Score, text, quadrilateral")
for region in result["regions"]:
    print(f'{region["rec_score"]:.3f}', region["text"], region["quad"].tolist())
```

#### AR (accuracy-focused)

```python
import numpy as np
import torch
from PIL import Image
from transformers import AutoModel

REPO = "Yesianrohn/openocrv2"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

with Image.open("page.jpg") as source:
    image = np.asarray(source.convert("RGB"))

det = AutoModel.from_pretrained(REPO, subfolder="det", trust_remote_code=True).to(device).eval()

rec = AutoModel.from_pretrained(REPO, subfolder="rec_ar", trust_remote_code=True).to(device).eval()
result = det.ocr(image, rec)

print("AR — text in reading order")
print("\n".join(result["text"]))
print("Score, text, quadrilateral")
for region in result["regions"]:
    print(f'{region["rec_score"]:.3f}', region["text"], region["quad"].tolist())
```

### Detection only

Pass the full RGB image to the detector. It returns polygons in original image
pixel coordinates and a detection score for each polygon.

```python
import numpy as np
import torch
from PIL import Image
from transformers import AutoModel

REPO = "Yesianrohn/openocrv2"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

with Image.open("page.jpg") as source:
    image = np.asarray(source.convert("RGB"))

det = AutoModel.from_pretrained(REPO, subfolder="det", trust_remote_code=True).to(device).eval()
page = det.detect([image])[0]

for polygon, score in zip(page["polygons"], page["scores"]):
    print(f"{score:.3f}", polygon.tolist())
```

### Recognition only

Use one upright text line per crop. The recognizers take a list of BGR
`uint8` arrays and return `(text, confidence)` pairs in input order.

#### CTC (speed-focused)

```python
import numpy as np
import torch
from PIL import Image
from transformers import AutoModel

REPO = "Yesianrohn/openocrv2"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

crop_paths = ["line_1.jpg", "line_2.jpg"]
crops = []
for path in crop_paths:
    with Image.open(path) as source:
        rgb = np.asarray(source.convert("RGB"))
    crops.append(rgb[..., ::-1].copy())  # RGB -> BGR

rec = AutoModel.from_pretrained(REPO, subfolder="rec_ctc", trust_remote_code=True).to(device).eval()
print("CTC — text and confidence")
for path, (text, confidence) in zip(crop_paths, rec.recognize(crops)):
    print(f"{path}: {text} ({confidence:.3f})")
```

#### AR (accuracy-focused)

```python
import numpy as np
import torch
from PIL import Image
from transformers import AutoModel

REPO = "Yesianrohn/openocrv2"
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

crop_paths = ["line_1.jpg", "line_2.jpg"]
crops = []
for path in crop_paths:
    with Image.open(path) as source:
        rgb = np.asarray(source.convert("RGB"))
    crops.append(rgb[..., ::-1].copy())  # RGB -> BGR

rec = AutoModel.from_pretrained(REPO, subfolder="rec_ar", trust_remote_code=True).to(device).eval()
print("AR — text and confidence")
for path, (text, confidence) in zip(crop_paths, rec.recognize(crops)):
    print(f"{path}: {text} ({confidence:.3f})")
```

If you crop yourself, keep these two details — they are the difference between
correct output and silently degraded output:

- Quad order is TL, TR, BR, BL after `cv2.minAreaRect` + sort-by-x, not
  sort-by-angle.
- Crops taller than 1.5× their width must be rotated 90° before recognition,
  or vertical text comes back wrong.

`det.ocr` already does both. `modeling_openocrv2_det` exposes the helpers as
`polygon_to_quad` and `get_rotate_crop_image` if you want to reuse them.

### Batching

`rec.recognize(crops, batch_size=32, micro_batch_size=8)`:

- `batch_size` is the **logical** batch — it decides the padding size, and the
  attention length changes with it, so predictions depend on it.
- `micro_batch_size` is how many run per forward pass (memory only, does not
  affect output).

Accuracy figures were produced with `batch_size=32`. Changing it will shift
results slightly.

## Preprocessing contracts

These must match the pipeline the weights were frozen with, or output changes:

- **Detection:** RGB, letterbox to 1600×1600 with zero padding, ImageNet
  normalisation (mean `[123.675, 116.28, 103.53]`, std `[58.395, 57.12, 57.375]`).
- **Recognition:** aspect-preserving resize onto the ladder
  `[[64,64], [96,48], [112,40], [128,32]]` (width, height), then
  `(x/255 - 0.5) / 0.5`, zero-padded to the batch maximum.

Defaults live in each `config.json` (`preprocess`, `image_size`, thresholds) and
are applied automatically.

## Files

```text
modeling_openocrv2_det.py
modeling_openocrv2_rec_ctc.py
modeling_openocrv2_rec_ar.py
det/
├── config.json
└── model.safetensors           # 447 tensors, incl. the 14x28 polygon basis
rec_ctc/  rec_ar/
├── config.json
├── model.safetensors
└── characters.json             # 19685 / 19687-entry alphabet, bundled
```

The `auto_map` entries resolve modeling modules from the repository root.
The `subfolder` argument selects the config and weights. To load a local
download, pass the path to its root in place of `REPO` and keep `subfolder=`.

The character table ships inside each recogniser repo, so decoding needs nothing
beyond `from_pretrained`.