File size: 5,552 Bytes
8f60837 cc6c8e0 8f60837 001802f 8f60837 001802f 8f60837 001802f 8f60837 001802f 8f60837 001802f 8f60837 95fdbe1 8f60837 001802f 5c9d5e4 001802f 8f60837 5c9d5e4 001802f 8f60837 001802f 8f60837 001802f 8f60837 001802f 8f60837 cc6c8e0 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | ---
license: apache-2.0
base_model:
- M2key1/deepocr-v3
- M2key1/HabsburgLLM
library_name: pytorch
pipeline_tag: image-to-text
tags:
- ocr
- htr
- handwritten-text-recognition
- historical-documents
---
# DeepOCR v4.1
A line-level text recognition model for historical print and handwriting. v4.1 is [DeepOCR v4](#versions) trained further on five more collections and corrected labels. It keeps v4's subword decoder, which writes the line while the CTC head checks the candidates against the image. On unseen handwritten letters it makes 11% fewer errors than v4, and on a blank crop it writes nothing where v4 invented a line. Code, training and architecture: [DeepOCR](https://github.com/m2key1/DeepOCR).
## Files
| File | What it is |
|---|---|
| `deepocr-v4.pt` | The model weights (EMA), with every architecture setting, the decoder's config and its tokenizer stored inside |
| `union_bytes_v3.bin` | Byte-level 6-gram KenLM, for CTC beam search |
| `union_bytes_v3.dict` | The LM's token list, index-parallel to the CTC head's 261-token byte vocabulary |
## Use
```python
import torch
from PIL import Image
from infer import load_model, transcribe
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model, tokenizer = load_model('deepocr-v4.pt', device)
print(transcribe(model, tokenizer, [Image.open('line.png')], device, use_decoder=True)[0])
```
`use_decoder=True` reads with the subword decoder: it writes four candidate lines, the CTC head scores each against the image, and the best overall wins. This is the most accurate way to read with v4.1. CTC beam search with the KenLM (`transcribe_beam`, set up as on the v3 card) also works.
Input is one text line as an RGB image; it is scaled to 256 px high and read up to 6144 px wide. Four flags choose the transcription convention: `long_s` (ſ), `oblique_hyphen` (⸗), `typographic_punctuation` and `expand_abbreviations`. Set them in `infer.CONVENTION`; all off by default.
Use the DeepOCR code from commit `163c1b2` onwards. [DeepLabel](https://github.com/m2key1/DeepLabel) reads with the subword decoder as well.
## Model
461.6M parameters, fine-tuned from v4:
- 156.0M for reading the image: a grouped CNN backbone, four mixture-of-experts transformer layers (64 experts, 8 active per column, plus one shared expert) and a CTC head over bytes, so it can emit any Unicode character.
- 305.6M for the subword decoder: a 238M OLMo 3 language model distilled from [HabsburgLLM](https://huggingface.co/M2key1/HabsburgLLM), with its 48k tokenizer, extended by cross-attention to the image features.
A line reads the same whether it is processed alone or in a batch with longer lines, up to floating-point rounding.
## Evaluation
Character error rate (CER), lower is better.
| Material | Decoding | v3 | v4 | v4.1 |
|---|---|---|---|---|
| Held-out validation sample, 1,721 lines | greedy | 5.49% | 5.19% | 3.48% |
| | beam 10 + KenLM | 5.08% | 4.77% | 3.11% |
| | subword decoder | | 4.45% | 2.94% |
| Unseen handwritten letters from around 1900, 1,972 lines | beam 10 + KenLM | 9.15% | 5.94% | 5.13% |
| | subword decoder | | 4.54% | 4.03% |
| Held-out lines of the five collections added in v4.1, 2,000 lines | subword decoder | | 18.04% | 4.79% |
All models are scored on the same lines and labels. The validation sample holds lines from the collections the models train on. Earlier versions of this card used a 2,000-line sample that also held 279 lines from collections left out of training, and labels that held only a line's first word; this one leaves those collections out and uses the corrected labels, so the v3 and v4 figures differ from before.
On 300 blank crops, v4.1 returns an empty line for every one; v4 wrote text on all of them.
## Training data
Public datasets of historical print and handwriting, and line transcriptions made in DeepLabel. v4.1 adds collections in Czech, Estonian, Danish, Portuguese and English.
During training, v4.1 kept masked image columns in place, let the decoder read the unmasked features, and saw 3% blank or upside-down lines with an empty transcription.
## Limitations
- Reads single lines only; it needs a line detector or segmentation in front of it.
- A line of one or two words gives the decoder little to go on, and it may write a more common word than the one on the page.
- Upside-down lines are not read: v4.1 returns an empty line for nearly all of them. Turn the crop the right way up first.
- In runs of leader dots the decoder can write one dot more or fewer; beam search counts them more exactly.
- Reads best on material like its training data: European print and handwriting from the medieval to the modern period. Languages that are rare in the training data read clearly worse.
## Versions
| Tag | Released | Notes |
|---|---|---|
| `v4.1` | November 2026 | Current. Trained further on more collections and corrected labels |
| `v4.0` | November 2026 | First release: v3 fine-tuned with the subword decoder |
The files keep their names across versions. To load v4.0: `hf download M2key1/deepocr-v4 --revision v4.0`.
## License
Apache-2.0.
## Citation
```bibtex
@software{fleischhacker_deepocr,
author = {Fleischhacker, David},
title = {DeepOCR},
year = {2026},
url = {https://github.com/m2key1/DeepOCR}
}
```
The subword decoder is distilled from HabsburgLLM:
```bibtex
@software{fleischhacker_habsburgllm,
author = {Fleischhacker, David},
title = {HabsburgLLM},
year = {2026},
url = {https://github.com/m2key1/HabsburgLLM}
}
```
|