Thareah/ThaOCR-models

ThaOCR models — Khmer + English document OCR (ONNX, CPU / phone size)

Five-stage document OCR pipeline. Every model is plain ONNX (opset 17) and runs with onnxruntime on CPU, in the browser (ONNX Runtime Web) or on a phone (ORT Mobile). models.json is the manifest the ThaOCR backend reads: file/vocab are relative to this folder, defaults names the model used for each stage.

stage	default model	file	size	what it does
layout	layout_yolo11n_v4_22cls (_int8: 3.1 MB, less exact)	layout/layout_v4_n_22cls.onnx	10.9 MB	22 region classes: text, title, section_header, list, table, figure, caption, header, footer, page_number, footnote, formula, signature, stamp, logo, qr_code, barcode, photo, handwriting, checkbox, form_field, decoration. Input [1,3,1024,1024].
textdet	db_ppocrv5_khmer_v2	textdet/db_v2.onnx	4.8 MB	DB text-line detector (PP-OCRv5 mobile, fine-tuned for Khmer). Input [1,3,H,W], H and W multiples of 32.
rec	crnn_m_v2_stamp2 / _int8	rec/crnn_m_v2_stamp2*.onnx + .vocab.json	19 MB / 4.8 MB	CRNN-CTC line recogniser, Khmer + English, 195 symbols, line height 48, dynamic width [N,1,48,W].
font	font_cnn_v2	font/font_cnn_v2.onnx	1.9 MB	per line: font family (29 Khmer/Latin families), weight, slant, em size. Input 32x256 grey.
structure	struct_v1	structure/struct_v1.onnx	4.0 MB	small Transformer over the page's lines: role, paragraph start, list level, bullet, reading order.

Older versions kept for comparison: layout v1 (YOLO11s, 12 classes; .onnx + .pt), layout v2 (YOLO11s round 2, 21 classes), v2 YOLO11n and v3 YOLO11n, textdet v1 and the original PP-OCRv5 detector, recogniser v1, v2, v2.1 (_hw), v2.2 (_stamp), v2.4 (_stamp3), font v1.

How the models were trained

Layout detector (Ultralytics YOLO11, imgsz 1024, AdamW, cosine LR, mosaic 0.6, no horizontal flip)

1. v1 — YOLO11s, 12 classes, trained in the earlier Khm-Dataset-Creation repo on synthetic Khmer pages, real Khmer PDF pages with heuristic labels and DocLayNet.
2. v2 round 1 (layout_r1) — YOLO11s from v1 weights, 21-class taxonomy, 30 epochs (stopped at 26 by the time budget), lr0 1e-3, batch 8. Val mAP50 0.964, mAP50-95 0.834.
3. v2 round 2 (layout_r2, layout_v2_r2_21cls.onnx) — continued from round 1 on the round-2 set (32,290 train / 923 val pages), lr0 5e-4. mAP50 0.962, mAP50-95 0.834.
4. v2 nano (layout_v2_n_21cls.onnx) — YOLO11n (2.6 M params) from the COCO-pretrained yolo11n.pt on the same round-2 set, 31 epochs, lr0 2e-3, batch 16. mAP50 0.956, mAP50-95 0.798.
5. v3 nano (layout_v3_n_22cls.onnx, default) — fine-tune of v2 nano on layout_v3 (18,000 train / 1,100 val: 10,000 new docgen v3 pages with the decoration class + 8,000 pages replayed from round 2), 37 epochs, lr0 6e-4. mAP50 0.972, mAP50-95 0.842.
6. v4 nano (layout_v4_n_22cls.onnx, default) — fine-tune of v3 nano on layout_v4 (19,000 train / 1,200 val: 9,000 new docgen v4 pages — Khmer government notices and letters with emblem letterheads and mottos, handwritten reference numbers and dates, label-value blocks, tables with spanning group rows, stamps printed over signatures and dates — + 10,000 pages replayed from layout_v3), 57 epochs in a 3 h budget, lr0 0.0003. On the v4 validation set: mAP50 0.974, mAP50-95 0.850 (v3 on the same set: 0.935 / 0.802). Largest gains: handwriting 0.42 → 0.75, header 0.70 → 0.93, text 0.72 → 0.87, stamp 0.76 → 0.82, signature 0.66 → 0.73 (mAP50-95). layout_v4_n_22cls_int8.onnx is the same network after onnxruntime dynamic weight quantisation (3.1 MB); boxes are visibly less tight on tables and forms, so the app does not pick it by default.

Layout training data (round 2 set): docgen v2 synthetic pages (12,000 Cambodian + US/English documents: letters, invoices, forms, ID cards, exam sheets, receipts, reports…), DocLayNet v2 (6,895 train / 647 val real pages, 11 classes + page_number, plus degraded scan/photo copies), real Khmer PDF pages with silver heuristic labels (548 train / 173 val, each also degraded), SignverOD signatures and stamp scans (round 3 of the v1 repo), D4LA real US scans (2,834 train, only form/invoice/letter/memo; missing classes filled by a teacher model, training/layout/pseudo_label.py), CommonForms (1,517 pages, checkbox boxes only), docgen exam answer sheets (3,000 pages). Taxonomy in training/taxonomy.py.

Text-line detector (DB, PP-OCRv5 mobile backbone, 1.16 M params)

det/train_textdet.py converts the original PP-OCRv5 mobile detector ONNX to PyTorch (onnx2torch) and fine-tunes the probability map with DB shrunk kernels (r = 0.4) so the original unclip post-processing still works.

• v1 — fine-tuned on synthetic Khmer pages + real Khmer PDF pages + DocLayNet text-layer line boxes.
• v2 (db_v2.onnx, default) — 20,000 iterations, batch 16, --robust augmentation, from the textdet_robust checkpoint; loss weights synth 0.45 / Khmer PDF 0.20 / DocLayNet 0.35. Data: 15,000 synthetic train pages (docgen v2 + exam sheets + handwriting lines) / 750 val, DocLayNet 6,895 / 647, Khmer PDF 548 / 173. Final F1: synthetic 0.894, DocLayNet 0.880, Khmer PDF 0.697 (0.690 low-res).

Text recogniser (CRNN-CTC, arch "M": conv 48-96-192-256, proj 320, BiLSTM 2x256, 4.76 M params, line height 48)

Trainer: training/rec/train.py (the Khm-Dataset-Creation trainer + packed line sets), --aug strong (GPU warp, slant, low resolution, polarity), greedy CTC decoding. Vocabulary 195 symbols (Khmer, Latin, digits, punctuation), text normalised with normalize_khmer. Lineage (each step initialised from the previous best checkpoint):

1. v1 (crnn_m_v1) — 6 epochs on mix_print_scene: synthetic print sections S01–S08 (200k lines sampled; news corpus rendered with 21 open-licence Khmer fonts + Khmer OS fonts, print/scan effects) + synthetic scene/signboard sets (dataset_v2 scene/heavy/medium/clean, nf_mix, hard, nf_hard, long). 739k lines/epoch.
2. expA — 2 epochs adding real English lines from DocLayNet text layers (rec_doclaynet, 90,860 train lines).
3. v2 (crnn_m_v2) — 6 epochs, lr 1e-3, on final_v2_hand: everything above + docgen printed lines (99,869) + synthetic handwriting lines from docgen exam sheets (69,081, Khmer handwriting fonts) + docgen form handwriting (11,905). 1.008 M lines/epoch. Val CER: synthetic print 0.22 %, English real 0.30 %, synthetic handwriting 4.8 %.
4. v2.1 (crnn_m_v2_hw) — 3 epochs, lr 5e-4, on final_v2_hand_real: + real Khmer handwriting from Thareah/khmer-handwriting (2,311 train lines, x8 oversampled) and SoyVitou/khmer-handwritten-dataset-4.2k (3,549, x4). 1.041 M lines/epoch.
5. v2.2 (crnn_m_v2_stamp) — 2 epochs with STAMP_AUG=0.12: training/rec/stamp_aug.py draws grey ring arcs and stamp strokes over 12 % of the crops so text under official red stamps is still read.
6. v2.3 (crnn_m_v2_stamp2, default) — 4 epochs, lr 1.2e-4, same data and stamp augmentation, from v2.1. Val CER: synthetic print 0.216 %, English real 0.29 %, docgen 7.5 %, synthetic exam handwriting 4.5 %, real handwriting Thareah 21.3 % / SoyVitou 9.8 %, owner's real document dev set 12.7 %.
7. v2.4 (crnn_m_v2_stamp3) — 4 more epochs, lr 1e-4. Slightly better on every val set (print 0.213 %, real_dev 12.6 %); not yet made the default. int8 variants are dynamic-quantised exports of the same checkpoints (training/rec/export.py).

Font classifier (font/font_cnn_v2.onnx, FontNet, ~0.45 M params)

Trained entirely on synthetic line crops rendered on the fly (training/font/gen.py: Khmer news corpus + English sentences, PIL + HarfBuzz shaping, 29 font families: all Khmer OS faces, Noto Sans/Serif Khmer, Hanuman, Kantumruy Pro, Nokora, Odor Mean Chey, Chenla, Preahvihear, Suwannaphum, the Khmer display fonts (Koulen, Dangrek, Bayon, Angkor, Taprom) and Latin serif / sans; synthetic bold by dilation and italic by shear; neighbour-line intrusion, blur, noise, JPEG, paper tint). v1: 40 epochs x 400 steps x batch 256. v2: 24 epochs from v1 adding the em-size regression head. Val (8,000 held-out crops): family 92.0 %, weight 98.0 %, slant 99.9 %, size error median 1.7 %.

Structure model (structure/struct_v1.onnx, StructureNet, ~0.6 M params)

4-layer Transformer (d 160) over per-line geometric + text features, with a pointer head for reading order. Trained 30 epochs, batch 16, on docgen v3 pages with struct labels (10,000 train / 500 val synthetic pages). Val: role accuracy 0.988, paragraph-start F1 0.971, successor accuracy 0.964, Kendall tau 0.943.

Datasets

data	type	used by	licence
Khmer news corpus (5,601 articles, 257k lines; scraped by the owner for research) rendered with open Khmer fonts	synthetic	rec, font	text: research use; fonts OFL / LGPL
docgen v2 / v3 / v4 / exam synthetic pages (Project Gutenberg English, Faker identities, procedural logos/stamps/figures)	synthetic	layout, textdet, rec, structure	generated
DocLayNet v2	real	layout, textdet, rec (English lines)	CDLA-Permissive 1.0
D4LA (form/invoice/letter/memo)	real	layout	Apache-2.0 (mirror), page content third-party
CommonForms (checkbox boxes)	real	layout	Apache-2.0
Real Khmer PDF pages (government/NGO PDFs crawl, heuristic labels)	real	layout, textdet	research use
SignverOD signatures, mapo80 stamps	real	layout	Apache-2.0 / CC-BY-4.0
Thareah/khmer-handwriting, SoyVitou/khmer-handwritten-dataset-4.2k	real	rec	MIT
FUNSD, KhmerEduDoc	real	evaluation only (non-commercial)	NC

Base weights: Ultralytics YOLO11 (AGPL-3.0 — the layout models inherit it), PaddleOCR PP-OCRv5 mobile detector (Apache-2.0). The owner's private exam photos were never used for training.

Licence

The repo mixes licences, so the Hub tag is license: other. Per stage:

Stage	Files	Licence
Layout	layout/*.onnx, layout/layout_v1_12cls.pt	AGPL-3.0 — fine-tuned from Ultralytics YOLO11 weights and inherits that licence
Text detection	textdet/db_v1.onnx, textdet/db_v2.onnx, textdet/db_ppocrv5_orig.onnx	Apache-2.0 — derived from (or identical to, for db_ppocrv5_orig) the PaddleOCR PP-OCRv5 mobile detector
Recognition	rec/*	owner's own models, trained from scratch
Font	font/*	owner's own models, trained from scratch on synthetic data
Structure	structure/struct_v1.onnx	owner's own model, trained from scratch on synthetic data

Training data licences are listed in the Datasets table above. FUNSD and KhmerEduDoc were used for evaluation only (non-commercial); no model weights were trained on them.

Use

import onnxruntime as ort, json
sess = ort.InferenceSession("rec/crnn_m_v2_stamp2_int8.onnx")
vocab = json.load(open("rec/crnn_m_v2_stamp2.vocab.json"))["itos"]
# grey line crop, height 48, shape [1,1,48,W] float32 in 0..1 -> greedy CTC over the output, blank id 0

Pre/post-processing (letterbox + NMS, DB unclip, width bucketing + CTC) is small numpy/OpenCV code in the ThaOCR backend (backend/app/services/stages/).