Image-Text-to-Text
MLX
Thai
English
ocr
thai
lora
mlx-vlm
dots.mocr
handwriting

Thai OCR LoRA for dots.mocr (BoomBigNoseLab/thai-ocr-dots-mocr-lora)

Built with dots.mocr.

Independent community fine-tune by BoomBigNoseLab; not affiliated with or endorsed by rednote hi lab / Xingyin Information Technology, the dots.mocr authors.

A LoRA adapter that fine-tunes dots-studio/dots.mocr (~3B: 1.2B vision encoder + 1.7B Qwen2 LM) for Thai OCR of printed documents, phone photos of documents and handwriting. Trained on an Apple-silicon MacBook with mlx-vlm 0.8.0.

πŸ‡ΉπŸ‡­ ΰΈͺΰΈ£ΰΈΈΰΈ›ΰΈ ΰΈ²ΰΈ©ΰΈ²ΰΉ„ΰΈ—ΰΈ’: LoRA adapter ΰΈͺำหรับ dots.mocr ΰΈ—ΰΈ΅ΰΉˆΰΈΰΈΆΰΈΰΉ€ΰΈžΰΈ΄ΰΉˆΰΈ‘ΰΉƒΰΈ«ΰΉ‰ΰΈ­ΰΉˆΰΈ²ΰΈ™ΰΈ ΰΈ²ΰΈ©ΰΈ²ΰΉ„ΰΈ—ΰΈ’ΰΉ„ΰΈ”ΰΉ‰ΰΈ”ΰΈ΅ΰΈ‚ΰΈΆΰΉ‰ΰΈ™ (เอกΰΈͺารพิฑพ์ ΰΈ ΰΈ²ΰΈžΰΈ–ΰΉˆΰΈ²ΰΈ’ΰΉ€ΰΈ­ΰΈΰΈͺΰΈ²ΰΈ£ แΰΈ₯ΰΈ°ΰΈ₯ΰΈ²ΰΈ’ΰΈ‘ΰΈ·ΰΈ­) ฝยกด้วฒ mlx-vlm ΰΈšΰΈ™ MacBook ΰΈšΰΈ™ΰΈŠΰΈΈΰΈ”ΰΈ—ΰΈ”ΰΈͺอบ 150 ΰΈ ΰΈ²ΰΈžΰΈŠΰΈΈΰΈ”ΰΉ€ΰΈ”ΰΈ΅ΰΈ’ΰΈ§ΰΈΰΈ±ΰΈ™ΰΈ—ΰΈΈΰΈΰΉ‚ΰΈ‘ΰΉ€ΰΈ”ΰΈ₯ ΰΉ„ΰΈ”ΰΉ‰ CER 0.621 (baseline 0.682) ΰΈ­ΰΉˆΰΈ²ΰΈ™ΰΉ€ΰΈ­ΰΈΰΈͺารพิฑพ์/ΰΈ ΰΈ²ΰΈžΰΈ–ΰΉˆΰΈ²ΰΈ’ΰΈ”ΰΈ΅ΰΈ‚ΰΈΆΰΉ‰ΰΈ™ΰΈŠΰΈ±ΰΈ” ΰΉΰΈ•ΰΉˆΰΈ₯ΰΈ²ΰΈ’ΰΈ‘ΰΈ·ΰΈ­ΰΈ’ΰΈ±ΰΈ‡ΰΉΰΈ’ΰΉˆΰΈ₯ΰΈ‡ΰΉ€ΰΈ₯็กน้อฒ แΰΈ₯ΰΈ° Typhoon OCR2-2B ΰΈ’ΰΈ±ΰΈ‡ΰΈ”ΰΈ΅ΰΈΰΈ§ΰΉˆΰΈ²ΰΉƒΰΈ™ΰΈ„ΰΈ­ΰΈ₯ΰΈ±ΰΈ‘ΰΈ™ΰΉŒ clean-ref Β· ΰΈͺัญญาอนุญาต CC-BY-NC-SA-4.0 (ΰΉƒΰΈŠΰΉ‰ΰΉ€ΰΈžΰΈ·ΰΉˆΰΈ­ΰΈΰΈ²ΰΈ£ΰΉ„ΰΈ‘ΰΉˆΰΉΰΈͺΰΈ§ΰΈ‡ΰΈ«ΰΈ²ΰΈΰΈ³ΰΉ„ΰΈ£ΰΉ€ΰΈ—ΰΉˆΰΈ²ΰΈ™ΰΈ±ΰΉ‰ΰΈ™)

Intended use

  • Transcribing Thai (and mixed Thai/English) printed documents, phone photos of documents and short handwritten lines into plain text, on Apple-silicon Macs via MLX.
  • Research and comparison of Thai OCR systems.
  • Not intended for: extracting personal sensitive data without a lawful basis, counterfeiting or fraud, or unauthorised bulk digitisation of copyrighted publications (see the dots.mocr License Agreement Β§3.3 and Β§5.2). Outputs can contain errors and hallucinated text (especially on handwriting) and must be checked before use in any decision.

Model details

Item Value
Base dots-studio/dots.mocr (MIT + dots.mocr License Agreement)
Method LoRA via mlx_vlm.train (mlx-vlm 0.8.0); language-model layers only (q, k, v, o, gate, up, down of all 28 layers); vision tower frozen
LoRA rank 16, alpha 32 (scale 2.0), dropout 0.05
Optimiser AdamW, lr 1e-4, 50 warmup steps, cosine β†’ 1e-5, batch 1, completion-only loss, gradient checkpointing
Steps 3,600 steps (1 epoch over 3,600 samples); released checkpoint = step 3000
Images ≀ 1.0 MP (β‰ˆ1280 visual tokens) for training and inference
Selection best checkpoint by CER on a fixed 60-item validation split (never the test set): step 3000 val CER 0.1052 (baseline 0.1821)

Checkpoint candidates (val60 CER, rep-penalty 1.05 + loop cleanup): 1800 0.1165 Β· 2400 0.1179 Β· 3000 0.1052 Β· 3600 0.1072 Β· soup(2400+3000+3600) 0.1092. A handwriting-heavy stage-2 continuation (val60 0.1177 / 0.1221) did not beat it and is not released.

Training data

Source License How it was used
mekpro/ocr_th Apache-2.0 1,200 synthetic rendered Thai documents (half photo-augmented); 60 held out for test (by exact text)
iapp/thai_handwriting_dataset (shards 0000–0003) Apache-2.0 (as declared by iApp; see note below) 900 handwriting lines (30% photo-augmented): 821 from shard 0000 (= BEST 2019 handwriting, NECTEC) and 79 from shards 0001–0003 (Wang "Thai Handwritten Free Dataset"); 40 other texts held out for test
Synthetic Thai documents (own generator) β€” 1,500 memos / tables / receipts / forms / paragraphs / scene text (45% photo-augmented, 15% Thai digits)
↳ text source: Thai Wikipedia (20231101.th, shard 0) CC-BY-SA-3.0 / GFDL (per dataset card) sentences rendered into the synthetic docs
↳ text source: training-split texts of mekpro/ocr_th and iApp handwriting Apache-2.0 lines reused as synthetic text (test/val texts excluded)
↳ names / provinces: PyThaiNLP corpus (Thai names corpus by Korkeat Wannapat; provinces list) CC-BY-SA-4.0 (names) / CC0-1.0 (provinces) filler names and places in forms / memos
↳ fonts: 30 Google Fonts Thai families OFL-1.1 rendering only (fonts not redistributed)

Note on iApp handwriting provenance: iApp redistributes BEST 2019 (NECTEC, original page) and the Wang dataset (original page) under Apache-2.0. We could not independently confirm the original licenses of those two upstream sources.

Not used for training: typhoon-ai/ThaiOCRBench and openthaigpt/thai-ocr-evaluation (test only); pepder/thai-ocr-text-images (excluded β€” file naming overlaps the openthaigpt eval set); Korn1886/thai_handwriting_dataset (byte-identical copy of the iApp set); datasets without a declared license.

How to use (MLX, Apple silicon)

pip install "mlx-vlm==0.8.0"
huggingface-cli download BoomBigNoseLab/thai-ocr-dots-mocr-lora --local-dir thai-ocr-lora
from PIL import Image
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

# evaluated with base revision 7c53434cd403d2e7f5f1d745eb2a2467a6473423
model, processor = load("dots-studio/dots.mocr", adapter_path="thai-ocr-lora",
                         revision="7c53434cd403d2e7f5f1d745eb2a2467a6473423", trust_remote_code=True)
config = model.config if isinstance(model.config, dict) else model.config.__dict__

# resize to <= 1.0 MP, as in training
img = Image.open("page.jpg").convert("RGB")
w, h = img.size; max_px = 1_003_520
if w * h > max_px:
    r = (max_px / (w * h)) ** 0.5
    img = img.resize((max(28, int(w * r)), max(28, int(h * r))), Image.BICUBIC)
img.save("page_small.png")

prompt = apply_chat_template(processor, config, "Extract the text content from this image.", num_images=1)
out = generate(model, processor, prompt, image=["page_small.png"],
               max_tokens=1280, temperature=0.0, repetition_penalty=1.05)
print(out.text)

Recommended post-processing (used for all reported numbers of this model) β€” collapse runaway repetition loops:

import re
PAT = re.compile(r"(.{2,80}?)(?:\s*\1){7,}", re.DOTALL)
def collapse(s):
    prev = None
    while prev != s:
        prev, s = s, PAT.sub(lambda m: m.group(1) if re.search(r"[\u0e01-\u0e4eA-Za-z0-9]", m.group(1)) else m.group(0), s)
    return s

Evaluation

subset150 = a fixed, stratified 150-image subset (seed 42) of a frozen 444-image Thai test set (built once before training and never changed). Composition: 81 ThaiOCRBench items (56 text recognition, 11 handwritten content extraction, 14 full-page OCR), 35 openthaigpt/thai-ocr-evaluation items (document, document_enth, real_document, handwritten, scene_text), 20 held-out mekpro/ocr_th synthetic documents and 14 held-out iApp handwriting scans; the last two groups were turned into phone-photo style with simulated augmentation (skew, lighting, blur, JPEG noise). Every model was scored on exactly the same 150 images with the same metric code: CER/WER after NFC normalisation, whitespace collapsing and markdown/HTML stripping; Thai WER tokenised with PyThaiNLP newmm. Lower is better.

Repetition-loop cleanup (below) is applied to our model's outputs; the same cleanup applied to every competitor is reported in a separate line.

clean-ref = subset150 (83 items) without the ThaiOCRBench Text-recognition / Handwritten items, whose references only cover the part of the image a question asked about (a full transcription gets punished with insertions). It reflects pure OCR accuracy better.

Model CER ↓ WER ↓ CER clean-ref ↓ Full test (444) CER
dots.mocr + Thai LoRA (this adapter) 0.6211 0.8702 0.1101 0.6000
dots.mocr baseline (no fine-tune) 0.6823 0.9309 0.1445 0.7835
typhoon-ocr2-2b 0.7260 0.8586 0.0928 β€”
typhoon-ocr2-0.8b 0.7416 0.9752 0.1145 β€”
OpenThai2.1-OCR-Text (iApp) 0.6585 0.9040 0.0968 0.6017
P110-ThaiOCR-4B (PorsukR) 1.7020 (0.748 after loop cleanup) 1.6662 0.2592 (0.094 after cleanup) β€”
Claude Haiku 5.5 (claude CLI) 0.8111 0.9120 0.1178 β€”
Gemini / ChatGPT not evaluated β€” β€” β€”

Per-source CER on subset150 (ours vs baseline): mekpro photos 0.0463 vs 0.0844 Β· openthaigpt-eval 0.0788 vs 0.1328 Β· ThaiOCRBench 0.8026 vs 0.8732 Β· iApp handwriting photos 0.4855 vs 0.4457 (worse).

With the same repetition-loop cleanup applied to every model, subset150 CER: baseline 0.6395, typhoon-ocr2-2b 0.7255, typhoon-ocr2-0.8b 0.7416, OpenThai 0.6585, P110 0.7484, Claude Haiku 0.8111; ours 0.6211 (0.7108 without cleanup).

Not part of this release: an all-local MBR consensus ensemble (this adapter + typhoon-ocr2-2b + OpenThai + P110) reached subset150 CER 0.5294 / clean-ref 0.0372; it uses other models and is not what this repo contains.

Train/test overlap check: no test image (MD5) and no exact test reference text appears in the training set. However, 4 subset150 references (3 mekpro-heldout items and 1 short openthaigpt-eval boilerplate line) appear as substrings inside longer training documents, because mekpro reuses template sentences. Excluding those 4 items: ours 0.6290 vs baseline 0.6910 CER (146 items), so the gap is unchanged.

Limitations

  • Handwriting did not improve: iApp handwriting photos CER got worse on subset150 (0.4457 β†’ 0.4855, 14 items) and only slightly better on the full test (0.529 β†’ 0.515, 40 items). typhoon-ocr2-2b is much stronger on handwriting (subset150 0.1721) and still wins on clean-ref (0.0928 vs 0.1101).
  • typhoon-ocr2-2b also has a lower subset150 WER (0.8586 vs 0.8702); this adapter leads only on overall CER among single models.
  • The single best single-model CER here (0.6211) is not state of the art in general; it is one overnight benchmark on 150 images.
  • ThaiOCRBench Text-recognition / Handwritten references are partial (question answers), which inflates CER for every full-transcription model β€” see the clean-ref column.
  • Decoding is not identical across models: this adapter uses greedy + repetition_penalty=1.05 + loop cleanup; competitors used their model-card defaults (the same loop cleanup is reported for all of them above).
  • Gemini and ChatGPT were not evaluated (CLI not signed in / usage limit).
  • Single overnight run (1 epoch, no hyper-parameter sweep); the validation CER was still improving.
  • Inherits the base model's acceptable-use restrictions (dots.mocr License Agreement Β§3.3: no fraud/counterfeiting, no unauthorised extraction of personal sensitive information, no unauthorised bulk digitisation of copyrighted publications).

License

The adapter is released under CC-BY-NC-SA-4.0 (Attribution-NonCommercial-ShareAlike 4.0): non-commercial use only.

Why non-commercial: 821 of the 900 handwriting training lines come from BEST 2019 (NECTEC), redistributed by iApp under Apache-2.0. We could not confirm the original BEST 2019 terms, and earlier NECTEC BEST corpora were released under CC BY-NC-SA. Until those upstream terms are confirmed, we release this adapter as non-commercial. The ShareAlike part also covers the CC-BY-SA training text (Thai Wikipedia, CC-BY-SA-3.0/GFDL; PyThaiNLP Thai names corpus, CC-BY-SA-4.0). Other inputs: mekpro/ocr_th (Apache-2.0), fonts (OFL-1.1).

Use of the adapter also requires the base model dots-studio/dots.mocr, which is governed by the MIT License and the dots.mocr License Agreement (copy included in this repo as dots.mocr LICENSE AGREEMENT). Built with dots.mocr.

Citation / attribution

Built with dots.mocr. If you use this adapter, please also cite the base model and the benchmark used:

ThaiOCRBench: see the citation on https://huggingface.co/datasets/typhoon-ai/ThaiOCRBench . Training data credits: mekpro/ocr_th, iApp Technology (Thai handwriting dataset; BEST 2019 by NECTEC; Wang Thai Handwritten Free Dataset), Wikipedia contributors, PyThaiNLP / Korkeat Wannapat (Thai names corpus), Google Fonts.

Downloads last month

-

Downloads are not tracked for this model. How to track
MLX
Hardware compatibility
Log In to add your hardware

Quantized

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for BoomBigNoseLab/thai-ocr-dots-mocr-lora

Adapter
(1)
this model

Datasets used to train BoomBigNoseLab/thai-ocr-dots-mocr-lora

Paper for BoomBigNoseLab/thai-ocr-dots-mocr-lora