burapha-HMk100 — Joint Thai + Handwritten Math LoRA (Qwen2.5-VL-7B)

LoRA adapter that fine-tunes Qwen/Qwen2.5-VL-7B-Instruct on both BURAPHA-TH and HME100K (experiment e3 of THIRA — Thai Handwriting Intelligence Recognition & Analysis). Unlike the single-dataset siblings, it keeps both skills: it does not catastrophically forget Thai the way the math-only adapter does.

Model Details

• Base model: Qwen/Qwen2.5-VL-7B-Instruct, revision cc594898137f460bfe9f0759e9844b3ce807cfb5
• Adapter: LoRA, r=16, alpha=32, dropout=0.05
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Task: read a handwritten Thai character or a handwritten mathematical expression from an image crop and transcribe it

Dataset

Trained jointly on:

• BURAPHA-TH (release 2021-03-06) — https://services.informatics.buu.ac.th/datasets/Burapha-TH/ Citation: Burapha-TH: A Multi-Purpose Character, Digit, and Syllable Handwriting Dataset, Applied Sciences, 2022.
• HME100K via the Kaggle mirror used by the training pipeline (kagglehub.dataset_download("prajwalchy/hme100k-dataset")) — https://www.kaggle.com/datasets/prajwalchy/hme100k-dataset

License / usage terms: BURAPHA-TH is granted by the Burapha-TH team (Faculty of Informatics, Burapha University) for research and education use only — baselines, training/evaluation of prototypes, academic reports and publications. Commercial use is explicitly excluded; any commercial application requires contacting the dataset authors separately. Any publication using this dataset must cite the paper above. Because this adapter is trained on BURAPHA-TH, the same restriction carries over.

This repository is kept private in line with that non-commercial research use.

Evaluation

From the project's internal eval harness (runs/e3, see results.json in this repo for full detail).

Test set	N	CER	char F1	Exact match
burapha (Thai characters)	13,600	0.0400	0.9600	96.00%
hme (handwritten maths)	24,607	0.0421	0.9767	65.79%
thai_sentence (Thai prose)	2,032	0.6753	0.5930	0.89%
combined	38,207	0.0421	0.9765	76.54%

Findings

1. Joint training beats both single-dataset adapters, on their own home turf. This is the result the e3 run exists to establish:

	burapha CER	hme CER
e1 (BURAPHA only)	0.0388	0.4324
e2 (HME100K only)	3.1985	0.0425
e3 (joint)	0.0400	0.0421

e3 ties e1 on Thai characters (0.0400 vs 0.0388) and beats e2 on maths (0.0421 vs 0.0425), while each single-dataset adapter is unusable on the other's domain. Note e2's BURAPHA CER of 3.1985 — above 1.0 means it emits more erroneous characters than the reference contains; training on maths alone destroyed Thai entirely.

2. Weight-space merging is strictly worse than joint training. Merges of e1+e2 were tried at three ratios, and a three-way merge (e7) as well. None reaches e3:

configuration	burapha CER	hme CER	thai_sentence CER
e4 merge 0.7–0.3	0.0480	0.1134	0.4386
e4 merge 0.5–0.5	0.1215	0.0592	0.4133
e4 merge 0.3–0.7	0.5144	0.0483	0.4797
e7 3-way 0.34/0.33/0.33	0.1093	0.3687	0.3043
e7 3-way 0.25/0.25/0.5	0.1710	0.4429	0.2326
e3 (joint training)	0.0400	0.0421	0.6753

Every merge ratio trades one skill against the other. Do not merge these adapters — retrain jointly. (The one thing merging does buy is Thai sentences: e7 reaches 0.2326 there, better than e3's 0.6753 — but at 8–10× the maths error, which is not a trade this pipeline can accept.)

Where it is weak — read this before using it

3. Thai sentences, not just characters. BURAPHA is isolated characters. On connected Thai sentences this adapter scores CER 0.675 and 0.89% exact match — it learned Thai glyphs, not Thai text. For Thai prose use a sentence-trained adapter instead.

4. Real handwritten worksheets are much harder than HME100K. In the Learnly MVP's evaluation on real Thai student worksheets, symbol-level CER on mathematical regions was 0.451 (n=20 regions), against 0.042 on HME100K. Benchmark performance does not transfer; treat 0.042 as an in-domain number only.

5. How to read the CER numbers — this adapter is space-separated. e3 writes maths as \frac { \log _ { 2 } 6 }, one space between every token. A character-level comparison against a normally spaced reference charges every one of those spaces as an edit. In this project that artifact temporarily made e3 look worse than a prose adapter on real worksheets; with whitespace stripped, the ranking reversed. Always report symbol-level (whitespace-stripped) CER alongside raw CER when comparing this adapter to a non-space-separated decoder.

Intended use in the Learnly pipeline

e3 is the selected mathematics recogniser in a routed two-adapter configuration: Thai-script regions go to a sentence-trained Thai adapter, symbolic/mathematical regions go to e3. Routing beat either adapter alone on real worksheets (symbol-level CER 0.453 routed, vs 0.532 Thai-only and 0.507 e3-only, n=34 matched regions).

e3 also preserves LaTeX structure far better than a prose-trained reader — 34.7% of lines carry LaTeX structure under e3 versus 10.0% under the Thai adapter — which matters because downstream mathematical verification needs the structure, and character error rate cannot see it.

How to Use

import torch
from peft import PeftModel
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor

base = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen2.5-VL-7B-Instruct",
    revision="cc594898137f460bfe9f0759e9844b3ce807cfb5",
    torch_dtype=torch.bfloat16,
)
model = PeftModel.from_pretrained(base, "TLDRKKU/burapha-HMk100")
processor = AutoProcessor.from_pretrained("TLDRKKU/burapha-HMk100")

Training Procedure

• 1 epoch, cosine LR schedule, learning rate 1e-4, warmup ratio 0.03
• Effective batch size 32 (per-device 8 × grad accumulation 4)
• bf16, gradient checkpointing enabled
• Framework: PEFT 0.20.0

Related

• TLDRKKU/burapha — BURAPHA-only adapter (e1), the Thai-character baseline
• TLDRKKU/HMk100 — HME100K-only adapter (e2), the maths baseline and the catastrophic-forgetting result

---

Part of THIRA — Thai Handwriting Intelligence Recognition & Analysis.