guoguo27/handwriting-segmenter

Handwriting Segmenter (IS-Net, 4-class, ONNX)

Segments scanned quiz/test pages into background, printed, handwriting and handwriting-over-print, so student answers can be erased and the sheet redone for practice. Fine-tuned from Inoob/DIS-Handwriting-Remover (IS-Net, from xuebinqin/DIS).

Training code: https://github.com/Illvuu/handwriting-segmenter

Files

file	sha256
finetuned_e.onnx (176 MB, opset 17)	e396488615dff875476934b2d43c94ecc1d1545ad608e67be79bf8ef9d0b0087

Contract

• Input input: float32 [N, 3, H, W], RGB, pixel / 255 - 0.5. Dynamic N/H/W. Trained for 1536 px long edge.
• Output logits: float32 [N, 4, H, W], raw logits for {0 = background, 1 = printed, 2 = handwriting, 3 = overlap}.
• Recommended decision: softmax over the 4 logits, ink = p[2] + p[3] > 0.6.

import numpy as np, onnxruntime as ort
from huggingface_hub import hf_hub_download
from PIL import Image

path = hf_hub_download("guoguo27/handwriting-segmenter", "finetuned_e.onnx")
sess = ort.InferenceSession(path)
img = Image.open("page.png").convert("RGB")
img.thumbnail((1536, 1536))
x = (np.asarray(img, np.float32) / 255.0 - 0.5).transpose(2, 0, 1)[None]
logits = sess.run(["logits"], {"input": x})[0][0]
p = np.exp(logits - logits.max(0)); p /= p.sum(0)
ink = (p[2] + p[3]) > 0.6

Training

• Warm start from Inoob/DIS-Handwriting-Remover; binary side heads lifted to 4 classes.
• 111 real pairs: scanned answered pages (printer and phone) aligned to the clean original (SIFT + RANSAC + ECC), labels by differencing; a band around printed glyphs is ignored (label 255).
• 768 px crops, 12 per page, 40 epochs, AdamW; best epoch 21 by held-out ink IoU.

Evaluation (15 held-out pages, whole quizzes, threshold 0.6)

ink precision	ink recall	ink F1	IoU printed	IoU handwriting
0.848	0.834	0.841	0.732	0.726

The ignored band around printed glyphs means faint strokes and stroke edges are partly invisible to these metrics.

Limitations

• Smooth printed curves (function graphs) are often classified as handwriting, independent of colour or threshold.
• Trained on a small set of high-school math/science quizzes; other layouts, languages and scripts are untested.
• Output is a mask. Erase deterministically (e.g. fill with local background median); do not inpaint generatively, which can fabricate content on a study sheet.

License

Apache-2.0. See NOTICE in the training repository for attribution to DIS and Inoob/DIS-Handwriting-Remover.