azeranex/kemolate-ocr

Kemolate OCR

Reads the lettering in comic balloons — Japanese, Korean, and simplified and traditional Chinese — from a crop of one block of text. It is the default reader in Kemolate, an app for translating comics, and runs there in the browser or desktop window on the reader's own machine.

It is a retrain of manga-ocr: the same vision encoder-decoder, with its tokenizer extended from 6,144 tokens to 11,331 for hangul and Chinese, trained on lettering from manga, manhwa and manhua.

Builds

Each training run is published as a tag on this repository, and main holds the newest. Use a tag, not main: a build is an encoder, a decoder and a tokenizer that only work together, and main moves on with the next run.

Tag	Date	What changed
out26	2026-10-02	About 3,800 more real Chinese labels, from some twenty letterers no earlier run had seen

Files

The ONNX export is laid out for raw sessions rather than for transformers.js' pipeline: a cacheless decoder, with the encoder-to-decoder projection folded into it.

File	Precision	Size
encoder_model.onnx, decoder_model.onnx	fp32	328 MB, 142 MB
encoder_model_fp16.onnx, decoder_model_fp16.onnx	fp16	164 MB, 71 MB
encoder_model_quantized.onnx, decoder_model_quantized.onnx	int8	83 MB, 36 MB
tokenizer.json and the configs	—	—

• fp32 is the reference. fp16 has its inputs and outputs kept fp32, so it is fed and read exactly as fp32 is, and keeps each layer norm's variance in fp32 (the encoder's squared activations would overflow fp16). It read 499 of 500 test crops exactly as fp32 did. int8 is for CPU and WebAssembly, where integer arithmetic is the cheap kind.
• tokenizer.json was made from the checkpoint's BertJapaneseTokenizer (vocab.txt and added_tokens.json), which transformers.js cannot read; every one of the 11,331 ids decodes as the original does.

Reading

• Input: one crop, resized to 224 × 224 RGB and normalised with mean and standard deviation 0.5 (preprocessor_config.json). Vertical and horizontal lettering both read from the same model.
• Decoding: greedy, from token 2 until token 3; Kemolate stops at 300 tokens. The numbers below are greedy. generation_config.json describes the four-beam search the training pipeline evaluates with, which can differ on a hard balloon.

Results

Character error rate (CER, lower is better) and exact matches.

On real pages no run trained on — the comparison between builds:

Benchmark	Crops	out26	Previous build
Chinese chapters, eleven books	2,025	0.049 CER	0.054 (out25)
Korean books, six	975	tied with out25	out25: 0.065 CER, 74% exact

On the Chinese set out26 wins 123 crops to 88, and nine of the eleven books read better, the largest gains on letterers absent from its new labels. Japanese real pages and a held-out Chinese title read the same as out25.

By precision, 100 crops a language, greedy decode as Kemolate runs it. Every column but ja-REAL is this run's own synthetic evaluation slice, so this table compares the files of one build with each other, never one build with another:

	ja-REAL	ja	ko	zh-Hans	zh-Hant
int8	0.015 / 95%	0.058 / 77%	0.079 / 78%	0.098 / 82%	0.089 / 84%
fp32	0.020 / 95%	0.057 / 77%	0.074 / 79%	0.098 / 82%	0.097 / 83%

Limitations

• Korean comes back without word spacing. No token is a space, so the tokenizer has no way to write one.
• Long blocks. Crops of 23 or more characters in a box no taller than it is wide have been left out of training, so a long paragraph — a disclaimer, a credits block — reads worse than a balloon.
• Ellipses are written as three periods in Korean and as … in Japanese and Chinese, as the training labels write them.
• One block at a time: it does not find text on a page. Kemolate pairs it with a comic text detector.

Training

Initialised from kha-white/manga-ocr-base, with its vocabulary extended (every existing id kept) to cover hangul and simplified and traditional Chinese, and later 152 hangul syllables that comic sound effects use and prose almost never does (읏, 흣, 큿 among them). Fine-tuned on synthetic lettering rendered in many typefaces, plus real lettering cropped from comics and labelled for this project: about 2,800 Korean and 6,900 Chinese labels across the runs up to out26.

Licence and attribution

Released under CC BY-SA 4.0, because the decoder descends from tohoku-nlp/bert-base-japanese-char-v2, which is licensed CC BY-SA 4.0 and asks that adaptations share alike.

• Encoder: facebook/deit-base-patch16-224, Apache-2.0.
• Decoder: tohoku-nlp/bert-base-japanese-char-v2, CC BY-SA 4.0.
• Starting weights: kha-white/manga-ocr-base by Maciej Budyś, Apache-2.0, itself trained on Manga109-s.