comma-lightonocr-3-0.8b

lightonai/LightOnOCR-3-0.8B fine-tuned for CATMuS-compliant, line-by-line transcription of medieval Latin manuscript pages: one line of output per physical written line, in reading order. CATMuS is a graphemic standard — the letters and signs as written, abbreviations kept, no expansion or normalisation.

LightOnOCR-3 is LightOn's document-OCR model, built on the Qwen3.5 vision-language architecture. This fine-tune keeps that architecture unchanged and retrains it on 9,906 pages of this collection.

Results

243 held-out pages, each in a different hand, none seen in training. CER is quoted after NFD normalisation, because the corpus writes ũ both precomposed and as u + combining tilde, and charging two edits for one glyph measures the encoding rather than the reading.

model	CER (NFD)	macro	WER (NFD)	line recall (NFD)	truncated
LightOnOCR-3-0.8B, zero-shot (same job)	0.4654	0.4557	0.9265	0.0024	7.8%
this model	0.1206	0.1018	0.3613	0.1678	0.4%
Qwen3.5-0.8B, full FT — exact control	0.1458	0.1318	0.4029	0.1672	1.6%
LightOnOCR-3-4B, full FT	0.1228	0.1036	0.3605	0.1681	0.4%
chandra-ocr-2 (5.3B), LoRA	0.1232	0.1039	0.3639	0.1668	0.8%
Qwen3.5-2B, full FT	0.1251	0.1066	0.3616	0.1647	0.8%
Qwen3.5-9B, full FT	0.1168	0.0985	0.3541	0.1751	0.4%

Greedy decoding, 3,072 max new tokens, 2,048 visual tokens per page, eval batch 4. Treat differences under 0.02 CER as ties — four seeds of one configuration in this project span 0.021.

OCR pretraining is worth 0.025 CER here, measured against an exact control. Qwen3.5-0.8B fine-tuned with the identical recipe — same 9,906 pages, learning rate, epochs, seed, effective batch, image budget, holdout, and evaluation GPU type — scores 0.1458. The only difference is the starting checkpoint. The gap is outside this project's noise floor (four seeds of one configuration span 0.021), so it is a result rather than a fluctuation.

That puts a 0.8B level with models 2.5 to 6.6 times its size: chandra-ocr-2 (0.1232), the 2B (0.1251), and the LightOnOCR-3-4B fine-tune (0.1228) — all within 0.02 of it.

Comparing across GPUs

This model and the comparison rows were scored on RTX PRO 6000 Blackwell GPUs, except the LightOnOCR-3-4B and Qwen3.5-9B rows, which were scored on H200s. That matters: the same untrained LightOnOCR-3-4B scored 0.0202 CER apart on those two GPU types (bf16 arithmetic differs and greedy decoding amplifies it), so gaps under ~0.03 between this model and the H200 rows are ties. The comparisons with the zero-shot row and the RTX-scored rows are like-for-like.

Usage

import torch
from PIL import Image
from huggingface_hub import hf_hub_download
from transformers import AutoModelForImageTextToText, AutoProcessor

REPO = "wjbmattingly/comma-lightonocr-3-0.8b"
model = AutoModelForImageTextToText.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")
processor = AutoProcessor.from_pretrained(REPO)

# The prompt is part of the model: it was trained to answer this exact instruction.
# Use it verbatim -- LightOnOCR's own empty transcription prompt is a different task.
prompt = open(hf_hub_download(REPO, "prompt.txt"), encoding="utf-8").read()

image = Image.open("page.jpg").convert("RGB")
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": prompt}]}]
text = processor.apply_chat_template(messages, tokenize=False,
                                     add_generation_prompt=True, enable_thinking=False)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=3072, do_sample=False,
                     repetition_penalty=1.1, use_cache=True)
print(processor.tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

• prompt.txt is required. This model was trained on a 11,629-character instruction, shipped in this repo; LightOnOCR-3 itself was trained on an empty prompt and emits block-labelled markdown with bounding boxes, which is not this task.
• enable_thinking=False. Passed explicitly so the generation prompt closes the thinking block regardless of which template variant is loaded.
• The image budget is in the repo. processor_config.json is capped at 2,048 visual tokens, the budget used in training and scoring. LightOnOCR-3 ships uncapped (~16k tokens); loading a stock processor shows the page at a resolution this model never trained on.
• repetition_penalty=1.1 is this project's measured serving setting: it removes most looping on sparse pages at a small cost in line recall. The table above is greedy (penalty 1.0); set 1.0 to reproduce it.
• generation_config.json sets use_cache: true. LightOnOCR-3-0.8B ships false (a training artefact), which makes generation recompute the whole sequence at every step; every number above was produced with the cache on.

Training

Full fine-tune of all 852,985,920 parameters (vision tower included), 3 epochs, 3,717 steps, learning rate 5e-05, cosine schedule, effective batch 8, bf16, 2,048 visual tokens per page, on RTX PRO 6000 Blackwell. 2 of 9,906 pages exceeded the 8,192-token length budget and were dropped (the stock-Qwen control dropped 1).

Data: comma-project/deep-jsonl.

Limits

A research artifact for cultural-heritage and digital-humanities work, not a production OCR service. Trained on one collection, it should be expected to degrade on other hands, languages, genres and centuries. It reproduces what is written, including historical spelling and language; transcribing a record is not an endorsement of its contents. Verify any name, date or figure before relying on it.