# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM
processor = AutoProcessor.from_pretrained("lschiemanowski/tablereader-lightonocr-v0")
model = AutoModelForMultimodalLM.from_pretrained("lschiemanowski/tablereader-lightonocr-v0", device_map="auto")tablereader-lightonocr-v0
LightOnOCR-3-4B fine-tuned to transcribe a crop of a printed
table into a fixed HTML format: a # Title line when a title is printed, a table with thead for printed
column headings, data-level for the printed indentation of row labels, <sup> for footnote markers and
tfoot for footnote lines (transcription format,
contract revision 1).
It is the model behind tablereader, which detects the tables
in a PDF, crops them over their titles and footnotes, runs this model, and converts the transcription into
table JSON. release.json records the settings the model was trained with: crop preparation, renderer,
detector, image size and generation settings. Use it through tablereader so crops are prepared the same way:
uv run tablereader read report.pdf --pages 13-20
Prompt
The image alone, with no system prompt or text, scaled so its longest edge is at most 2048 pixels; thinking disabled, temperature 0, up to 16,000 new tokens.
Training
LoRA (rank 16, alpha 32) on every language-model projection, the vision encoder frozen; 3 epochs at learning rate 1e-4 on 995 table crops from 107 publication series (UK public-sector reports, US federal and historical statistics, German budget annexes and others), digital and scanned. The targets are GPT-6.1 Terra transcriptions that parse into tables without blocking issues; they were not reviewed by a person. The training crops include some that the scannedtables dataset holds back for rights or privacy reasons. The adapter is merged into the bf16 weights.
Evaluation
1,006 validation crops from series not seen in training, scored as the share of printed values whose row path,
column path and text match the reference transcription exactly (metric records-1):
| All | Digital PDFs | Scans |
|---|---|---|
| 0.797 | 0.835 | 0.511 |
72% of crops are read without a difference. The model does not yet answer NOT_A_TABLE for crops without a
table, and scans read much worse than digital PDFs. A self-training round on 7,698 of its own
transcriptions did not improve these scores.
License
Apache 2.0, as the base model.
- Downloads last month
- -
Model tree for lschiemanowski/tablereader-lightonocr-v0
Base model
lightonai/LightOnOCR-3-4B
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("image-to-text", model="lschiemanowski/tablereader-lightonocr-v0")