Card: more general framing; reviewer fixes

README.md

@@ -10,20 +10,26 @@ tags:
   - page-classification
   - blank-page-detection
   - multi-label
+  - document-ai
+  - digitised-books
 ---
 
 # Page OCR flags
 
-A small image model that looks at a scanned book page **before** OCR and answers two questions:
+A small image model that looks at a scanned page **before** OCR and answers two questions:
 
 1. **Is there any text on this side of the leaf to transcribe?** (`no_text`)
-2. **Is a VLM OCR model likely to fail on this page?** (`lo_blank_write`, `lo_trunc`, `lo_loop`; trained on the failures of [LightOnOCR-3-0.8B](https://huggingface.co/lightonai/LightOnOCR-3-0.8B))
+2. **Is [LightOnOCR-3-0.8B](https://huggingface.co/lightonai/LightOnOCR-3-0.8B) likely to fail on this page?** (`lo_blank_write`, `lo_trunc`, `lo_loop`, trained on that model's real outputs)
 
-Use it to skip pages with nothing to read, to route risky pages to a different OCR setup, or to explain bad OCR output after the fact ("this page only has show-through, so the text in the output was read from the other side of the leaf").
+It is for libraries, archives and anyone else running OCR over a large number of page scans, for example to build a text corpus. Use it to skip pages with nothing to read, to send risky pages to a different OCR setup, or to explain bad OCR output after the fact ("this page only has show-through, so the text in the output was read from the other side of the leaf").
 
-MobileNetV4-conv-small, 2.5M parameters, 384 × 384 input, about **23 ms per page on one CPU thread**. A companion model, [text-page-filter](https://huggingface.co/davanstrien/text-page-filter), answers the stricter question "is this a page of body text?".
+It was trained on pages from about 3,000 public-domain books in the [Biodiversity Heritage Library](https://www.biodiversitylibrary.org/) (BHL), mostly printed in the 19th and 20th centuries, and most of the evaluation pages come from BHL too. Check it on a sample of your own pages before you rely on it.
 
-![Examples: no_text on labelled pages](assets/flags_notext.png)
+MobileNetV4-conv-small, 2.5M parameters, 384 × 384 input, about 50 pages per second on one CPU thread. A companion model, [text-page-filter](https://huggingface.co/davanstrien/text-page-filter), answers the stricter question "is this a page of body text?".
+
+![Hand-labelled example pages with the model's no_text score](assets/flags_notext.png)
+
+A score near 1 means no text. The picture page is a miss: most labelled picture-only pages score as text.
 
 ## Outputs
 
@@ -33,12 +39,12 @@ Seven independent sigmoid outputs (multi-label, not a softmax):
 |---|---|---|
 | `no_text` | No text to transcribe on this side of the leaf: blank paper, show-through only, a library stamp only | Skip OCR, or treat OCR output as suspect |
 | `lo_blank_write` | Page has no text, but LightOnOCR-3 writes text anyway (usually it reads show-through) | Explain invented output |
-| `lo_trunc` | LightOnOCR-3 stops at its 4,096-token limit (long tables, or repetition) | Route to a larger token budget or another engine |
+| `lo_trunc` | LightOnOCR-3 reaches the 4,096-token output limit we used (long tables, or repetition) | Route to a larger token budget or another engine |
 | `lo_loop` | LightOnOCR-3 repeats itself (dot leaders, repeated lines) | Route or re-run |
 | `table`, `dense` | Page has a table / dense text (soft teacher answers) | Routing hints |
 | `faint` | **Experimental, do not use** (see limitations) | – |
 
-![Examples: LightOn risk outputs](assets/flags_lighton.png)
+![Example pages where LightOnOCR-3 truncated, looped or wrote text on a no-text page, with the model's scores](assets/flags_lighton.png)
 
 ## How to use
 
@@ -61,17 +67,15 @@ The whole page is squashed to 384 × 384 (no centre crop, so margins stay in vie
 
 ### Suggested thresholds for `no_text`
 
-| Score | Decision | On the labelled hard pages below |
+| Score | Decision | On the 250 hand-labelled hard pages (see Evaluation) |
 |---|---|---|
 | > 0.8 | No text: skip, or flag any OCR output as invented | 67/67 blank, 83/90 show-through, 1/54 text |
-| 0.2 – 0.8 | Unsure: route or check | most faint, stamp and picture pages |
+| 0.2 – 0.8 | Unsure: route or check | a mix: 5 faint, 5 show-through, 4 text, 3 stamp, 1 picture |
 | < 0.2 | Text: OCR normally | 49/54 text |
 
-At a single cut of 0.5 on the BHL board pages: 99.9% of text pages kept, 99.4% of empty pages removed.
-
 ## Speed
 
-The model itself is cheap; decoding the page image usually costs more than running it.
+The model itself is fast. Decoding the page image usually takes longer than running it.
 
 | Setup | Pages per second, model only |
 |---|---|
@@ -88,11 +92,12 @@ End to end (encoded page in, scores out), the decoder sets the pace:
 | JPEG 2000, decoded on the GPU with nvImageCodec (nvJPEG2000) (L4 / RTX PRO 6000) | ~80 / ~120 |
 | JPEG 2000, decoded on the CPU (Pillow/OpenJPEG) | ~1.5–3 per CPU core |
 
-Notes from testing on Biodiversity Heritage Library pages:
+### Running at scale (notes from testing on BHL pages)
 
-- NVIDIA's GPU JPEG 2000 decoder (nvJPEG2000, via nvImageCodec or DALI) decoded all 60 BHL JP2 files we tested, from Kakadu, LuraWave, JasPer and OpenJPEG encoders. In a batch, one unreadable file can fail the whole batch: check files first (a complete codestream ends with the EOC marker `FF D9`) and decode any page the GPU returns empty on the CPU. For repeated runs over one collection, store small JPEG derivatives once: at 1,024 px (long side) DALI reaches ~2,000 pages per second on one A10G with the same decisions as full-size input; 512 px is about twice as fast but agrees slightly less.
-- ONNX export (fp32) gives identical scores. Static int8 quantization changed the scores a lot in our test; we do not recommend it.
-- GPU preprocessing agrees with the reference preprocessing: DALI with JPEG input gave the same decision on every test page (largest score difference 0.015); nvImageCodec with JPEG 2000 input and an area resize on 99.6% of pages.
+- **JPEG derivatives:** for repeated runs over one collection, store small JPEG derivatives once. At 1,024 px (long side) they give the same decisions as full-size input (see the A10G row above). 512 px is about twice as fast but agrees slightly less.
+- **GPU preprocessing** matches the reference: the same decision on every test page with DALI and JPEG input (largest score difference 0.015), and on 99.6% of pages with nvImageCodec, JPEG 2000 input and an area resize.
+- **JPEG 2000 on the GPU:** nvJPEG2000 (via nvImageCodec or DALI) decoded all 60 BHL files we tested. Before batching, check that each file is complete (a complete codestream ends with `FF D9`), and decode on the CPU any page the GPU returns empty.
+- **ONNX:** fp32 export gives identical scores. Static int8 quantisation changed the scores a lot in our test, so we do not recommend it.
 
 Batch inference with GPU decoding (JPEG bytes in):
 
@@ -125,9 +130,9 @@ def score(jpeg_bytes):                                  # a list of BATCH encode
 
 ## Evaluation
 
-All numbers are for the released checkpoint (seed 1). Mean ± SD over three training seeds where shown.
+Numbers are for the released checkpoint (seed 1), plus mean ± SD over three training seeds where shown.
 
-**BHL board** (2,165 Biodiversity Heritage Library pages with ground-truth transcriptions; text = at least 80 characters of ground truth):
+**Pages with ground-truth transcriptions:** 2,165 BHL pages. Text = at least 80 characters of ground truth.
 
 | | Result |
 |---|---|
@@ -136,12 +141,12 @@ All numbers are for the released checkpoint (seed 1). Mean ± SD over three trai
 | Empty pages removed at 0.5 | 0.994 |
 | Pages with 1–79 characters (mostly plates with captions) | kept as text (removed: 0.047) |
 
-**Human labels** (one labeller; 250 deliberately hard pages chosen where page judges disagreed or flagged something, plus random audits; not a random sample):
+**Hand-labelled hard pages:** 250 pages, one labeller. They were chosen where page models disagreed or flagged something, plus random audits, so they are not a random sample. Round 1 labels were also used to set the teacher's cut (see Training).
 
 | | AUROC, no text vs text |
 |---|---|
 | Round 1, 78 pages (BHL, Library of Congress and London Medical Officer of Health reports) | 0.970 (3 seeds: 0.976 ± 0.005) |
-| Round 2, 172 pages (BHL, MOH; held-out books) | 0.935 (3 seeds: 0.923 ± 0.015) |
+| Round 2, 172 pages (BHL and Medical Officer of Health reports; books not used in training) | 0.935 (3 seeds: 0.923 ± 0.015) |
 
 Confusion on these 250 pages (label × `no_text` score):
 
@@ -154,7 +159,7 @@ Confusion on these 250 pages (label × `no_text` score):
 | blank | 0 | 0 | 67 |
 | show-through only | 2 | 5 | 83 |
 
-**LightOnOCR-3 failures** on 2,086 pages from 133 held-out books (the real LightOnOCR-3-0.8B outputs on these pages are the reference):
+**LightOnOCR-3 failures:** 2,086 pages from 133 books not used in training. The reference is the real LightOnOCR-3-0.8B output on each page.
 
 | Output | Positives | AUROC | Precision @0.5 | Recall @0.5 | AUROC, 3 seeds |
 |---|---|---|---|---|---|
@@ -166,7 +171,7 @@ The loop reference is a simple rule (tail compression, repeated characters). It
 
 ## Training
 
-- **Pages:** 17,653 training and 1,124 validation pages from about 3,000 public-domain Biodiversity Heritage Library books, stratified by BHL page-type metadata (random, Blank, plate versos, plates). Books that hold any human-labelled page are held out. Medical Officer of Health pages (CC-BY-NC images) are not used for training.
+- **Pages:** 17,653 training and 1,124 validation pages from about 3,000 public-domain BHL books, sampled using BHL's page-type metadata (random pages, plus pages marked Blank, plate versos and plates). Books that hold any human-labelled page are held out. Medical Officer of Health pages (CC-BY-NC images) are not used for training.
 - **Targets** (soft, no human labels in training):
   - `no_text`: [openjev](https://huggingface.co/AlexWortega/openjev) 4B (MIT) with a page-question LoRA adapter, asked "The page has foreground text worth transcribing." at 896 × 1280, softened around a cut chosen on round-1 human labels.
   - `lo_*`: the actual outputs of LightOnOCR-3-0.8B (Apache-2.0) on every training page (truncation = `finish_reason == "length"`; loop = compression rule; blank-write = no-text page with ≥ 20 characters of output).
@@ -180,5 +185,9 @@ The loop reference is a simple rule (tail compression, repeated characters). It
 - **Labels:** one labeller, one pass. The boundary between blank and show-through, and between faint text and blank, is fuzzy.
 - **Conventions:** a page with only a library stamp, or an inserted "BLANK PAGE" card, counts as no text. Short captions and page numbers count as text: use [text-page-filter](https://huggingface.co/davanstrien/text-page-filter) to remove those.
 - **`faint` is not a faint-text detector.** It copies the teacher's "faded" answer, which fires on faded paper and show-through. On human faint-text labels it ranks backwards. Do not use it.
-- **LightOn heads are engine-specific.** They predict LightOnOCR-3-0.8B (vLLM, temperature 0.2, 4,096 tokens) failures. Other VLMs fail differently: in tests, GLM-OCR and PaddleOCR-VL wrote text on far more no-text pages.
+- **The LightOn outputs only apply to one OCR model.** They predict LightOnOCR-3-0.8B (vLLM, temperature 0.2, 4,096 tokens) failures. Other OCR models fail in other ways, so do not use these outputs for them.
 - **Teacher provenance:** the openjev page-question adapter was trained on DocLayNet, BHL page types and RVL-CDIP pages. RVL-CDIP has unclear licence terms. This model was trained on BHL images and the teacher's answers only.
+
+## Other kinds of material
+
+This model was built for printed books. If a more general version (manuscripts, newspapers, modern documents) would be useful to you, feel free to open a discussion in the Community tab. Or, even better, train one yourself! The recipe under Training is the starting point. For a different OCR model, use its outputs on your training pages as the `lo_*` targets.