ocr-projects-archive / ocr-tech /docs /LIMBU_PAGE_OCR_HARNESS.md
prosperious's picture
Add 1000 files (OCR_INDRA/.gitignore ...)
f9bc0c1 verified
|
Raw History Blame Contribute Delete
68.2 kB

Limbu Page OCR Harness

Updated: 2026-06-23

This document tracks the page-level Limbu OCR harness. Do not confuse the recognizer result with full page OCR. The current measured recognizer champion is useful on line crops, but raw phone photos require additional stages: page rectification, text/non-text segmentation, line grouping, reading order, and end-to-end page scoring.

Current State

The June 23 phone-photo smoke run used the two latest PXL_* images from ~/Downloads:

  • /Users/cdjk/Downloads/PXL_20260623_132115733.jpg
  • /Users/cdjk/Downloads/PXL_20260623_132120681.jpg

For reproducible local runs, those Downloads originals are staged under:

  • incoming/limbu-phone-pxl-raw-2026-06-23/PXL_20260623_132115733.jpg
  • incoming/limbu-phone-pxl-raw-2026-06-23/PXL_20260623_132120681.jpg

The staged hashes match the Downloads originals and the raw page manifest:

  • PXL_20260623_132115733.jpg: 4167ec57b84731bce3e275224d15cf6589dafdadac110c6c9b51ae9ff36b570d
  • PXL_20260623_132120681.jpg: 8df48d79a22521b0be536d3f4d87d6a1678107e164013a082b22f4cd8a99a7ca

Plain UTF-8 OCR extracts from the current best filtered phone run are saved at examples/limbu-phone-photo/ as ocr1.txt and ocr2.txt. They come from ocr-line-filter-sirijonga-conf050-margin-markers/*/document.json, not from the older naive full-frame run. Current counts are 20 lines / 743 characters for ocr1.txt and 58 lines / 2334 characters for ocr2.txt.

The first naive full-frame run is:

outputs/limbu-phone-photo-ocr-test-2026-06-23/

It completed, but it OCR'd background and non-text because the prepared images still contained keyboard/table/illustration regions. Summary:

  • pages: 2
  • detected lines: 497
  • review rows: 388
  • operator attention pages: 2
  • suspicious layout gaps: 10

The manually cropped baseline run is:

outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/

It uses the current recognizer package:

runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json

The cropped run is better, but still fails the page-OCR production gate:

  • pages: 2
  • detected lines: 280
  • review rows: 191
  • operator attention pages: 2
  • suspicious layout gaps: 2
  • phone-pxl-ocr-1: 82 lines, 54 review rows
  • phone-pxl-ocr-2: 198 lines, 183 review rows

The current best automatic text dumps are written to:

  • examples/limbu-phone-photo/ocr1.txt
  • examples/limbu-phone-photo/ocr2.txt

These are Unicode, but they are not clean OCR output. Treat them as a failure sample for the page harness, not as digitized text.

New Harness Gate

tools/audit_limbu_phone_page_ocr_run.py audits page OCR runs for production readiness. It checks:

  • too many detected lines per page
  • high review-row ratio
  • high other-script ratio
  • suspicious layout gaps
  • operator-attention flags
  • detector fragmentation from image-line-crops/manifest.jsonl
  • materializer-wrapper runs from tools/run_limbu_phone_page_ocr.py
  • optional raw-vs-baseline line-inventory deltas against a manually prepared materializer run

Current audit command:

uv run python tools/audit_limbu_phone_page_ocr_run.py \
  outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/limbu-book-ocr-run.json \
  --out outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/phone-page-ocr-audit \
  --exit-zero

Current result:

outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/phone-page-ocr-audit/phone-page-ocr-audit.md

It fails with twelve blockers. The important new signal is geometry, not script: the recognizer keeps many narrow fragments as confident Sirijonga. The audit now reads each page crop manifest and blocks the run when kept crops look like word/glyph fragments instead of full text lines.

Current detector-fragmentation numbers:

  • phone-pxl-ocr-1: 82 kept crops, median kept width ratio 0.060, narrow kept crop ratio 0.500, max same-row fragments 9
  • phone-pxl-ocr-2: 198 kept crops, median kept width ratio 0.045, narrow kept crop ratio 0.712, max same-row fragments 15

The production gate defaults are:

  • median kept crop width ratio must be at least 0.080
  • no more than 0.350 of kept crops may be narrower than 0.060 of page width
  • no estimated row may contain more than 4 kept detector fragments

These thresholds are not a recognition fix. They are a stop sign: the current morphology detector is splitting page text into fragments, and the recognizer is then hallucinating valid Limbu text on those fragments.

This is consistent with the existing CDC PDF-text detector sweep:

outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-detector-sweep-v1/limbu-pdf-text-detector-sweep.json

The best morphology sweep covers 76 of 81 source PDF-text lines, but emits 335 detected boxes. That is useful as a recall baseline, but it is not a production page detector because the candidate count is far above the source line count.

Detector Training Dataset Export

tools/export_limbu_line_detector_dataset.py now exports supervised text-line detector labels from the page image plus PDF text-layer geometry artifacts. It writes:

  • coco-text-line-detection.json
  • YOLO labels under labels/train/ and labels/val/
  • copied/augmented page images under images/train/ and images/val/
  • yolo-data.yaml
  • line-detector-labels.jsonl with per-line provenance
  • line-detector-dataset.json with hashes and source-boundary metadata

The exporter supports repeated --page-manifest and repeated --pdf-text-context-jsonl arguments, so multiple rendered books can be combined without hand-concatenating files. It also has an explicit --line-script-filter for limbu, devanagari, limbu-or-devanagari, or any; this matters because the page OCR goal covers both Sirijanga/Namdhinggo Limbu and Devanagari-written Limbu.

The text boundary is strict: PDF text-layer Unicode is provenance/review context only. The exported labels are geometry supervision for a text-line detector, not recognizer gold text.

The first 5-page CDC smoke export is:

uv run python tools/export_limbu_line_detector_dataset.py \
  --page-manifest outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/raw-page-manifest/book-pages.jsonl \
  --pdf-text-context-jsonl outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/pdf-text-context/limbu-pdf-text-context-lines.jsonl \
  --out outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23 \
  --include-phone-augmentations

Current output:

outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/line-detector-dataset.json

Summary:

  • pages: 5
  • source context rows: 82
  • base line boxes: 81
  • exported images: 20
  • exported annotations: 324
  • train images: 16
  • val images: 4
  • train pages: cdc-maths-p0011 through cdc-maths-p0014
  • val page: cdc-maths-p0015
  • variants: original, phone-blur-lowlight, phone-rotate-left, phone-rotate-right

This is the first actual training handoff for replacing the morphology detector. It proved the training path, but it is too small for a real detector.

The current larger CDC Grade 1 source pack is:

outputs/limbu-cdc-grade1-detector-sources-2026-06-23/

It renders the three mirrored CDC curriculum PDFs at 200 DPI and maps their PDF text-layer boxes onto the rendered page images:

  • Maths PDF: 111 rendered pages, 4352 text-layer lines, 3915 lines with Limbu codepoints after legacy conversion
  • Science PDF: 89 rendered pages, 1386 text-layer lines, 1316 lines with Limbu codepoints after legacy conversion
  • Social PDF: 87 rendered pages, 1316 text-layer lines, 1310 lines with Limbu codepoints after legacy conversion

The combined detector export command is:

uv run python tools/export_limbu_line_detector_dataset.py \
  --page-manifest outputs/limbu-cdc-grade1-detector-sources-2026-06-23/maths/book-pages.jsonl \
  --page-manifest outputs/limbu-cdc-grade1-detector-sources-2026-06-23/science/book-pages.jsonl \
  --page-manifest outputs/limbu-cdc-grade1-detector-sources-2026-06-23/social/book-pages.jsonl \
  --pdf-text-context-jsonl outputs/limbu-cdc-grade1-detector-sources-2026-06-23/maths/pdf-text-context/limbu-pdf-text-context-lines.jsonl \
  --pdf-text-context-jsonl outputs/limbu-cdc-grade1-detector-sources-2026-06-23/science/pdf-text-context/limbu-pdf-text-context-lines.jsonl \
  --pdf-text-context-jsonl outputs/limbu-cdc-grade1-detector-sources-2026-06-23/social/pdf-text-context/limbu-pdf-text-context-lines.jsonl \
  --out outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23 \
  --include-phone-augmentations \
  --split-strategy hash \
  --val-page-ratio 0.15

Current combined dataset:

outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23/line-detector-dataset.json

Summary:

  • source page manifests: 3
  • source context files: 3
  • rendered pages: 287
  • source context rows: 7054
  • base line boxes: 6540
  • skipped non-Limbu/script-filter rows: 513
  • skipped invalid bboxes: 1
  • exported images: 1148
  • exported annotations: 26160
  • train images: 976
  • val images: 172
  • split strategy: deterministic hash
  • variants: original, phone-blur-lowlight, phone-rotate-left, phone-rotate-right

The split remains page-level, not augmentation-level, so augmented copies of the same page do not leak across train and validation.

tools/preflight_limbu_line_detector_training.py validates the exported detector dataset and writes the exact training command. Current preflight:

uv run python tools/preflight_limbu_line_detector_training.py \
  outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/line-detector-dataset.json \
  --out outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/training-preflight \
  --model yolov8n.pt \
  --epochs 100 \
  --imgsz 1280 \
  --batch 4 \
  --project runs/limbu-line-detector \
  --name cdc-maths-p0011-p0015-v2

Current result:

outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/training-preflight/line-detector-training-preflight.json

The 5-page smoke dataset preflight passes with 0 issues. The larger all-books dataset preflight also passes with 0 issues:

outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23/training-preflight/line-detector-training-preflight.json

Local runtime still lacks torch/ultralytics, so detector training runs on the AWS G6 host.

AWS G6 Detector Smoke Training

The first supervised line-detector smoke run was executed on the existing AWS playaround G6 instance:

  • account: 798006441552
  • instance: i-0fa19dff7b233a353
  • name: ocrtech-g6-training-20260615-v4
  • type: g6.xlarge
  • runtime path: /opt/ocrtech
  • GPU: NVIDIA L4

Remote preflight passed with runtime checks enabled:

cd /opt/ocrtech
.venv/bin/python tools/preflight_limbu_line_detector_training.py \
  outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/line-detector-dataset.json \
  --out outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/aws-training-preflight-v2 \
  --require-runtime \
  --model yolov8n.pt \
  --epochs 100 \
  --imgsz 1280 \
  --batch 4 \
  --project /opt/ocrtech/runs/limbu-line-detector \
  --name cdc-maths-p0011-p0015-v2

Local copy of the AWS preflight:

outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/aws-training-preflight-v2/line-detector-training-preflight.json

The training run completed:

cd /opt/ocrtech/outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23
/opt/ocrtech/.venv/bin/yolo detect train \
  data=yolo-data.yaml \
  model=/opt/ocrtech/yolov8n.pt \
  epochs=100 \
  imgsz=1280 \
  batch=4 \
  project=/opt/ocrtech/runs/limbu-line-detector \
  name=cdc-maths-p0011-p0015-v2 \
  exist_ok=True

Local copy of the detector run:

outputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/

Important artifacts:

  • outputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/weights/best.pt
  • outputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/weights/last.pt
  • outputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/results.csv
  • outputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/train.log

Best validation row by mAP50 was epoch 94:

  • precision: 0.39716
  • recall: 0.31944
  • mAP50: 0.22702
  • mAP50-95: 0.13800

The final validation of best.pt reported precision 0.399, recall 0.319, mAP50 0.228, and mAP50-95 0.136 over 4 validation images with 72 line instances. This checkpoint proves the supervised training path works, but it is not production-quality. The validation split is one CDC page plus augmentations, so the metric is a smoke signal only.

The checkpoint was also smoke-tested against the two real PXL prepared page images. Local prediction artifacts:

outputs/aws-g6-limbu-line-detector-phone-predict/

Detector counts:

  • cropped pages, conf=0.05: 125, 245
  • cropped pages, conf=0.25: 21, 48
  • cropped pages, conf=0.50: 0, 0
  • auto-detected pages, conf=0.05: 128, 228
  • auto-detected pages, conf=0.25: 25, 44
  • auto-detected pages, conf=0.50: 0, 1

At conf=0.25, the predicted boxes are sometimes line-sized, but they still fragment rows, miss lower text, and admit illustration/photo regions. This is not good enough to replace the current page detector. The right conclusion is: the training path is alive, but the detector needs substantially more page-level supervision and likely negative/non-text examples before recognizer integration.

AWS G6 Detector All-Books Training

The larger CDC Grade 1 all-books detector dataset was synced to the same AWS G6 host and passed remote runtime preflight:

outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23/aws-training-preflight/line-detector-training-preflight.json

Remote training command:

cd /opt/ocrtech/outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23
/opt/ocrtech/.venv/bin/yolo detect train \
  data=yolo-data.yaml \
  model=/opt/ocrtech/yolov8n.pt \
  epochs=80 \
  imgsz=1280 \
  batch=4 \
  project=/opt/ocrtech/runs/limbu-line-detector \
  name=cdc-grade1-allbooks-v1 \
  exist_ok=True

Launch metadata:

  • AWS profile: playaround
  • instance: i-0fa19dff7b233a353
  • public IP used for artifact pull and phone prediction: 3.219.242.144
  • remote run directory: /opt/ocrtech/runs/limbu-line-detector/cdc-grade1-allbooks-v1
  • remote launch log: /opt/ocrtech/runs/limbu-line-detector-launch/cdc-grade1-allbooks-v1/train.log
  • remote PID at launch: 1776

The run completed 80 epochs in 0.720 hours. YOLO scanned 976 train images and 172 validation images with 0 background, corrupt, or missing-label issues.

Local copy of the completed detector run:

outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/

Important artifacts:

  • outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/weights/best.pt
  • outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/weights/last.pt
  • outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/results.csv
  • outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/train.log

Best validation row by mAP50 was epoch 79:

  • precision: 0.93159
  • recall: 0.85800
  • mAP50: 0.92478
  • mAP50-95: 0.72025

The final epoch 80 row reported precision 0.92947, recall 0.85318, mAP50 0.91988, and mAP50-95 0.71604. The final validation of best.pt reported precision 0.937, recall 0.845, mAP50 0.921, and mAP50-95 0.724 over 172 validation images and 4148 text-line instances.

This checkpoint is no longer a toy detector on the CDC rendered/augmented validation split. It still does not prove page OCR quality, because the validation set is derived from PDF-rendered curriculum pages, not raw phone photos.

Operational note: on 2026-06-23, the local AWS playaround profile had an expired SSO session. SSH access to the known instance still worked with ~/.ssh/ocrtech-g6-training-20260615.pem, but EC2 API actions such as describe-instances and stop-instances could not be run until SSO was refreshed. The remote host also had no AWS credentials or instance role available to aws sts get-caller-identity, so it could not stop itself through the AWS CLI either.

All-Books Checkpoint Phone-Photo Smoke

The completed all-books checkpoint was tested on the same two prepared PXL phone page images used by the failed morphology-detector run:

  • cropped prepared pages: outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/capture-prep/
  • auto-detected prepared pages: outputs/limbu-phone-photo-auto-detect-capture-2026-06-23/

Remote prediction command pattern:

cd /opt/ocrtech
.venv/bin/yolo detect predict \
  model=/opt/ocrtech/runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt \
  source=/opt/ocrtech/tmp/limbu-phone-detector-input/<cropped-or-auto> \
  imgsz=1280 \
  conf=<threshold> \
  project=/opt/ocrtech/runs/limbu-line-detector-phone-predict-allbooks \
  name=<run-name> \
  save=True \
  save_txt=True \
  save_conf=True \
  exist_ok=True

Local prediction artifacts:

outputs/aws-g6-limbu-line-detector-phone-predict-allbooks/

Detector counts:

  • cropped pages, conf=0.25: 23, 81
  • cropped pages, conf=0.35: 23, 74
  • cropped pages, conf=0.50: 23, 67
  • auto-detected pages, conf=0.25: 24, 78
  • auto-detected pages, conf=0.35: 22, 72
  • auto-detected pages, conf=0.50: 22, 64

Geometry audit results:

  • cropped-conf025: failed, 104 predictions, blocker same_row_fragmentation_too_high:6>4
  • cropped-conf035: passed, 97 predictions
  • cropped-conf050: passed, 90 predictions
  • auto-conf025: failed, 102 predictions, blocker same_row_fragmentation_too_high:5>4
  • auto-conf035: failed, 94 predictions, blocker same_row_fragmentation_too_high:5>4
  • auto-conf050: passed, 86 predictions

The most useful smoke candidate is cropped-conf050: it passes the geometry gate and visually produces real line-sized boxes instead of the earlier word/glyph fragments. It still has false positives on non-page/background text and small decorative fragments. The detector materializer below now handles direct checkpoint inference, saved-label ingestion, basic filtering, duplicate suppression, and explicit two-column reading order. This is still not a finished end-to-end page OCR benchmark because there is no gold page text/CER for these phone photos and the raw-photo wrapper still has to combine rectification, detector inference, recognition, scoring, and audit gates in one reproducible command.

YOLO Detector Page OCR Materialization

tools/materialize_page_image_line_boxes.py now supports using the trained detector instead of the old morphology detector:

  • --line-detector yolo-labels
  • --yolo-labels-dir <dir>
  • --line-detector yolo-model
  • --yolo-model-path <best.pt>
  • confidence, size, area, and page-region filters
  • same-row overlap duplicate suppression for YOLO detections
  • explicit wide-column reading orders: wide_column_left_to_right, wide_column_right_to_left, and split-tall variants
  • header-preserving wide-column ordering, so page titles above body columns stay before the column body text
  • per-page document.json, document.md, and document.body.md

The duplicate suppression is deliberately recorded in the run metadata. It drops a nested same-row fragment when the fragment substantially overlaps a kept row. A much wider row wins over a contained fragment even if the fragment has higher detector confidence; if widths are similar, higher confidence wins. The current defaults are:

  • enabled: true
  • overlap fraction of smaller box: 0.75
  • wider-box preference ratio: 1.12
  • y-tolerance: automatic when 0.0

Current PXL eval manifest:

outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-2026-06-23/eval-manifest.jsonl

Saved-label materialization command:

PYTHONPATH=src /tmp/ocrtech-paddle311/bin/python tools/materialize_page_image_line_boxes.py \
  --eval-manifest outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-2026-06-23/eval-manifest.jsonl \
  --model-config runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json \
  --out outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents \
  --engine candidate \
  --recognition-mode line \
  --line-detector yolo-labels \
  --yolo-labels-dir outputs/aws-g6-limbu-line-detector-phone-predict-allbooks/cropped-conf050/labels \
  --yolo-min-confidence 0.50 \
  --yolo-min-width-ratio 0.04 \
  --yolo-region-min-y-ratio 0.05 \
  --reading-order wide_column_left_to_right \
  --crop-padding 8 \
  --allow-empty-lines

Saved-label output:

outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/

Run summary after same-row overlap suppression:

  • samples: 2
  • total materialized OCR lines: 78
  • empty recognizer outputs: 0
  • detector source: yolo_line_detector_labels_v1
  • reading order: wide_column_left_to_right on both pages
  • crop manifest rows: 78
  • phone-pxl-ocr-1: 21 lines, 0 suppressed duplicate detector rows
  • phone-pxl-ocr-2: 57 lines, 7 suppressed duplicate detector rows
  • high-overlap same-row duplicate pairs after suppression: 0 on both pages

Key output files:

  • outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-run.json
  • outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-crops.jsonl
  • outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/phone-pxl-ocr-1/document.md
  • outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/phone-pxl-ocr-2/document.md

Direct checkpoint materialization was run on the AWS G6 box because the local machine currently has no combined PaddleOCR plus Ultralytics/Torch runtime and only about 9.6GiB free disk. The G6 runtime has torch 2.12.1+cu130, ultralytics 8.4.75, paddle 3.3.1, and paddleocr 3.7.0.

Remote manifest:

/opt/ocrtech/outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/eval-manifest.jsonl

Direct checkpoint materialization command:

cd /opt/ocrtech
PYTHONPATH=src .venv/bin/python tools/materialize_page_image_line_boxes.py \
  --eval-manifest outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/eval-manifest.jsonl \
  --model-config runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json \
  --out outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents \
  --engine candidate \
  --recognition-mode line \
  --line-detector yolo-model \
  --yolo-model-path runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt \
  --yolo-imgsz 1280 \
  --yolo-device 0 \
  --yolo-min-confidence 0.50 \
  --yolo-min-width-ratio 0.04 \
  --yolo-region-min-y-ratio 0.05 \
  --reading-order wide_column_left_to_right \
  --crop-padding 8 \
  --allow-empty-lines

Local copy of direct-checkpoint output:

outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/

Direct-checkpoint run summary:

  • samples: 2
  • total materialized OCR lines: 78
  • empty recognizer outputs: 0
  • detector source: yolo_line_detector_model_v1
  • detector source pass count: yolo-model: 78
  • reading order: wide_column_left_to_right on both pages
  • crop manifest rows: 78
  • phone-pxl-ocr-1: 21 lines, 0 suppressed duplicate detector rows
  • phone-pxl-ocr-2: 57 lines, 7 suppressed duplicate detector rows
  • high-overlap same-row duplicate pairs after suppression: 0 on both pages

Key direct-checkpoint output files:

  • outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-run.json
  • outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-crops.jsonl
  • outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/phone-pxl-ocr-1/document.md
  • outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/phone-pxl-ocr-2/document.md

The direct checkpoint and saved-label paths produce the same line counts and the same page-1 body Markdown. Page 2 is not byte-identical because the saved labels store rounded normalized box coordinates while direct model inference returns full-precision boxes; small crop-boundary differences change some recognizer outputs. The direct-checkpoint output should be treated as the authoritative current page OCR artifact.

This is the first page-level path that is meaningfully better than the toy morphology OCR: the trained detector checkpoint feeds line crops directly, the recognizer returns Unicode for every kept crop, document JSON/Markdown are emitted, title lines stay above wide-column body text, and duplicated detector fragments are audited and suppressed. It is still not enough to claim "feed in Limbu books and get OCR" because the current entrypoint starts from prepared page images rather than raw captures, has only geometry-based text/non-text filtering, and lacks a gold page transcript/CER benchmark for these real phone photos.

YOLO Prediction Geometry Audit

tools/audit_yolo_line_detector_predictions.py audits saved YOLO prediction labels before the detector is allowed into phone-page OCR. It does not need text references. It checks:

  • total prediction count per page
  • median predicted box width ratio
  • narrow predicted box ratio
  • same-row fragmentation from normalized y-centers

This targets the real phone-photo failure mode: many confident narrow boxes that look like words/glyph fragments, not text lines. The default blockers are:

  • more than 120 predictions on a page
  • median box width below 0.080 of page width
  • more than 0.350 of boxes narrower than 0.060 of page width
  • more than 4 boxes in the same estimated row

Current focused test command:

uv run pytest \
  tests/test_audit_yolo_line_detector_predictions.py \
  tests/test_export_limbu_line_detector_dataset.py \
  tests/test_preflight_limbu_line_detector_training.py \
  tests/test_build_limbu_book_page_manifest.py \
  tests/test_build_limbu_pdf_text_context.py \
  -q

Current result: 24 passed.

Capture Fix Added

src/ocrtech/capture.py now has an automatic page-detection fallback. The previous implementation only accepted a closed four-corner Canny contour. The fallback detects the dominant bright, low-saturation page-like region and uses a minimum-area rectangle when the contour path fails.

On the two real PXL images, auto-detection now completes instead of crashing:

/tmp/ocrtech-paddle311/bin/python tools/prepare_limbu_book_captures.py \
  --page-manifest outputs/limbu-phone-photo-ocr-test-2026-06-23/raw-page-manifest/book-pages.jsonl \
  --out outputs/limbu-phone-photo-auto-detect-capture-2026-06-23 \
  --auto-detect-page \
  --auto-deskew \
  --allow-failed-capture-audit

The run writes:

outputs/limbu-phone-photo-auto-detect-capture-2026-06-23/

Both pages use method=light_region_min_area_rect and record touches_image_border=true. The capture audit passes but warns that the source may be clipped or include background. This is deliberate: the harness should surface bad capture geometry instead of silently producing confident OCR.

Capture Quality Gates

src/ocrtech/capture.py now records laplacian_variance in image_quality.source and image_quality.prepared. This is the current sharpness/blur signal. The existing entropy, luminance_stddev, and edge_stddev metrics remain in place, and the audit can now enforce all of these as prepared-image thresholds.

audit_limbu_capture() and tools/prepare_limbu_book_captures.py now expose phone-capture policy gates for:

  • minimum prepared width and height;
  • minimum prepared entropy;
  • minimum prepared luminance standard deviation;
  • minimum prepared edge standard deviation;
  • minimum prepared Laplacian variance;
  • requiring automatic page detection;
  • minimum detected page area ratio;
  • optionally failing when auto-detected page corners touch the source image border;
  • requiring applied auto-deskew;
  • maximum absolute applied deskew angle.

For the current PXL smoke pages, border-touch remains a warning rather than a failure. Both pages touch the source image border, which means the page may be clipped or include background, but rejecting them would reject the actual hard phone photos we need to handle. Use --fail-on-auto-detect-page-border-touch only for a stricter capture-quality batch.

Local quality-gate replay manifest:

outputs/limbu-phone-capture-quality-gates-2026-06-23/raw-page-manifest/book-pages.jsonl

Replay command:

uv run python tools/prepare_limbu_book_captures.py \
  --page-manifest outputs/limbu-phone-capture-quality-gates-2026-06-23/raw-page-manifest/book-pages.jsonl \
  --out outputs/limbu-phone-capture-quality-gates-2026-06-23/capture-prep \
  --auto-detect-page \
  --auto-deskew \
  --min-prepared-width 2500 \
  --min-prepared-height 3500 \
  --min-prepared-entropy 5.0 \
  --min-prepared-luminance-stddev 20 \
  --min-prepared-edge-stddev 10 \
  --min-prepared-laplacian-variance 50 \
  --require-auto-detect-page \
  --min-auto-detect-page-area-ratio 0.50 \
  --require-auto-deskew-applied \
  --max-abs-auto-deskew-degrees 4.0

Replay artifact:

outputs/limbu-phone-capture-quality-gates-2026-06-23/capture-prep/

Result:

  • pages: 2
  • failed capture audits: 0
  • both pages warn: auto_detect_page touches image border

Page quality measurements:

  • phone-pxl-raw-1: prepared size 2945x4059, entropy 6.126979, luminance stddev 57.978733, edge stddev 26.103824, Laplacian variance 243.456146, page area ratio 0.6772529951108047, deskew 1.6 degrees.
  • phone-pxl-raw-2: prepared size 3013x4165, entropy 6.104708, luminance stddev 46.921525, edge stddev 16.186986, Laplacian variance 109.91304, page area ratio 0.7292922076056986, deskew 1.7 degrees.

The raw phone wrapper exposes the same policy switches, so a future end-to-end OCR run can fail before recognition when a photo is too small, too flat, too blurry, lacks page detection, lacks deskew evidence, or is more skewed than the allowed limit.

Compatibility note: capture metadata written before laplacian_variance existed does not fail replay only because that newer metric is absent. The audit emits a warning such as image_quality.prepared metrics missing newer replay fields: laplacian_variance. If a caller sets --min-prepared-laplacian-variance, then legacy metadata without that metric fails because the requested sharpness gate cannot be evaluated.

Raw Phone-Photo Wrapper Run

tools/run_limbu_phone_page_ocr.py is now the reproducible raw-photo entrypoint. It does three things in one command:

  1. Runs tools/prepare_limbu_book_captures.py from a raw page manifest.
  2. Converts the prepared-page manifest into a materializer eval manifest.
  3. Runs tools/materialize_page_image_line_boxes.py with the trained YOLO line detector and the Limbu recognizer package.

The wrapper preserves the handoff files required by other scripts:

  • source raw manifest: outputs/limbu-phone-page-ocr-raw-yolo-model-auto-2026-06-23/raw-page-manifest/book-pages.jsonl
  • capture prep run: outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/capture-prep/book-capture-prep-run.json
  • prepared page manifest: outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/capture-prep/prepared-page-manifest.jsonl
  • materializer eval manifest: outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/materializer-eval-manifest.jsonl
  • materializer run: outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/documents/image-line-box-run.json
  • final per-page documents: outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/documents/phone-pxl-raw-1/document.md and outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/documents/phone-pxl-raw-2/document.md
  • wrapper summary with hashes: outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/limbu-phone-page-ocr-run.json

Replication-critical provenance rule: every script boundary must preserve the source page fields, not just the prepared image path. The capture-prep script now copies source fields from the raw page manifest into capture-prep/prepared-page-manifest.jsonl, the raw phone wrapper copies them into materializer-eval-manifest.jsonl under metadata, and the materializer copies that eval-manifest metadata into each final document.json under metadata.source_manifest_metadata and pages[].metadata.source_manifest_metadata. The fields currently preserved are:

  • page_sequence_number
  • book_page_number
  • source_filename
  • source_document_path
  • source_document_sha256
  • source_document_page_number
  • source_document_page_index
  • source_kind
  • rendered_pdf_dpi
  • source_url
  • publication_date
  • edition
  • pdf_page_number
  • language
  • script
  • source_metadata

This is required for replication with other scripts: scoring, review packets, and future corpus exports must be able to trace an OCR line back to the exact source document/page without relying on filenames or memory. Do not strip these fields when creating derived manifests. The refreshed provenance run carries the available phone-capture metadata through all three handoff files. It does not yet carry CDC/PDF source-document fields because the raw PXL manifest itself does not identify the source PDF/book page.

Current provenance-handoff verification:

uv run pytest \
  tests/test_prepare_limbu_book_captures.py \
  tests/test_run_limbu_phone_page_ocr.py \
  tests/test_materialize_page_image_line_boxes.py \
  -q

Result: 40 passed. The broader focused page-OCR suite, including the central page scorer, page-output audit tests, and page eval-pack metadata tests, also passed: 139 passed, with only the existing SWIG deprecation warnings from PDF rendering imports.

The raw source manifest rows point at the two uploaded PXL JPEGs on the AWS G6 box:

  • incoming/limbu-phone-pxl-raw-2026-06-23/PXL_20260623_132115733.jpg
  • incoming/limbu-phone-pxl-raw-2026-06-23/PXL_20260623_132120681.jpg

AWS G6 command:

cd /opt/ocrtech
PYTHONPATH=src .venv/bin/python tools/run_limbu_phone_page_ocr.py \
  --page-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-auto-2026-06-23/raw-page-manifest/book-pages.jsonl \
  --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23 \
  --model-config runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json \
  --engine candidate \
  --recognition-mode line \
  --line-detector yolo-model \
  --yolo-model-path runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt \
  --yolo-imgsz 1280 \
  --yolo-device 0 \
  --yolo-min-confidence 0.50 \
  --yolo-min-width-ratio 0.04 \
  --yolo-header-min-width-ratio 0.015 \
  --yolo-header-max-y-ratio 0.30 \
  --yolo-header-min-confidence 0.75 \
  --yolo-region-min-y-ratio 0.05 \
  --reading-order auto_layout_left_to_right \
  --crop-padding 8 \
  --allow-empty-lines

Wrapper summary:

  • pages: 2
  • OCR lines: 79
  • empty OCR lines: 0
  • capture failed audit count: 0
  • materializer detector: yolo_line_detector_model_v1
  • detector source pass count: yolo-model: 79
  • requested reading order: auto_layout_left_to_right
  • reading-order selection counts: top_to_bottom: 1, wide_column_left_to_right: 1
  • detector checkpoint: runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt

Capture operations are recorded per page in capture-prep/book-capture-prep-run.json under pages[].capture_operations. Both pages used the bright-region fallback, then perspective rectification, deskew, grayscale, and autocontrast:

  • phone-pxl-raw-1: light_region_min_area_rect, area ratio 0.6772529951108047, touches_image_border=true, rectified size 2833x3981, deskew angle 1.6, prepared size 2945x4059
  • phone-pxl-raw-2: light_region_min_area_rect, area ratio 0.7292922076056986, touches_image_border=true, rectified size 2893x4079, deskew angle 1.7, prepared size 3013x4165

Materializer page summary:

  • phone-pxl-raw-1: 21 OCR lines, 0 empty lines, 0 same-row duplicate suppressions, reading order top_to_bottom
  • phone-pxl-raw-2: 58 OCR lines, 0 empty lines, 4 same-row duplicate suppressions, reading order wide_column_left_to_right

This raw-wrapper run is useful but not yet good enough to call done. The manually prepared direct-checkpoint run materialized 21 + 57 lines. The first raw auto-wrapper run materialized 20 + 58 lines because page rectification and region filtering altered the detector input. The current header-rescue run materializes 21 + 58: it fixes the missing top marker on page 1, but page 2 is still one line over the manually prepared baseline.

A previous run forced wide_column_left_to_right, which was wrong for the first page and placed the title around line 14. The corrected auto_layout_left_to_right logic now selects top_to_bottom for phone-pxl-raw-1 and wide_column_left_to_right for phone-pxl-raw-2. The first raw page starts with the rescued narrow top marker:

ᤍᤁ

ᤈᤠ᤺ᤵᤐᤠ ᤛᤒᤠ

The rescued marker crop is very small and the recognized Unicode text is poor. That is expected for now. The important change is that the detector inventory is no longer dropping the page-1 header/page-marker line. The next real page-understanding bug is page-2 line inventory, not title order.

The current YOLO filter policy has a narrow-header rescue:

  • normal minimum detector width is still controlled by --yolo-min-width-ratio 0.04
  • a narrow detection can pass only near the top of the image when all three header flags allow it: --yolo-header-min-width-ratio 0.015, --yolo-header-max-y-ratio 0.30, and --yolo-header-min-confidence 0.75
  • the rescue still obeys the normal confidence, region, height, and border filters, so bottom fragments do not reopen

OCR Line Filter / Text-Nontext Gate

tools/filter_page_ocr_documents.py is the current non-oracle post-materializer line filter. It reads existing page-eval-compatible document.json files, removes lines/blocks using only live OCR attributes, writes filtered document.json files, records one JSONL decision row per source line, and can be replay-audited by tools/audit_page_ocr_line_filter.py.

This is not a learned text/non-text classifier yet. It is a reproducible gate for script, confidence, geometry, and structural-role decisions. The filter now supports:

  • --drop-empty
  • --require-script sirijonga|devanagari|limbu_or_devanagari
  • confidence thresholds with --min-confidence
  • absolute and page-relative width/height thresholds
  • --drop-structural-role and --keep-structural-role
  • --infer-margin-page-markers, which derives page_marker for tiny centered short boxes near the top or bottom page margin when older documents only recorded structural_role: text

The materializer also marks the same small centered margin marker as structural_role=page_marker on new runs. The filter-side inference exists so older artifacts can be filtered without rerunning YOLO and OCR. This matters on the local machine because the recognizer model card exists, but the current YOLO detector weights are not present at runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt; the recorded full wrapper command is an AWS G6 command.

Current replay command on the two-page PXL artifact:

uv run python tools/filter_page_ocr_documents.py \
  --eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/materializer-eval-manifest.jsonl \
  --documents-dir outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/documents \
  --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers \
  --drop-empty \
  --require-script sirijonga \
  --min-confidence 0.50 \
  --infer-margin-page-markers \
  --drop-structural-role page_marker

Audit command:

uv run python tools/audit_page_ocr_line_filter.py \
  outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers

Filter artifact:

outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers/

Audit result: passed with issues=0.

Current result:

  • source lines: 79
  • kept lines: 78
  • removed lines: 1
  • removal counts: structural_role_dropped: 1, low_confidence: 1

The removed line is phone-pxl-raw-1/img-line-0001. The crop is visually the page marker 52, but the recognizer read it as ᤍᤁ with confidence 0.36888888478279114. Its detected box is [1345.6043701171875, 954.5286254882812, 54.516357421875, 66.0487060546875] on a 2945x4059 prepared page, so the filter marks it structural_role=page_marker, structural_role_reason=centered_margin_short_small_box, and structural_role_source=filter_inference.

Replication notes for downstream scripts:

  • Use the filtered directory as --documents-dir only when you explicitly want non-text/page-marker suppression.
  • Use the unfiltered documents/ directory for detector recall, missed-line, and reference-review work, otherwise the review packet can hide detector output that still needs adjudication.
  • ocr-line-filter-run.json records the exact source manifest, source documents directory, filter parameters, counts, and output hashes.
  • ocr-line-filter-lines.jsonl is the per-line decision table; it is the file to inspect when a later script disagrees with the filtered line count.
  • ocr-line-filter-audit.json and .md are the replay proof. A filtered directory without a passing audit is not claim-ready.

Page Output Audit For Scoring

tools/audit_page_ocr_documents.py audits page-eval-compatible document.json output directories and writes the per-sample limbu-output-audit.json files required by tools/run_page_ocr_eval.py --require-output-audit. This is separate from ocrtech audit-limbu-output, which expects the older limbu-output-manifest.json pipeline-output shape and does not directly fit the page materializer/filter directories.

The page document audit checks:

  • document.json parses as the shared Document schema;
  • document.source_path is readable;
  • document.source_path matches the eval-manifest input by resolved path or by SHA-256, so copied prepared images are allowed only when byte-identical;
  • page width/height and line bounding boxes are positive and inside the page;
  • line IDs are present and unique per page;
  • block line_ids point at real lines;
  • expected ocr_output_kind and detector source pass match policy;
  • line confidence is present when required;
  • line crop paths exist and crop_sha256 matches when crop checking is enabled.

Current command on the active filtered phone documents:

uv run python tools/audit_page_ocr_documents.py \
  --eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \
  --documents-dir outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers \
  --expected-ocr-output-kind live_detector \
  --expected-detector-source-pass yolo-model \
  --require-line-crops \
  --require-line-confidence \
  --min-line-count 1

Audit artifacts:

  • aggregate: outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers/page-ocr-documents-audit.json
  • per page: outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers/phone-pxl-raw-1/limbu-output-audit.json
  • per page: outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers/phone-pxl-raw-2/limbu-output-audit.json

Current result:

  • samples: 2
  • passed samples: 2
  • failed samples: 0
  • audited lines: 78
  • empty lines: 0
  • source binding for both pages: same_sha256
  • crop hashes checked: 20 on phone-pxl-raw-1, 58 on phone-pxl-raw-2
  • detector source pass counts: yolo-model: 78

After this audit and the scorer SHA-256 source-binding fix, the current claim-safe scorer dry run against reference-draft/eval-manifest.jsonl still fails, as it should, but it no longer reports either:

  • page output audits are not all passing
  • document source_path mismatches detected

Against the original reference-draft/eval-manifest.jsonl, the remaining scorer blockers are now the real benchmark blockers plus missing scorer sidecars: draft references, non-frozen pack metadata, missing eval-pack datasheet/script inventory/dictionary/leakage evidence, only 2 scoring pages, and no layout-detection reference coverage. For a reproducible scorer dry run, first build the scorer sidecars described in the next section and use that rewritten manifest.

Page Eval Pack Metadata Sidecars

tools/prepare_page_ocr_eval_pack_metadata.py prepares the central scorer sidecars for an existing page-level OCR eval manifest. It writes a rewritten manifest with scorer-visible source provenance, a summary.json with the exact page-pack pack_content_sha256, and a script-inventory.json derived from the loaded reference text. It does not certify labels: draft references remain draft, claim_evidence_eligible=false remains false, and --require-claim-safe still fails until the references are human verified.

This tool exists because the first phone reference draft had good reference files but did not expose whole-page bbox and source_image fields in the manifest keys that tools/run_page_ocr_eval.py audits. The phone draft writer now emits those fields for future runs, but this sidecar tool is the replication path for existing drafts and other page-manifest builders.

Current sidecar-generation command:

uv run python tools/prepare_page_ocr_eval_pack_metadata.py \
  --eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \
  --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack \
  --eval-pack-version limbu-phone-page-reference-draft-v1 \
  --force

Sidecar output:

outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack/

Key files:

  • eval-manifest.jsonl
  • summary.json
  • script-inventory.json
  • page-ocr-eval-pack-metadata.json
  • README.md

Current sidecar result:

  • samples: 2
  • reference lines: 79
  • claim-ready references: 0/2
  • pack_content_sha256: b20abcbbe4c6f38e39be055d9d618d29d0ecdc3751a5eb38d50384d89fabed8e
  • metadata coverage under the scorer: missing_source_document=0, missing_page=0, missing_bbox=0, missing_claim_eligibility=0, missing_language=0, missing_script=0, missing_domain=0, missing_gt_status=0
  • script inventory coverage under the scorer: missing_required_codepoints=[], missing_required_graphemes=[]

Scorer dry-run command using the sidecars and the filtered phone OCR documents:

uv run python tools/run_page_ocr_eval.py \
  --eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack/eval-manifest.jsonl \
  --documents-dir outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers \
  --ocr-output-kind live_detector \
  --require-output-audit \
  --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/page-eval-scorer-pack-dry-run \
  --summary-file outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack/summary.json \
  --script-inventory-file outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack/script-inventory.json \
  --system-name limbu-phone-page-ocr \
  --model-name phone-page-yolo-recognizer-filtered-draft \
  --language limbu \
  --script Limbu \
  --domain real_print \
  --gt-status draft_machine_ocr_needs_human_review \
  --eval-pack-version limbu-phone-page-reference-draft-v1 \
  --selected-checkpoint-on-test \
  --force

Dry-run scorer output:

outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/page-eval-scorer-pack-dry-run/

Current dry-run metrics:

  • scored pages: 2/2
  • page text codepoint CER: 0.000975
  • page text grapheme CER: 0.001976
  • text exact pages: 1/2
  • line reference/prediction/match counts: 79/78/78
  • line detection precision/recall/F1 micro: 1.000000/0.987342/0.993631
  • matched-line text CER: 0
  • matched-line reading-order pair accuracy: 1.000000
  • full reading-order pair accuracy: 0.952381

The sidecar dry run removes the avoidable scorer warnings:

  • no eval pack summary/datasheet missing
  • no eval pack summary missing pack_content_sha256
  • no pack content digest mismatch
  • no script inventory file missing
  • no missing_source_document
  • no missing_bbox
  • no page output audits are not all passing
  • no document source_path mismatches detected

The remaining warnings are honest blockers, not metadata bugs:

  • checkpoint/config/dictionary artifacts were not supplied as readable local files;
  • eval pack is not frozen and not unseen for selection;
  • selected checkpoint on this draft pack is declared;
  • eval pack and rows are not claim eligible;
  • references are not claim-ready: 0/2;
  • sample size is only 2, below the 30 page claim threshold;
  • GT status is draft_machine_ocr_needs_human_review;
  • no layout detection references are present;
  • no train/eval leakage manifest was supplied.

One metric caveat matters: the filtered OCR output has 78 lines, while the draft references still have 79 lines. The unmatched reference line is the page marker removed by the text/non-text filter. That is useful signal, not a failure to hide; review/promotion must either mark that reference line as structural/non-text or remove it from the claim-facing reference.

The raw-wrapper run now has an explicit audit gate comparing it against the manually prepared direct-checkpoint baseline. The audit reads the wrapper summary, resolves materializer_run, reads each document.json using the real pages[].text_lines[] structure, checks detector and reading-order invariants, and blocks when the per-page line inventory differs from baseline.

Raw-vs-baseline inventory audit command:

uv run python tools/audit_limbu_phone_page_ocr_run.py \
  outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/limbu-phone-page-ocr-run.json \
  --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/phone-page-ocr-audit \
  --baseline-materializer-run outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-run.json \
  --baseline-page-map phone-pxl-raw-1=phone-pxl-ocr-1 \
  --baseline-page-map phone-pxl-raw-2=phone-pxl-ocr-2 \
  --required-detector yolo_line_detector_model_v1 \
  --expected-reading-order phone-pxl-raw-1=top_to_bottom \
  --expected-reading-order phone-pxl-raw-2=wide_column_left_to_right \
  --max-baseline-line-count-delta 0 \
  --exit-zero

Audit artifact:

outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/phone-page-ocr-audit/phone-page-ocr-audit.md

Current raw-vs-baseline audit result: failed with one blocker.

  • phone-pxl-raw-2: candidate 58 lines, baseline 57, delta +1

The audit now also records physical row-band diagnostics from the actual document.json line boxes. This matters because a line-count delta can mean either a new physical row or a different decomposition of the same row into column boxes.

  • phone-pxl-raw-1: candidate 21 lines, baseline 21, candidate 21 row bands, baseline 20 row bands. The difference is the rescued tiny top marker being separated rather than grouped with the title/header band.
  • phone-pxl-raw-2: candidate 58 lines, baseline 57, but both have 33 physical row bands. The mismatch is a box-composition issue inside the same row structure, not an extra physical text row.

The audit report now names the row-band composition disagreements rather than only reporting the bare +1 count:

  • phone-pxl-raw-1: band 1 differs because the raw run separates the narrow top marker/header behavior differently than the manually prepared baseline.
  • phone-pxl-raw-2: band 18 is candidate 1 box vs baseline 2; band 27 is candidate 2 boxes vs baseline 1; band 31 is candidate 2 boxes vs baseline 1.

Visual diagnostics for the lower page-2 disagreement are:

  • outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/diagnostics/candidate-page2-lower-lines.png
  • outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/diagnostics/page2-suspicious-crops-candidate-vs-baseline.png

There is also a reviewer-facing row-composition packet:

uv run python tools/prepare_limbu_phone_row_composition_review.py \
  --audit-json outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/phone-page-ocr-audit/phone-page-ocr-audit.json \
  --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/row-composition-review

Packet artifact:

outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/row-composition-review/phone-row-composition-review.html

The packet has 4 pending rows and copied crop images for every candidate and baseline side; an HTML link check found 12 images, 12 links, and 0 missing targets.

Those diagnostics show the disagreement is over real printed lower-page text, not keyboard/table background or illustration content. Therefore do not solve this by blindly suppressing the extra candidate box. The stricter line-count gate is useful as a warning against unnoticed drift, but the manually prepared baseline is not a human gold reference. The next benchmark needs a reviewed phone-page reference that can decide whether these lower rows should be merged or split in final reading order.

The current diagnosis is:

  • phone-pxl-raw-1 no longer has a title-order problem and no longer has a line-count problem. The first line is the rescued narrow marker, but OCR on that tiny crop is low quality.
  • phone-pxl-raw-2 now uses the right two-column reading order, but it still keeps one extra line relative to the manually prepared direct-checkpoint baseline. The new row-band and visual evidence shows this is a lower-page row-composition disagreement on real text, not a reading-order change and not a text/non-text false positive.

Replication note: --max-baseline-row-band-count-delta exists as an optional audit gate, but it is intentionally not part of the default strict command above. The row-band comparison is diagnostic unless a caller explicitly chooses to block on it, because page 1 shows that row-band grouping can differ even when the line inventory is correct.

This gate is intentionally strict, but it is still a pseudo-baseline gate. For these same phone photos, raw rectification and filtering must preserve physical row inventory and keep row-composition disagreements explicit before we can trust page-level OCR metrics or move on to claiming full book OCR quality. Exact box-count equality against the manually prepared page is not a substitute for a reviewed phone-page reference.

Phone Benchmark Reference Draft

tools/prepare_limbu_phone_page_benchmark_reference_draft.py turns the current raw phone OCR artifact into a reviewable reference scaffold. It deliberately does not create gold data. It copies the raw and prepared page images, writes one .ref.json file per page using the current OCR text and line boxes, writes an eval manifest that points at those draft references, and writes a browser HTML page with line-box overlays plus a textarea view of the draft text.

Replication command:

uv run python tools/prepare_limbu_phone_page_benchmark_reference_draft.py \
  --run-json outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/limbu-phone-page-ocr-run.json \
  --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft

Reference-draft output:

outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/

Key files:

  • limbu-phone-page-reference-draft.json
  • eval-manifest.jsonl
  • limbu-phone-page-reference-draft.html
  • references/phone-pxl-raw-1.ref.json
  • references/phone-pxl-raw-2.ref.json
  • images/phone-pxl-raw-1-prepared.png
  • images/phone-pxl-raw-1-raw.jpg
  • images/phone-pxl-raw-2-prepared.png
  • images/phone-pxl-raw-2-raw.jpg

Generated summary:

  • samples: 2
  • draft lines: 79
  • phone-pxl-raw-1: 21 draft reference lines
  • phone-pxl-raw-2: 58 draft reference lines
  • claim boundary: machine OCR draft references only; not a gold benchmark until reviewed

Every draft reference sets:

  • metadata.reference_status: draft_machine_ocr_needs_human_review
  • metadata.claim_evidence_eligible: false
  • metadata.text_ground_truth_certified: false
  • metadata.requires_human_review: true

This is required for replication: downstream scripts can use the manifest and reference paths, but no metric from this draft is claim evidence. A human review must correct text, decide the row-composition disagreements, and then change the reference metadata to verified/claim-eligible before page CER or reading-order claims are allowed.

Structural reference audit command:

uv run ocrtech audit-references \
  outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \
  --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/reference-audit

Audit result:

  • passed: true
  • samples: 2
  • parsed references: 2
  • text references: 2
  • reading-order references: 2
  • claim-ready references: 0
  • issues: 0
  • warnings: 0

Path verification after generation found all copied raw images, prepared images, and reference JSON files present. The HTML review page had 4 local links and 0 missing targets.

After the draft is corrected and marked claim-ready, score the saved page OCR documents with tools/run_page_ocr_eval.py using reference-draft/eval-manifest.jsonl as --eval-manifest and the raw-wrapper documents/ directory as --documents-dir. For a claim-grade run, first require:

  • uv run ocrtech audit-references ... --require-claim-ready
  • a passing page-output audit for every scored page output directory
  • --require-claim-safe in tools/run_page_ocr_eval.py

The central scorer now has its own reference claim-readiness gate. It records metrics.reference_claim_readiness with:

  • ready_count
  • scored_pages
  • status_counts
  • claim_eligible_counts
  • not_ready_count
  • not_ready_sample_ids

Under --require-claim-safe, tools/run_page_ocr_eval.py now rejects any page whose loaded reference file is not both reference_status=verified and claim_evidence_eligible=true. This protects the harness from accidentally scoring reference-draft/ as if it were a phone-photo benchmark.

Current fail-closed check against the active draft references and filtered phone documents:

failed_as_expected=true
contains_reference_gate=true
references are not all claim-ready: 0/2 have reference_status=verified and claim_evidence_eligible=true

This check is intentionally not a benchmark result. It proves that the current draft cannot be promoted by mistake. A real phone-page benchmark still needs reference-verified/eval-manifest.jsonl from the apply step below, plus a claim-safe page eval run that writes page-eval-run.json, metrics.json, metrics.md, claim-safety.md, page-metrics.csv, line-alignment.csv, and the review artifacts.

Phone Reference Review And Promotion Gate

tools/prepare_limbu_phone_page_reference_review.py converts the draft references into a line-level review packet. This is the missing bridge between machine OCR output and a real phone-photo benchmark. It copies every line crop into the packet, writes a CSV that can be edited/exported from the browser, and writes an HTML review surface with per-line crop, OCR draft text, review text, review status, reviewer, and notes.

Replication command:

uv run python tools/prepare_limbu_phone_page_reference_review.py \
  --eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \
  --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-review

Review packet output:

outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-review/

Key files:

  • phone-page-reference-review.json
  • phone-page-reference-review.csv
  • phone-page-reference-review.jsonl
  • phone-page-reference-review.html
  • crops/phone-pxl-raw-1/*.png
  • crops/phone-pxl-raw-2/*.png

Generated summary:

  • samples: 2
  • review rows: 79
  • copied crop images: 79
  • missing crop images: 0
  • initial status counts: pending: 79
  • HTML local image references: 79
  • missing HTML targets: 0

The review status contract is:

  • accept: keep OCR text after human verification.
  • corrected: keep review_text; review_text must be non-empty.
  • drop: omit a non-text/noise/duplicate line from the verified reference.
  • pending, needs_context, bad_segmentation, and needs_resegmentation block promotion.

Every accepted, corrected, or dropped row must carry a reviewer. The HTML export keeps the same CSV schema, so a reviewed CSV can be applied without hand conversion.

tools/apply_limbu_phone_page_reference_review.py is the only path that should promote this packet into claim-ready references. It refuses to write verified references unless:

  • every draft line has exactly one review row;
  • every row has a final non-blocking status;
  • kept rows have non-empty text;
  • kept rows satisfy any requested target codepoint range, for the current Sirijonga pages use --required-codepoint-range U+1900..U+194F;
  • a reviewer is present, either per row or through --reviewer;
  • --tables-reviewed, --figures-reviewed, and --captions-reviewed are explicitly supplied.

Apply command after human review:

uv run python tools/apply_limbu_phone_page_reference_review.py \
  --packet-dir outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-review \
  --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-verified \
  --reviewer <human-reviewer-id> \
  --required-codepoint-range U+1900..U+194F \
  --tables-reviewed \
  --figures-reviewed \
  --captions-reviewed \
  --dataset limbu-phone-page-verified \
  --split eval

The same apply command was intentionally run against the untouched packet and failed closed because all rows are still pending:

review_status pending blocks verification

No verified phone benchmark exists yet. The output directory reference-verified/ is reserved for the future completed review result.

Focused verification for this work:

uv run pytest \
  tests/test_limbu_phone_page_reference_review.py \
  tests/test_prepare_limbu_phone_page_benchmark_reference_draft.py \
  tests/test_prepare_page_ocr_eval_pack_metadata.py \
  tests/test_prepare_limbu_phone_row_composition_review.py \
  tests/test_audit_limbu_phone_page_ocr_run.py \
  tests/test_audit_page_ocr_documents.py \
  tests/test_run_page_ocr_eval.py \
  tests/test_materialize_page_image_line_boxes.py \
  tests/test_filter_page_ocr_documents.py \
  tests/test_run_limbu_phone_page_ocr.py \
  tests/test_prepare_limbu_book_captures.py \
  tests/test_capture.py \
  tests/test_audit_yolo_line_detector_predictions.py \
  tests/test_export_limbu_line_detector_dataset.py \
  tests/test_preflight_limbu_line_detector_training.py \
  tests/test_build_limbu_book_page_manifest.py \
  tests/test_build_limbu_pdf_text_context.py \
  -q

Result: 139 passed, with only the existing SWIG deprecation warnings from PDF rendering imports.

What Is Still Missing

The missing part is not mainly recognition. It is page understanding.

Required next work:

  • Harden the raw-phone-photo path so automatic rectification and detector filtering preserve physical text rows and make row-composition decisions auditable. The current wrapper exists, the corrected auto-order run fixes the title ordering, and the header rescue fixes page 1 line inventory. Page 2 still has a lower-page row-composition disagreement relative to the manually prepared direct-checkpoint run.
  • Package or document a stable combined runtime. The direct checkpoint path runs on the AWS G6 environment, but the local machine currently lacks a combined PaddleOCR plus Ultralytics/Torch runtime.
  • Replace the current heuristic OCR-line filter with trained/evaluated text vs non-text classification. The replayable filter now handles script, confidence, geometry, and page-marker suppression, but images, headers, keyboard/table artifacts, decorative marks, and borderline marginalia still need scored classifier behavior before this becomes a general book pipeline.
  • Keep hardening reading-order logic for single-column book pages and two-column newspaper pages. Wide-column LTR/RTL ordering and header preservation exist, but they are not yet scored against gold page order.
  • Turn the phone-reference draft into a frozen phone-photo benchmark with raw image, rectified page image, reviewed text-line boxes, reviewed reading order, and final Unicode text.
  • Report page-level metrics separately from recognizer oracle-crop CER: line recall, line precision, reading-order accuracy, page CER, review burden, and failure flags.

Until those are done, we can say: the page OCR harness can run raw phone JPEGs through page prep, the trained line detector checkpoint, and the recognizer, but the result is not yet a scored, production-ready "feed in book pages and get OCR" system.