# Limbu Page OCR Harness Updated: 2026-06-23 This document tracks the page-level Limbu OCR harness. Do not confuse the recognizer result with full page OCR. The current measured recognizer champion is useful on line crops, but raw phone photos require additional stages: page rectification, text/non-text segmentation, line grouping, reading order, and end-to-end page scoring. ## Current State The June 23 phone-photo smoke run used the two latest `PXL_*` images from `~/Downloads`: - `/Users/cdjk/Downloads/PXL_20260623_132115733.jpg` - `/Users/cdjk/Downloads/PXL_20260623_132120681.jpg` For reproducible local runs, those Downloads originals are staged under: - `incoming/limbu-phone-pxl-raw-2026-06-23/PXL_20260623_132115733.jpg` - `incoming/limbu-phone-pxl-raw-2026-06-23/PXL_20260623_132120681.jpg` The staged hashes match the Downloads originals and the raw page manifest: - `PXL_20260623_132115733.jpg`: `4167ec57b84731bce3e275224d15cf6589dafdadac110c6c9b51ae9ff36b570d` - `PXL_20260623_132120681.jpg`: `8df48d79a22521b0be536d3f4d87d6a1678107e164013a082b22f4cd8a99a7ca` Plain UTF-8 OCR extracts from the current best filtered phone run are saved at `examples/limbu-phone-photo/` as `ocr1.txt` and `ocr2.txt`. They come from `ocr-line-filter-sirijonga-conf050-margin-markers/*/document.json`, not from the older naive full-frame run. Current counts are `20` lines / `743` characters for `ocr1.txt` and `58` lines / `2334` characters for `ocr2.txt`. The first naive full-frame run is: `outputs/limbu-phone-photo-ocr-test-2026-06-23/` It completed, but it OCR'd background and non-text because the prepared images still contained keyboard/table/illustration regions. Summary: - pages: `2` - detected lines: `497` - review rows: `388` - operator attention pages: `2` - suspicious layout gaps: `10` The manually cropped baseline run is: `outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/` It uses the current recognizer package: `runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json` The cropped run is better, but still fails the page-OCR production gate: - pages: `2` - detected lines: `280` - review rows: `191` - operator attention pages: `2` - suspicious layout gaps: `2` - `phone-pxl-ocr-1`: `82` lines, `54` review rows - `phone-pxl-ocr-2`: `198` lines, `183` review rows The current best automatic text dumps are written to: - `examples/limbu-phone-photo/ocr1.txt` - `examples/limbu-phone-photo/ocr2.txt` These are Unicode, but they are not clean OCR output. Treat them as a failure sample for the page harness, not as digitized text. ## New Harness Gate `tools/audit_limbu_phone_page_ocr_run.py` audits page OCR runs for production readiness. It checks: - too many detected lines per page - high review-row ratio - high other-script ratio - suspicious layout gaps - operator-attention flags - detector fragmentation from `image-line-crops/manifest.jsonl` - materializer-wrapper runs from `tools/run_limbu_phone_page_ocr.py` - optional raw-vs-baseline line-inventory deltas against a manually prepared materializer run Current audit command: ```bash uv run python tools/audit_limbu_phone_page_ocr_run.py \ outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/limbu-book-ocr-run.json \ --out outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/phone-page-ocr-audit \ --exit-zero ``` Current result: `outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/phone-page-ocr-audit/phone-page-ocr-audit.md` It fails with twelve blockers. The important new signal is geometry, not script: the recognizer keeps many narrow fragments as confident Sirijonga. The audit now reads each page crop manifest and blocks the run when kept crops look like word/glyph fragments instead of full text lines. Current detector-fragmentation numbers: - `phone-pxl-ocr-1`: `82` kept crops, median kept width ratio `0.060`, narrow kept crop ratio `0.500`, max same-row fragments `9` - `phone-pxl-ocr-2`: `198` kept crops, median kept width ratio `0.045`, narrow kept crop ratio `0.712`, max same-row fragments `15` The production gate defaults are: - median kept crop width ratio must be at least `0.080` - no more than `0.350` of kept crops may be narrower than `0.060` of page width - no estimated row may contain more than `4` kept detector fragments These thresholds are not a recognition fix. They are a stop sign: the current morphology detector is splitting page text into fragments, and the recognizer is then hallucinating valid Limbu text on those fragments. This is consistent with the existing CDC PDF-text detector sweep: `outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-detector-sweep-v1/limbu-pdf-text-detector-sweep.json` The best morphology sweep covers `76` of `81` source PDF-text lines, but emits `335` detected boxes. That is useful as a recall baseline, but it is not a production page detector because the candidate count is far above the source line count. ## Detector Training Dataset Export `tools/export_limbu_line_detector_dataset.py` now exports supervised text-line detector labels from the page image plus PDF text-layer geometry artifacts. It writes: - `coco-text-line-detection.json` - YOLO labels under `labels/train/` and `labels/val/` - copied/augmented page images under `images/train/` and `images/val/` - `yolo-data.yaml` - `line-detector-labels.jsonl` with per-line provenance - `line-detector-dataset.json` with hashes and source-boundary metadata The exporter supports repeated `--page-manifest` and repeated `--pdf-text-context-jsonl` arguments, so multiple rendered books can be combined without hand-concatenating files. It also has an explicit `--line-script-filter` for `limbu`, `devanagari`, `limbu-or-devanagari`, or `any`; this matters because the page OCR goal covers both Sirijanga/Namdhinggo Limbu and Devanagari-written Limbu. The text boundary is strict: PDF text-layer Unicode is provenance/review context only. The exported labels are geometry supervision for a text-line detector, not recognizer gold text. The first 5-page CDC smoke export is: ```bash uv run python tools/export_limbu_line_detector_dataset.py \ --page-manifest outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/raw-page-manifest/book-pages.jsonl \ --pdf-text-context-jsonl outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/pdf-text-context/limbu-pdf-text-context-lines.jsonl \ --out outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23 \ --include-phone-augmentations ``` Current output: `outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/line-detector-dataset.json` Summary: - pages: `5` - source context rows: `82` - base line boxes: `81` - exported images: `20` - exported annotations: `324` - train images: `16` - val images: `4` - train pages: `cdc-maths-p0011` through `cdc-maths-p0014` - val page: `cdc-maths-p0015` - variants: `original`, `phone-blur-lowlight`, `phone-rotate-left`, `phone-rotate-right` This is the first actual training handoff for replacing the morphology detector. It proved the training path, but it is too small for a real detector. The current larger CDC Grade 1 source pack is: `outputs/limbu-cdc-grade1-detector-sources-2026-06-23/` It renders the three mirrored CDC curriculum PDFs at `200` DPI and maps their PDF text-layer boxes onto the rendered page images: - Maths PDF: `111` rendered pages, `4352` text-layer lines, `3915` lines with Limbu codepoints after legacy conversion - Science PDF: `89` rendered pages, `1386` text-layer lines, `1316` lines with Limbu codepoints after legacy conversion - Social PDF: `87` rendered pages, `1316` text-layer lines, `1310` lines with Limbu codepoints after legacy conversion The combined detector export command is: ```bash uv run python tools/export_limbu_line_detector_dataset.py \ --page-manifest outputs/limbu-cdc-grade1-detector-sources-2026-06-23/maths/book-pages.jsonl \ --page-manifest outputs/limbu-cdc-grade1-detector-sources-2026-06-23/science/book-pages.jsonl \ --page-manifest outputs/limbu-cdc-grade1-detector-sources-2026-06-23/social/book-pages.jsonl \ --pdf-text-context-jsonl outputs/limbu-cdc-grade1-detector-sources-2026-06-23/maths/pdf-text-context/limbu-pdf-text-context-lines.jsonl \ --pdf-text-context-jsonl outputs/limbu-cdc-grade1-detector-sources-2026-06-23/science/pdf-text-context/limbu-pdf-text-context-lines.jsonl \ --pdf-text-context-jsonl outputs/limbu-cdc-grade1-detector-sources-2026-06-23/social/pdf-text-context/limbu-pdf-text-context-lines.jsonl \ --out outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23 \ --include-phone-augmentations \ --split-strategy hash \ --val-page-ratio 0.15 ``` Current combined dataset: `outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23/line-detector-dataset.json` Summary: - source page manifests: `3` - source context files: `3` - rendered pages: `287` - source context rows: `7054` - base line boxes: `6540` - skipped non-Limbu/script-filter rows: `513` - skipped invalid bboxes: `1` - exported images: `1148` - exported annotations: `26160` - train images: `976` - val images: `172` - split strategy: deterministic `hash` - variants: `original`, `phone-blur-lowlight`, `phone-rotate-left`, `phone-rotate-right` The split remains page-level, not augmentation-level, so augmented copies of the same page do not leak across train and validation. `tools/preflight_limbu_line_detector_training.py` validates the exported detector dataset and writes the exact training command. Current preflight: ```bash uv run python tools/preflight_limbu_line_detector_training.py \ outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/line-detector-dataset.json \ --out outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/training-preflight \ --model yolov8n.pt \ --epochs 100 \ --imgsz 1280 \ --batch 4 \ --project runs/limbu-line-detector \ --name cdc-maths-p0011-p0015-v2 ``` Current result: `outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/training-preflight/line-detector-training-preflight.json` The 5-page smoke dataset preflight passes with `0` issues. The larger all-books dataset preflight also passes with `0` issues: `outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23/training-preflight/line-detector-training-preflight.json` Local runtime still lacks `torch`/`ultralytics`, so detector training runs on the AWS G6 host. ## AWS G6 Detector Smoke Training The first supervised line-detector smoke run was executed on the existing AWS `playaround` G6 instance: - account: `798006441552` - instance: `i-0fa19dff7b233a353` - name: `ocrtech-g6-training-20260615-v4` - type: `g6.xlarge` - runtime path: `/opt/ocrtech` - GPU: NVIDIA L4 Remote preflight passed with runtime checks enabled: ```bash cd /opt/ocrtech .venv/bin/python tools/preflight_limbu_line_detector_training.py \ outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/line-detector-dataset.json \ --out outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/aws-training-preflight-v2 \ --require-runtime \ --model yolov8n.pt \ --epochs 100 \ --imgsz 1280 \ --batch 4 \ --project /opt/ocrtech/runs/limbu-line-detector \ --name cdc-maths-p0011-p0015-v2 ``` Local copy of the AWS preflight: `outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/aws-training-preflight-v2/line-detector-training-preflight.json` The training run completed: ```bash cd /opt/ocrtech/outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23 /opt/ocrtech/.venv/bin/yolo detect train \ data=yolo-data.yaml \ model=/opt/ocrtech/yolov8n.pt \ epochs=100 \ imgsz=1280 \ batch=4 \ project=/opt/ocrtech/runs/limbu-line-detector \ name=cdc-maths-p0011-p0015-v2 \ exist_ok=True ``` Local copy of the detector run: `outputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/` Important artifacts: - `outputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/weights/best.pt` - `outputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/weights/last.pt` - `outputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/results.csv` - `outputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/train.log` Best validation row by mAP50 was epoch `94`: - precision: `0.39716` - recall: `0.31944` - mAP50: `0.22702` - mAP50-95: `0.13800` The final validation of `best.pt` reported precision `0.399`, recall `0.319`, mAP50 `0.228`, and mAP50-95 `0.136` over `4` validation images with `72` line instances. This checkpoint proves the supervised training path works, but it is not production-quality. The validation split is one CDC page plus augmentations, so the metric is a smoke signal only. The checkpoint was also smoke-tested against the two real PXL prepared page images. Local prediction artifacts: `outputs/aws-g6-limbu-line-detector-phone-predict/` Detector counts: - cropped pages, `conf=0.05`: `125`, `245` - cropped pages, `conf=0.25`: `21`, `48` - cropped pages, `conf=0.50`: `0`, `0` - auto-detected pages, `conf=0.05`: `128`, `228` - auto-detected pages, `conf=0.25`: `25`, `44` - auto-detected pages, `conf=0.50`: `0`, `1` At `conf=0.25`, the predicted boxes are sometimes line-sized, but they still fragment rows, miss lower text, and admit illustration/photo regions. This is not good enough to replace the current page detector. The right conclusion is: the training path is alive, but the detector needs substantially more page-level supervision and likely negative/non-text examples before recognizer integration. ## AWS G6 Detector All-Books Training The larger CDC Grade 1 all-books detector dataset was synced to the same AWS G6 host and passed remote runtime preflight: `outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23/aws-training-preflight/line-detector-training-preflight.json` Remote training command: ```bash cd /opt/ocrtech/outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23 /opt/ocrtech/.venv/bin/yolo detect train \ data=yolo-data.yaml \ model=/opt/ocrtech/yolov8n.pt \ epochs=80 \ imgsz=1280 \ batch=4 \ project=/opt/ocrtech/runs/limbu-line-detector \ name=cdc-grade1-allbooks-v1 \ exist_ok=True ``` Launch metadata: - AWS profile: `playaround` - instance: `i-0fa19dff7b233a353` - public IP used for artifact pull and phone prediction: `3.219.242.144` - remote run directory: `/opt/ocrtech/runs/limbu-line-detector/cdc-grade1-allbooks-v1` - remote launch log: `/opt/ocrtech/runs/limbu-line-detector-launch/cdc-grade1-allbooks-v1/train.log` - remote PID at launch: `1776` The run completed `80` epochs in `0.720` hours. YOLO scanned `976` train images and `172` validation images with `0` background, corrupt, or missing-label issues. Local copy of the completed detector run: `outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/` Important artifacts: - `outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/weights/best.pt` - `outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/weights/last.pt` - `outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/results.csv` - `outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/train.log` Best validation row by mAP50 was epoch `79`: - precision: `0.93159` - recall: `0.85800` - mAP50: `0.92478` - mAP50-95: `0.72025` The final epoch `80` row reported precision `0.92947`, recall `0.85318`, mAP50 `0.91988`, and mAP50-95 `0.71604`. The final validation of `best.pt` reported precision `0.937`, recall `0.845`, mAP50 `0.921`, and mAP50-95 `0.724` over `172` validation images and `4148` text-line instances. This checkpoint is no longer a toy detector on the CDC rendered/augmented validation split. It still does not prove page OCR quality, because the validation set is derived from PDF-rendered curriculum pages, not raw phone photos. Operational note: on 2026-06-23, the local AWS `playaround` profile had an expired SSO session. SSH access to the known instance still worked with `~/.ssh/ocrtech-g6-training-20260615.pem`, but EC2 API actions such as `describe-instances` and `stop-instances` could not be run until SSO was refreshed. The remote host also had no AWS credentials or instance role available to `aws sts get-caller-identity`, so it could not stop itself through the AWS CLI either. ## All-Books Checkpoint Phone-Photo Smoke The completed all-books checkpoint was tested on the same two prepared PXL phone page images used by the failed morphology-detector run: - cropped prepared pages: `outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/capture-prep/` - auto-detected prepared pages: `outputs/limbu-phone-photo-auto-detect-capture-2026-06-23/` Remote prediction command pattern: ```bash cd /opt/ocrtech .venv/bin/yolo detect predict \ model=/opt/ocrtech/runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt \ source=/opt/ocrtech/tmp/limbu-phone-detector-input/ \ imgsz=1280 \ conf= \ project=/opt/ocrtech/runs/limbu-line-detector-phone-predict-allbooks \ name= \ save=True \ save_txt=True \ save_conf=True \ exist_ok=True ``` Local prediction artifacts: `outputs/aws-g6-limbu-line-detector-phone-predict-allbooks/` Detector counts: - cropped pages, `conf=0.25`: `23`, `81` - cropped pages, `conf=0.35`: `23`, `74` - cropped pages, `conf=0.50`: `23`, `67` - auto-detected pages, `conf=0.25`: `24`, `78` - auto-detected pages, `conf=0.35`: `22`, `72` - auto-detected pages, `conf=0.50`: `22`, `64` Geometry audit results: - `cropped-conf025`: failed, `104` predictions, blocker `same_row_fragmentation_too_high:6>4` - `cropped-conf035`: passed, `97` predictions - `cropped-conf050`: passed, `90` predictions - `auto-conf025`: failed, `102` predictions, blocker `same_row_fragmentation_too_high:5>4` - `auto-conf035`: failed, `94` predictions, blocker `same_row_fragmentation_too_high:5>4` - `auto-conf050`: passed, `86` predictions The most useful smoke candidate is `cropped-conf050`: it passes the geometry gate and visually produces real line-sized boxes instead of the earlier word/glyph fragments. It still has false positives on non-page/background text and small decorative fragments. The detector materializer below now handles direct checkpoint inference, saved-label ingestion, basic filtering, duplicate suppression, and explicit two-column reading order. This is still not a finished end-to-end page OCR benchmark because there is no gold page text/CER for these phone photos and the raw-photo wrapper still has to combine rectification, detector inference, recognition, scoring, and audit gates in one reproducible command. ## YOLO Detector Page OCR Materialization `tools/materialize_page_image_line_boxes.py` now supports using the trained detector instead of the old morphology detector: - `--line-detector yolo-labels` - `--yolo-labels-dir ` - `--line-detector yolo-model` - `--yolo-model-path ` - confidence, size, area, and page-region filters - same-row overlap duplicate suppression for YOLO detections - explicit wide-column reading orders: `wide_column_left_to_right`, `wide_column_right_to_left`, and split-tall variants - header-preserving wide-column ordering, so page titles above body columns stay before the column body text - per-page `document.json`, `document.md`, and `document.body.md` The duplicate suppression is deliberately recorded in the run metadata. It drops a nested same-row fragment when the fragment substantially overlaps a kept row. A much wider row wins over a contained fragment even if the fragment has higher detector confidence; if widths are similar, higher confidence wins. The current defaults are: - enabled: `true` - overlap fraction of smaller box: `0.75` - wider-box preference ratio: `1.12` - y-tolerance: automatic when `0.0` Current PXL eval manifest: `outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-2026-06-23/eval-manifest.jsonl` Saved-label materialization command: ```bash PYTHONPATH=src /tmp/ocrtech-paddle311/bin/python tools/materialize_page_image_line_boxes.py \ --eval-manifest outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-2026-06-23/eval-manifest.jsonl \ --model-config runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json \ --out outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents \ --engine candidate \ --recognition-mode line \ --line-detector yolo-labels \ --yolo-labels-dir outputs/aws-g6-limbu-line-detector-phone-predict-allbooks/cropped-conf050/labels \ --yolo-min-confidence 0.50 \ --yolo-min-width-ratio 0.04 \ --yolo-region-min-y-ratio 0.05 \ --reading-order wide_column_left_to_right \ --crop-padding 8 \ --allow-empty-lines ``` Saved-label output: `outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/` Run summary after same-row overlap suppression: - samples: `2` - total materialized OCR lines: `78` - empty recognizer outputs: `0` - detector source: `yolo_line_detector_labels_v1` - reading order: `wide_column_left_to_right` on both pages - crop manifest rows: `78` - `phone-pxl-ocr-1`: `21` lines, `0` suppressed duplicate detector rows - `phone-pxl-ocr-2`: `57` lines, `7` suppressed duplicate detector rows - high-overlap same-row duplicate pairs after suppression: `0` on both pages Key output files: - `outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-run.json` - `outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-crops.jsonl` - `outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/phone-pxl-ocr-1/document.md` - `outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/phone-pxl-ocr-2/document.md` Direct checkpoint materialization was run on the AWS G6 box because the local machine currently has no combined PaddleOCR plus Ultralytics/Torch runtime and only about `9.6GiB` free disk. The G6 runtime has `torch 2.12.1+cu130`, `ultralytics 8.4.75`, `paddle 3.3.1`, and `paddleocr 3.7.0`. Remote manifest: `/opt/ocrtech/outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/eval-manifest.jsonl` Direct checkpoint materialization command: ```bash cd /opt/ocrtech PYTHONPATH=src .venv/bin/python tools/materialize_page_image_line_boxes.py \ --eval-manifest outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/eval-manifest.jsonl \ --model-config runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json \ --out outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents \ --engine candidate \ --recognition-mode line \ --line-detector yolo-model \ --yolo-model-path runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt \ --yolo-imgsz 1280 \ --yolo-device 0 \ --yolo-min-confidence 0.50 \ --yolo-min-width-ratio 0.04 \ --yolo-region-min-y-ratio 0.05 \ --reading-order wide_column_left_to_right \ --crop-padding 8 \ --allow-empty-lines ``` Local copy of direct-checkpoint output: `outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/` Direct-checkpoint run summary: - samples: `2` - total materialized OCR lines: `78` - empty recognizer outputs: `0` - detector source: `yolo_line_detector_model_v1` - detector source pass count: `yolo-model: 78` - reading order: `wide_column_left_to_right` on both pages - crop manifest rows: `78` - `phone-pxl-ocr-1`: `21` lines, `0` suppressed duplicate detector rows - `phone-pxl-ocr-2`: `57` lines, `7` suppressed duplicate detector rows - high-overlap same-row duplicate pairs after suppression: `0` on both pages Key direct-checkpoint output files: - `outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-run.json` - `outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-crops.jsonl` - `outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/phone-pxl-ocr-1/document.md` - `outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/phone-pxl-ocr-2/document.md` The direct checkpoint and saved-label paths produce the same line counts and the same page-1 body Markdown. Page 2 is not byte-identical because the saved labels store rounded normalized box coordinates while direct model inference returns full-precision boxes; small crop-boundary differences change some recognizer outputs. The direct-checkpoint output should be treated as the authoritative current page OCR artifact. This is the first page-level path that is meaningfully better than the toy morphology OCR: the trained detector checkpoint feeds line crops directly, the recognizer returns Unicode for every kept crop, document JSON/Markdown are emitted, title lines stay above wide-column body text, and duplicated detector fragments are audited and suppressed. It is still not enough to claim "feed in Limbu books and get OCR" because the current entrypoint starts from prepared page images rather than raw captures, has only geometry-based text/non-text filtering, and lacks a gold page transcript/CER benchmark for these real phone photos. ## YOLO Prediction Geometry Audit `tools/audit_yolo_line_detector_predictions.py` audits saved YOLO prediction labels before the detector is allowed into phone-page OCR. It does not need text references. It checks: - total prediction count per page - median predicted box width ratio - narrow predicted box ratio - same-row fragmentation from normalized y-centers This targets the real phone-photo failure mode: many confident narrow boxes that look like words/glyph fragments, not text lines. The default blockers are: - more than `120` predictions on a page - median box width below `0.080` of page width - more than `0.350` of boxes narrower than `0.060` of page width - more than `4` boxes in the same estimated row Current focused test command: ```bash uv run pytest \ tests/test_audit_yolo_line_detector_predictions.py \ tests/test_export_limbu_line_detector_dataset.py \ tests/test_preflight_limbu_line_detector_training.py \ tests/test_build_limbu_book_page_manifest.py \ tests/test_build_limbu_pdf_text_context.py \ -q ``` Current result: `24 passed`. ## Capture Fix Added `src/ocrtech/capture.py` now has an automatic page-detection fallback. The previous implementation only accepted a closed four-corner Canny contour. The fallback detects the dominant bright, low-saturation page-like region and uses a minimum-area rectangle when the contour path fails. On the two real PXL images, auto-detection now completes instead of crashing: ```bash /tmp/ocrtech-paddle311/bin/python tools/prepare_limbu_book_captures.py \ --page-manifest outputs/limbu-phone-photo-ocr-test-2026-06-23/raw-page-manifest/book-pages.jsonl \ --out outputs/limbu-phone-photo-auto-detect-capture-2026-06-23 \ --auto-detect-page \ --auto-deskew \ --allow-failed-capture-audit ``` The run writes: `outputs/limbu-phone-photo-auto-detect-capture-2026-06-23/` Both pages use `method=light_region_min_area_rect` and record `touches_image_border=true`. The capture audit passes but warns that the source may be clipped or include background. This is deliberate: the harness should surface bad capture geometry instead of silently producing confident OCR. ## Capture Quality Gates `src/ocrtech/capture.py` now records `laplacian_variance` in `image_quality.source` and `image_quality.prepared`. This is the current sharpness/blur signal. The existing `entropy`, `luminance_stddev`, and `edge_stddev` metrics remain in place, and the audit can now enforce all of these as prepared-image thresholds. `audit_limbu_capture()` and `tools/prepare_limbu_book_captures.py` now expose phone-capture policy gates for: - minimum prepared width and height; - minimum prepared entropy; - minimum prepared luminance standard deviation; - minimum prepared edge standard deviation; - minimum prepared Laplacian variance; - requiring automatic page detection; - minimum detected page area ratio; - optionally failing when auto-detected page corners touch the source image border; - requiring applied auto-deskew; - maximum absolute applied deskew angle. For the current PXL smoke pages, border-touch remains a warning rather than a failure. Both pages touch the source image border, which means the page may be clipped or include background, but rejecting them would reject the actual hard phone photos we need to handle. Use `--fail-on-auto-detect-page-border-touch` only for a stricter capture-quality batch. Local quality-gate replay manifest: `outputs/limbu-phone-capture-quality-gates-2026-06-23/raw-page-manifest/book-pages.jsonl` Replay command: ```bash uv run python tools/prepare_limbu_book_captures.py \ --page-manifest outputs/limbu-phone-capture-quality-gates-2026-06-23/raw-page-manifest/book-pages.jsonl \ --out outputs/limbu-phone-capture-quality-gates-2026-06-23/capture-prep \ --auto-detect-page \ --auto-deskew \ --min-prepared-width 2500 \ --min-prepared-height 3500 \ --min-prepared-entropy 5.0 \ --min-prepared-luminance-stddev 20 \ --min-prepared-edge-stddev 10 \ --min-prepared-laplacian-variance 50 \ --require-auto-detect-page \ --min-auto-detect-page-area-ratio 0.50 \ --require-auto-deskew-applied \ --max-abs-auto-deskew-degrees 4.0 ``` Replay artifact: `outputs/limbu-phone-capture-quality-gates-2026-06-23/capture-prep/` Result: - pages: `2` - failed capture audits: `0` - both pages warn: `auto_detect_page touches image border` Page quality measurements: - `phone-pxl-raw-1`: prepared size `2945x4059`, entropy `6.126979`, luminance stddev `57.978733`, edge stddev `26.103824`, Laplacian variance `243.456146`, page area ratio `0.6772529951108047`, deskew `1.6` degrees. - `phone-pxl-raw-2`: prepared size `3013x4165`, entropy `6.104708`, luminance stddev `46.921525`, edge stddev `16.186986`, Laplacian variance `109.91304`, page area ratio `0.7292922076056986`, deskew `1.7` degrees. The raw phone wrapper exposes the same policy switches, so a future end-to-end OCR run can fail before recognition when a photo is too small, too flat, too blurry, lacks page detection, lacks deskew evidence, or is more skewed than the allowed limit. Compatibility note: capture metadata written before `laplacian_variance` existed does not fail replay only because that newer metric is absent. The audit emits a warning such as `image_quality.prepared metrics missing newer replay fields: laplacian_variance`. If a caller sets `--min-prepared-laplacian-variance`, then legacy metadata without that metric fails because the requested sharpness gate cannot be evaluated. ## Raw Phone-Photo Wrapper Run `tools/run_limbu_phone_page_ocr.py` is now the reproducible raw-photo entrypoint. It does three things in one command: 1. Runs `tools/prepare_limbu_book_captures.py` from a raw page manifest. 2. Converts the prepared-page manifest into a materializer eval manifest. 3. Runs `tools/materialize_page_image_line_boxes.py` with the trained YOLO line detector and the Limbu recognizer package. The wrapper preserves the handoff files required by other scripts: - source raw manifest: `outputs/limbu-phone-page-ocr-raw-yolo-model-auto-2026-06-23/raw-page-manifest/book-pages.jsonl` - capture prep run: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/capture-prep/book-capture-prep-run.json` - prepared page manifest: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/capture-prep/prepared-page-manifest.jsonl` - materializer eval manifest: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/materializer-eval-manifest.jsonl` - materializer run: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/documents/image-line-box-run.json` - final per-page documents: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/documents/phone-pxl-raw-1/document.md` and `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/documents/phone-pxl-raw-2/document.md` - wrapper summary with hashes: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/limbu-phone-page-ocr-run.json` Replication-critical provenance rule: every script boundary must preserve the source page fields, not just the prepared image path. The capture-prep script now copies source fields from the raw page manifest into `capture-prep/prepared-page-manifest.jsonl`, the raw phone wrapper copies them into `materializer-eval-manifest.jsonl` under `metadata`, and the materializer copies that eval-manifest metadata into each final `document.json` under `metadata.source_manifest_metadata` and `pages[].metadata.source_manifest_metadata`. The fields currently preserved are: - `page_sequence_number` - `book_page_number` - `source_filename` - `source_document_path` - `source_document_sha256` - `source_document_page_number` - `source_document_page_index` - `source_kind` - `rendered_pdf_dpi` - `source_url` - `publication_date` - `edition` - `pdf_page_number` - `language` - `script` - `source_metadata` This is required for replication with other scripts: scoring, review packets, and future corpus exports must be able to trace an OCR line back to the exact source document/page without relying on filenames or memory. Do not strip these fields when creating derived manifests. The refreshed provenance run carries the available phone-capture metadata through all three handoff files. It does not yet carry CDC/PDF source-document fields because the raw PXL manifest itself does not identify the source PDF/book page. Current provenance-handoff verification: ```bash uv run pytest \ tests/test_prepare_limbu_book_captures.py \ tests/test_run_limbu_phone_page_ocr.py \ tests/test_materialize_page_image_line_boxes.py \ -q ``` Result: `40 passed`. The broader focused page-OCR suite, including the central page scorer, page-output audit tests, and page eval-pack metadata tests, also passed: `139 passed`, with only the existing SWIG deprecation warnings from PDF rendering imports. The raw source manifest rows point at the two uploaded PXL JPEGs on the AWS G6 box: - `incoming/limbu-phone-pxl-raw-2026-06-23/PXL_20260623_132115733.jpg` - `incoming/limbu-phone-pxl-raw-2026-06-23/PXL_20260623_132120681.jpg` AWS G6 command: ```bash cd /opt/ocrtech PYTHONPATH=src .venv/bin/python tools/run_limbu_phone_page_ocr.py \ --page-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-auto-2026-06-23/raw-page-manifest/book-pages.jsonl \ --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23 \ --model-config runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json \ --engine candidate \ --recognition-mode line \ --line-detector yolo-model \ --yolo-model-path runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt \ --yolo-imgsz 1280 \ --yolo-device 0 \ --yolo-min-confidence 0.50 \ --yolo-min-width-ratio 0.04 \ --yolo-header-min-width-ratio 0.015 \ --yolo-header-max-y-ratio 0.30 \ --yolo-header-min-confidence 0.75 \ --yolo-region-min-y-ratio 0.05 \ --reading-order auto_layout_left_to_right \ --crop-padding 8 \ --allow-empty-lines ``` Wrapper summary: - pages: `2` - OCR lines: `79` - empty OCR lines: `0` - capture failed audit count: `0` - materializer detector: `yolo_line_detector_model_v1` - detector source pass count: `yolo-model: 79` - requested reading order: `auto_layout_left_to_right` - reading-order selection counts: `top_to_bottom: 1`, `wide_column_left_to_right: 1` - detector checkpoint: `runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt` Capture operations are recorded per page in `capture-prep/book-capture-prep-run.json` under `pages[].capture_operations`. Both pages used the bright-region fallback, then perspective rectification, deskew, grayscale, and autocontrast: - `phone-pxl-raw-1`: `light_region_min_area_rect`, area ratio `0.6772529951108047`, `touches_image_border=true`, rectified size `2833x3981`, deskew angle `1.6`, prepared size `2945x4059` - `phone-pxl-raw-2`: `light_region_min_area_rect`, area ratio `0.7292922076056986`, `touches_image_border=true`, rectified size `2893x4079`, deskew angle `1.7`, prepared size `3013x4165` Materializer page summary: - `phone-pxl-raw-1`: `21` OCR lines, `0` empty lines, `0` same-row duplicate suppressions, reading order `top_to_bottom` - `phone-pxl-raw-2`: `58` OCR lines, `0` empty lines, `4` same-row duplicate suppressions, reading order `wide_column_left_to_right` This raw-wrapper run is useful but not yet good enough to call done. The manually prepared direct-checkpoint run materialized `21 + 57` lines. The first raw auto-wrapper run materialized `20 + 58` lines because page rectification and region filtering altered the detector input. The current header-rescue run materializes `21 + 58`: it fixes the missing top marker on page 1, but page 2 is still one line over the manually prepared baseline. A previous run forced `wide_column_left_to_right`, which was wrong for the first page and placed the title around line 14. The corrected `auto_layout_left_to_right` logic now selects `top_to_bottom` for `phone-pxl-raw-1` and `wide_column_left_to_right` for `phone-pxl-raw-2`. The first raw page starts with the rescued narrow top marker: ```text ᤍᤁ ᤈᤠ᤺ᤵᤐᤠ ᤛᤒᤠ ``` The rescued marker crop is very small and the recognized Unicode text is poor. That is expected for now. The important change is that the detector inventory is no longer dropping the page-1 header/page-marker line. The next real page-understanding bug is page-2 line inventory, not title order. The current YOLO filter policy has a narrow-header rescue: - normal minimum detector width is still controlled by `--yolo-min-width-ratio 0.04` - a narrow detection can pass only near the top of the image when all three header flags allow it: `--yolo-header-min-width-ratio 0.015`, `--yolo-header-max-y-ratio 0.30`, and `--yolo-header-min-confidence 0.75` - the rescue still obeys the normal confidence, region, height, and border filters, so bottom fragments do not reopen ## OCR Line Filter / Text-Nontext Gate `tools/filter_page_ocr_documents.py` is the current non-oracle post-materializer line filter. It reads existing page-eval-compatible `document.json` files, removes lines/blocks using only live OCR attributes, writes filtered `document.json` files, records one JSONL decision row per source line, and can be replay-audited by `tools/audit_page_ocr_line_filter.py`. This is not a learned text/non-text classifier yet. It is a reproducible gate for script, confidence, geometry, and structural-role decisions. The filter now supports: - `--drop-empty` - `--require-script sirijonga|devanagari|limbu_or_devanagari` - confidence thresholds with `--min-confidence` - absolute and page-relative width/height thresholds - `--drop-structural-role` and `--keep-structural-role` - `--infer-margin-page-markers`, which derives `page_marker` for tiny centered short boxes near the top or bottom page margin when older documents only recorded `structural_role: text` The materializer also marks the same small centered margin marker as `structural_role=page_marker` on new runs. The filter-side inference exists so older artifacts can be filtered without rerunning YOLO and OCR. This matters on the local machine because the recognizer model card exists, but the current YOLO detector weights are not present at `runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt`; the recorded full wrapper command is an AWS G6 command. Current replay command on the two-page PXL artifact: ```bash uv run python tools/filter_page_ocr_documents.py \ --eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/materializer-eval-manifest.jsonl \ --documents-dir outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/documents \ --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers \ --drop-empty \ --require-script sirijonga \ --min-confidence 0.50 \ --infer-margin-page-markers \ --drop-structural-role page_marker ``` Audit command: ```bash uv run python tools/audit_page_ocr_line_filter.py \ outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers ``` Filter artifact: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers/` Audit result: passed with `issues=0`. Current result: - source lines: `79` - kept lines: `78` - removed lines: `1` - removal counts: `structural_role_dropped: 1`, `low_confidence: 1` The removed line is `phone-pxl-raw-1/img-line-0001`. The crop is visually the page marker `52`, but the recognizer read it as `ᤍᤁ` with confidence `0.36888888478279114`. Its detected box is `[1345.6043701171875, 954.5286254882812, 54.516357421875, 66.0487060546875]` on a `2945x4059` prepared page, so the filter marks it `structural_role=page_marker`, `structural_role_reason=centered_margin_short_small_box`, and `structural_role_source=filter_inference`. Replication notes for downstream scripts: - Use the filtered directory as `--documents-dir` only when you explicitly want non-text/page-marker suppression. - Use the unfiltered `documents/` directory for detector recall, missed-line, and reference-review work, otherwise the review packet can hide detector output that still needs adjudication. - `ocr-line-filter-run.json` records the exact source manifest, source documents directory, filter parameters, counts, and output hashes. - `ocr-line-filter-lines.jsonl` is the per-line decision table; it is the file to inspect when a later script disagrees with the filtered line count. - `ocr-line-filter-audit.json` and `.md` are the replay proof. A filtered directory without a passing audit is not claim-ready. ## Page Output Audit For Scoring `tools/audit_page_ocr_documents.py` audits page-eval-compatible `document.json` output directories and writes the per-sample `limbu-output-audit.json` files required by `tools/run_page_ocr_eval.py --require-output-audit`. This is separate from `ocrtech audit-limbu-output`, which expects the older `limbu-output-manifest.json` pipeline-output shape and does not directly fit the page materializer/filter directories. The page document audit checks: - `document.json` parses as the shared `Document` schema; - `document.source_path` is readable; - `document.source_path` matches the eval-manifest input by resolved path or by SHA-256, so copied prepared images are allowed only when byte-identical; - page width/height and line bounding boxes are positive and inside the page; - line IDs are present and unique per page; - block `line_ids` point at real lines; - expected `ocr_output_kind` and detector source pass match policy; - line confidence is present when required; - line crop paths exist and `crop_sha256` matches when crop checking is enabled. Current command on the active filtered phone documents: ```bash uv run python tools/audit_page_ocr_documents.py \ --eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \ --documents-dir outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers \ --expected-ocr-output-kind live_detector \ --expected-detector-source-pass yolo-model \ --require-line-crops \ --require-line-confidence \ --min-line-count 1 ``` Audit artifacts: - aggregate: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers/page-ocr-documents-audit.json` - per page: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers/phone-pxl-raw-1/limbu-output-audit.json` - per page: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers/phone-pxl-raw-2/limbu-output-audit.json` Current result: - samples: `2` - passed samples: `2` - failed samples: `0` - audited lines: `78` - empty lines: `0` - source binding for both pages: `same_sha256` - crop hashes checked: `20` on `phone-pxl-raw-1`, `58` on `phone-pxl-raw-2` - detector source pass counts: `yolo-model: 78` After this audit and the scorer SHA-256 source-binding fix, the current claim-safe scorer dry run against `reference-draft/eval-manifest.jsonl` still fails, as it should, but it no longer reports either: - `page output audits are not all passing` - `document source_path mismatches detected` Against the original `reference-draft/eval-manifest.jsonl`, the remaining scorer blockers are now the real benchmark blockers plus missing scorer sidecars: draft references, non-frozen pack metadata, missing eval-pack datasheet/script inventory/dictionary/leakage evidence, only `2` scoring pages, and no layout-detection reference coverage. For a reproducible scorer dry run, first build the scorer sidecars described in the next section and use that rewritten manifest. ## Page Eval Pack Metadata Sidecars `tools/prepare_page_ocr_eval_pack_metadata.py` prepares the central scorer sidecars for an existing page-level OCR eval manifest. It writes a rewritten manifest with scorer-visible source provenance, a `summary.json` with the exact page-pack `pack_content_sha256`, and a `script-inventory.json` derived from the loaded reference text. It does not certify labels: draft references remain draft, `claim_evidence_eligible=false` remains false, and `--require-claim-safe` still fails until the references are human verified. This tool exists because the first phone reference draft had good reference files but did not expose whole-page `bbox` and `source_image` fields in the manifest keys that `tools/run_page_ocr_eval.py` audits. The phone draft writer now emits those fields for future runs, but this sidecar tool is the replication path for existing drafts and other page-manifest builders. Current sidecar-generation command: ```bash uv run python tools/prepare_page_ocr_eval_pack_metadata.py \ --eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \ --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack \ --eval-pack-version limbu-phone-page-reference-draft-v1 \ --force ``` Sidecar output: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack/` Key files: - `eval-manifest.jsonl` - `summary.json` - `script-inventory.json` - `page-ocr-eval-pack-metadata.json` - `README.md` Current sidecar result: - samples: `2` - reference lines: `79` - claim-ready references: `0/2` - `pack_content_sha256`: `b20abcbbe4c6f38e39be055d9d618d29d0ecdc3751a5eb38d50384d89fabed8e` - metadata coverage under the scorer: `missing_source_document=0`, `missing_page=0`, `missing_bbox=0`, `missing_claim_eligibility=0`, `missing_language=0`, `missing_script=0`, `missing_domain=0`, `missing_gt_status=0` - script inventory coverage under the scorer: `missing_required_codepoints=[]`, `missing_required_graphemes=[]` Scorer dry-run command using the sidecars and the filtered phone OCR documents: ```bash uv run python tools/run_page_ocr_eval.py \ --eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack/eval-manifest.jsonl \ --documents-dir outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers \ --ocr-output-kind live_detector \ --require-output-audit \ --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/page-eval-scorer-pack-dry-run \ --summary-file outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack/summary.json \ --script-inventory-file outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack/script-inventory.json \ --system-name limbu-phone-page-ocr \ --model-name phone-page-yolo-recognizer-filtered-draft \ --language limbu \ --script Limbu \ --domain real_print \ --gt-status draft_machine_ocr_needs_human_review \ --eval-pack-version limbu-phone-page-reference-draft-v1 \ --selected-checkpoint-on-test \ --force ``` Dry-run scorer output: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/page-eval-scorer-pack-dry-run/` Current dry-run metrics: - scored pages: `2/2` - page text codepoint CER: `0.000975` - page text grapheme CER: `0.001976` - text exact pages: `1/2` - line reference/prediction/match counts: `79/78/78` - line detection precision/recall/F1 micro: `1.000000/0.987342/0.993631` - matched-line text CER: `0` - matched-line reading-order pair accuracy: `1.000000` - full reading-order pair accuracy: `0.952381` The sidecar dry run removes the avoidable scorer warnings: - no `eval pack summary/datasheet missing` - no `eval pack summary missing pack_content_sha256` - no `pack content digest mismatch` - no `script inventory file missing` - no `missing_source_document` - no `missing_bbox` - no `page output audits are not all passing` - no `document source_path mismatches detected` The remaining warnings are honest blockers, not metadata bugs: - checkpoint/config/dictionary artifacts were not supplied as readable local files; - eval pack is not frozen and not unseen for selection; - selected checkpoint on this draft pack is declared; - eval pack and rows are not claim eligible; - references are not claim-ready: `0/2`; - sample size is only `2`, below the `30` page claim threshold; - GT status is `draft_machine_ocr_needs_human_review`; - no layout detection references are present; - no train/eval leakage manifest was supplied. One metric caveat matters: the filtered OCR output has `78` lines, while the draft references still have `79` lines. The unmatched reference line is the page marker removed by the text/non-text filter. That is useful signal, not a failure to hide; review/promotion must either mark that reference line as structural/non-text or remove it from the claim-facing reference. The raw-wrapper run now has an explicit audit gate comparing it against the manually prepared direct-checkpoint baseline. The audit reads the wrapper summary, resolves `materializer_run`, reads each `document.json` using the real `pages[].text_lines[]` structure, checks detector and reading-order invariants, and blocks when the per-page line inventory differs from baseline. Raw-vs-baseline inventory audit command: ```bash uv run python tools/audit_limbu_phone_page_ocr_run.py \ outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/limbu-phone-page-ocr-run.json \ --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/phone-page-ocr-audit \ --baseline-materializer-run outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-run.json \ --baseline-page-map phone-pxl-raw-1=phone-pxl-ocr-1 \ --baseline-page-map phone-pxl-raw-2=phone-pxl-ocr-2 \ --required-detector yolo_line_detector_model_v1 \ --expected-reading-order phone-pxl-raw-1=top_to_bottom \ --expected-reading-order phone-pxl-raw-2=wide_column_left_to_right \ --max-baseline-line-count-delta 0 \ --exit-zero ``` Audit artifact: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/phone-page-ocr-audit/phone-page-ocr-audit.md` Current raw-vs-baseline audit result: failed with one blocker. - `phone-pxl-raw-2`: candidate `58` lines, baseline `57`, delta `+1` The audit now also records physical row-band diagnostics from the actual `document.json` line boxes. This matters because a line-count delta can mean either a new physical row or a different decomposition of the same row into column boxes. - `phone-pxl-raw-1`: candidate `21` lines, baseline `21`, candidate `21` row bands, baseline `20` row bands. The difference is the rescued tiny top marker being separated rather than grouped with the title/header band. - `phone-pxl-raw-2`: candidate `58` lines, baseline `57`, but both have `33` physical row bands. The mismatch is a box-composition issue inside the same row structure, not an extra physical text row. The audit report now names the row-band composition disagreements rather than only reporting the bare `+1` count: - `phone-pxl-raw-1`: band `1` differs because the raw run separates the narrow top marker/header behavior differently than the manually prepared baseline. - `phone-pxl-raw-2`: band `18` is candidate `1` box vs baseline `2`; band `27` is candidate `2` boxes vs baseline `1`; band `31` is candidate `2` boxes vs baseline `1`. Visual diagnostics for the lower page-2 disagreement are: - `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/diagnostics/candidate-page2-lower-lines.png` - `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/diagnostics/page2-suspicious-crops-candidate-vs-baseline.png` There is also a reviewer-facing row-composition packet: ```bash uv run python tools/prepare_limbu_phone_row_composition_review.py \ --audit-json outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/phone-page-ocr-audit/phone-page-ocr-audit.json \ --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/row-composition-review ``` Packet artifact: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/row-composition-review/phone-row-composition-review.html` The packet has `4` pending rows and copied crop images for every candidate and baseline side; an HTML link check found `12` images, `12` links, and `0` missing targets. Those diagnostics show the disagreement is over real printed lower-page text, not keyboard/table background or illustration content. Therefore do not solve this by blindly suppressing the extra candidate box. The stricter line-count gate is useful as a warning against unnoticed drift, but the manually prepared baseline is not a human gold reference. The next benchmark needs a reviewed phone-page reference that can decide whether these lower rows should be merged or split in final reading order. The current diagnosis is: - `phone-pxl-raw-1` no longer has a title-order problem and no longer has a line-count problem. The first line is the rescued narrow marker, but OCR on that tiny crop is low quality. - `phone-pxl-raw-2` now uses the right two-column reading order, but it still keeps one extra line relative to the manually prepared direct-checkpoint baseline. The new row-band and visual evidence shows this is a lower-page row-composition disagreement on real text, not a reading-order change and not a text/non-text false positive. Replication note: `--max-baseline-row-band-count-delta` exists as an optional audit gate, but it is intentionally not part of the default strict command above. The row-band comparison is diagnostic unless a caller explicitly chooses to block on it, because page 1 shows that row-band grouping can differ even when the line inventory is correct. This gate is intentionally strict, but it is still a pseudo-baseline gate. For these same phone photos, raw rectification and filtering must preserve physical row inventory and keep row-composition disagreements explicit before we can trust page-level OCR metrics or move on to claiming full book OCR quality. Exact box-count equality against the manually prepared page is not a substitute for a reviewed phone-page reference. ## Phone Benchmark Reference Draft `tools/prepare_limbu_phone_page_benchmark_reference_draft.py` turns the current raw phone OCR artifact into a reviewable reference scaffold. It deliberately does not create gold data. It copies the raw and prepared page images, writes one `.ref.json` file per page using the current OCR text and line boxes, writes an eval manifest that points at those draft references, and writes a browser HTML page with line-box overlays plus a textarea view of the draft text. Replication command: ```bash uv run python tools/prepare_limbu_phone_page_benchmark_reference_draft.py \ --run-json outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/limbu-phone-page-ocr-run.json \ --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft ``` Reference-draft output: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/` Key files: - `limbu-phone-page-reference-draft.json` - `eval-manifest.jsonl` - `limbu-phone-page-reference-draft.html` - `references/phone-pxl-raw-1.ref.json` - `references/phone-pxl-raw-2.ref.json` - `images/phone-pxl-raw-1-prepared.png` - `images/phone-pxl-raw-1-raw.jpg` - `images/phone-pxl-raw-2-prepared.png` - `images/phone-pxl-raw-2-raw.jpg` Generated summary: - samples: `2` - draft lines: `79` - `phone-pxl-raw-1`: `21` draft reference lines - `phone-pxl-raw-2`: `58` draft reference lines - claim boundary: `machine OCR draft references only; not a gold benchmark until reviewed` Every draft reference sets: - `metadata.reference_status`: `draft_machine_ocr_needs_human_review` - `metadata.claim_evidence_eligible`: `false` - `metadata.text_ground_truth_certified`: `false` - `metadata.requires_human_review`: `true` This is required for replication: downstream scripts can use the manifest and reference paths, but no metric from this draft is claim evidence. A human review must correct text, decide the row-composition disagreements, and then change the reference metadata to verified/claim-eligible before page CER or reading-order claims are allowed. Structural reference audit command: ```bash uv run ocrtech audit-references \ outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \ --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/reference-audit ``` Audit result: - passed: `true` - samples: `2` - parsed references: `2` - text references: `2` - reading-order references: `2` - claim-ready references: `0` - issues: `0` - warnings: `0` Path verification after generation found all copied raw images, prepared images, and reference JSON files present. The HTML review page had `4` local links and `0` missing targets. After the draft is corrected and marked claim-ready, score the saved page OCR documents with `tools/run_page_ocr_eval.py` using `reference-draft/eval-manifest.jsonl` as `--eval-manifest` and the raw-wrapper `documents/` directory as `--documents-dir`. For a claim-grade run, first require: - `uv run ocrtech audit-references ... --require-claim-ready` - a passing page-output audit for every scored page output directory - `--require-claim-safe` in `tools/run_page_ocr_eval.py` The central scorer now has its own reference claim-readiness gate. It records `metrics.reference_claim_readiness` with: - `ready_count` - `scored_pages` - `status_counts` - `claim_eligible_counts` - `not_ready_count` - `not_ready_sample_ids` Under `--require-claim-safe`, `tools/run_page_ocr_eval.py` now rejects any page whose loaded reference file is not both `reference_status=verified` and `claim_evidence_eligible=true`. This protects the harness from accidentally scoring `reference-draft/` as if it were a phone-photo benchmark. Current fail-closed check against the active draft references and filtered phone documents: ```text failed_as_expected=true contains_reference_gate=true references are not all claim-ready: 0/2 have reference_status=verified and claim_evidence_eligible=true ``` This check is intentionally not a benchmark result. It proves that the current draft cannot be promoted by mistake. A real phone-page benchmark still needs `reference-verified/eval-manifest.jsonl` from the apply step below, plus a claim-safe page eval run that writes `page-eval-run.json`, `metrics.json`, `metrics.md`, `claim-safety.md`, `page-metrics.csv`, `line-alignment.csv`, and the review artifacts. ## Phone Reference Review And Promotion Gate `tools/prepare_limbu_phone_page_reference_review.py` converts the draft references into a line-level review packet. This is the missing bridge between machine OCR output and a real phone-photo benchmark. It copies every line crop into the packet, writes a CSV that can be edited/exported from the browser, and writes an HTML review surface with per-line crop, OCR draft text, review text, review status, reviewer, and notes. Replication command: ```bash uv run python tools/prepare_limbu_phone_page_reference_review.py \ --eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \ --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-review ``` Review packet output: `outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-review/` Key files: - `phone-page-reference-review.json` - `phone-page-reference-review.csv` - `phone-page-reference-review.jsonl` - `phone-page-reference-review.html` - `crops/phone-pxl-raw-1/*.png` - `crops/phone-pxl-raw-2/*.png` Generated summary: - samples: `2` - review rows: `79` - copied crop images: `79` - missing crop images: `0` - initial status counts: `pending: 79` - HTML local image references: `79` - missing HTML targets: `0` The review status contract is: - `accept`: keep OCR text after human verification. - `corrected`: keep `review_text`; `review_text` must be non-empty. - `drop`: omit a non-text/noise/duplicate line from the verified reference. - `pending`, `needs_context`, `bad_segmentation`, and `needs_resegmentation` block promotion. Every accepted, corrected, or dropped row must carry a reviewer. The HTML export keeps the same CSV schema, so a reviewed CSV can be applied without hand conversion. `tools/apply_limbu_phone_page_reference_review.py` is the only path that should promote this packet into claim-ready references. It refuses to write verified references unless: - every draft line has exactly one review row; - every row has a final non-blocking status; - kept rows have non-empty text; - kept rows satisfy any requested target codepoint range, for the current Sirijonga pages use `--required-codepoint-range U+1900..U+194F`; - a reviewer is present, either per row or through `--reviewer`; - `--tables-reviewed`, `--figures-reviewed`, and `--captions-reviewed` are explicitly supplied. Apply command after human review: ```bash uv run python tools/apply_limbu_phone_page_reference_review.py \ --packet-dir outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-review \ --out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-verified \ --reviewer \ --required-codepoint-range U+1900..U+194F \ --tables-reviewed \ --figures-reviewed \ --captions-reviewed \ --dataset limbu-phone-page-verified \ --split eval ``` The same apply command was intentionally run against the untouched packet and failed closed because all rows are still pending: ```text review_status pending blocks verification ``` No verified phone benchmark exists yet. The output directory `reference-verified/` is reserved for the future completed review result. Focused verification for this work: ```bash uv run pytest \ tests/test_limbu_phone_page_reference_review.py \ tests/test_prepare_limbu_phone_page_benchmark_reference_draft.py \ tests/test_prepare_page_ocr_eval_pack_metadata.py \ tests/test_prepare_limbu_phone_row_composition_review.py \ tests/test_audit_limbu_phone_page_ocr_run.py \ tests/test_audit_page_ocr_documents.py \ tests/test_run_page_ocr_eval.py \ tests/test_materialize_page_image_line_boxes.py \ tests/test_filter_page_ocr_documents.py \ tests/test_run_limbu_phone_page_ocr.py \ tests/test_prepare_limbu_book_captures.py \ tests/test_capture.py \ tests/test_audit_yolo_line_detector_predictions.py \ tests/test_export_limbu_line_detector_dataset.py \ tests/test_preflight_limbu_line_detector_training.py \ tests/test_build_limbu_book_page_manifest.py \ tests/test_build_limbu_pdf_text_context.py \ -q ``` Result: `139 passed`, with only the existing SWIG deprecation warnings from PDF rendering imports. ## What Is Still Missing The missing part is not mainly recognition. It is page understanding. Required next work: - Harden the raw-phone-photo path so automatic rectification and detector filtering preserve physical text rows and make row-composition decisions auditable. The current wrapper exists, the corrected auto-order run fixes the title ordering, and the header rescue fixes page 1 line inventory. Page 2 still has a lower-page row-composition disagreement relative to the manually prepared direct-checkpoint run. - Package or document a stable combined runtime. The direct checkpoint path runs on the AWS G6 environment, but the local machine currently lacks a combined PaddleOCR plus Ultralytics/Torch runtime. - Replace the current heuristic OCR-line filter with trained/evaluated text vs non-text classification. The replayable filter now handles script, confidence, geometry, and page-marker suppression, but images, headers, keyboard/table artifacts, decorative marks, and borderline marginalia still need scored classifier behavior before this becomes a general book pipeline. - Keep hardening reading-order logic for single-column book pages and two-column newspaper pages. Wide-column LTR/RTL ordering and header preservation exist, but they are not yet scored against gold page order. - Turn the phone-reference draft into a frozen phone-photo benchmark with raw image, rectified page image, reviewed text-line boxes, reviewed reading order, and final Unicode text. - Report page-level metrics separately from recognizer oracle-crop CER: line recall, line precision, reading-order accuracy, page CER, review burden, and failure flags. Until those are done, we can say: the page OCR harness can run raw phone JPEGs through page prep, the trained line detector checkpoint, and the recognizer, but the result is not yet a scored, production-ready "feed in book pages and get OCR" system.