Download ocr-tech/docs/LIMBU_PAGE_OCR_HARNESS.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 68.2 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/LIMBU_PAGE_OCR_HARNESS.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/ocr-tech/docs/LIMBU_PAGE_OCR_HARNESS.md
-
curl -L -o LIMBU_PAGE_OCR_HARNESS.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/ocr-tech/docs/LIMBU_PAGE_OCR_HARNESS.md
Limbu Page OCR Harness
Updated: 2026-06-23
This document tracks the page-level Limbu OCR harness. Do not confuse the recognizer result with full page OCR. The current measured recognizer champion is useful on line crops, but raw phone photos require additional stages: page rectification, text/non-text segmentation, line grouping, reading order, and end-to-end page scoring.
Current State
The June 23 phone-photo smoke run used the two latest PXL_* images from
~/Downloads:
/Users/cdjk/Downloads/PXL_20260623_132115733.jpg/Users/cdjk/Downloads/PXL_20260623_132120681.jpg
For reproducible local runs, those Downloads originals are staged under:
incoming/limbu-phone-pxl-raw-2026-06-23/PXL_20260623_132115733.jpgincoming/limbu-phone-pxl-raw-2026-06-23/PXL_20260623_132120681.jpg
The staged hashes match the Downloads originals and the raw page manifest:
PXL_20260623_132115733.jpg:4167ec57b84731bce3e275224d15cf6589dafdadac110c6c9b51ae9ff36b570dPXL_20260623_132120681.jpg:8df48d79a22521b0be536d3f4d87d6a1678107e164013a082b22f4cd8a99a7ca
Plain UTF-8 OCR extracts from the current best filtered phone run are saved at
examples/limbu-phone-photo/ as ocr1.txt and ocr2.txt. They come from
ocr-line-filter-sirijonga-conf050-margin-markers/*/document.json, not from
the older naive full-frame run. Current counts are 20 lines / 743
characters for ocr1.txt and 58 lines / 2334 characters for ocr2.txt.
The first naive full-frame run is:
outputs/limbu-phone-photo-ocr-test-2026-06-23/
It completed, but it OCR'd background and non-text because the prepared images still contained keyboard/table/illustration regions. Summary:
- pages:
2 - detected lines:
497 - review rows:
388 - operator attention pages:
2 - suspicious layout gaps:
10
The manually cropped baseline run is:
outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/
It uses the current recognizer package:
runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json
The cropped run is better, but still fails the page-OCR production gate:
- pages:
2 - detected lines:
280 - review rows:
191 - operator attention pages:
2 - suspicious layout gaps:
2 phone-pxl-ocr-1:82lines,54review rowsphone-pxl-ocr-2:198lines,183review rows
The current best automatic text dumps are written to:
examples/limbu-phone-photo/ocr1.txtexamples/limbu-phone-photo/ocr2.txt
These are Unicode, but they are not clean OCR output. Treat them as a failure sample for the page harness, not as digitized text.
New Harness Gate
tools/audit_limbu_phone_page_ocr_run.py audits page OCR runs for production
readiness. It checks:
- too many detected lines per page
- high review-row ratio
- high other-script ratio
- suspicious layout gaps
- operator-attention flags
- detector fragmentation from
image-line-crops/manifest.jsonl - materializer-wrapper runs from
tools/run_limbu_phone_page_ocr.py - optional raw-vs-baseline line-inventory deltas against a manually prepared materializer run
Current audit command:
uv run python tools/audit_limbu_phone_page_ocr_run.py \
outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/limbu-book-ocr-run.json \
--out outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/phone-page-ocr-audit \
--exit-zero
Current result:
outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/phone-page-ocr-audit/phone-page-ocr-audit.md
It fails with twelve blockers. The important new signal is geometry, not script: the recognizer keeps many narrow fragments as confident Sirijonga. The audit now reads each page crop manifest and blocks the run when kept crops look like word/glyph fragments instead of full text lines.
Current detector-fragmentation numbers:
phone-pxl-ocr-1:82kept crops, median kept width ratio0.060, narrow kept crop ratio0.500, max same-row fragments9phone-pxl-ocr-2:198kept crops, median kept width ratio0.045, narrow kept crop ratio0.712, max same-row fragments15
The production gate defaults are:
- median kept crop width ratio must be at least
0.080 - no more than
0.350of kept crops may be narrower than0.060of page width - no estimated row may contain more than
4kept detector fragments
These thresholds are not a recognition fix. They are a stop sign: the current morphology detector is splitting page text into fragments, and the recognizer is then hallucinating valid Limbu text on those fragments.
This is consistent with the existing CDC PDF-text detector sweep:
outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/pdf-text-detector-sweep-v1/limbu-pdf-text-detector-sweep.json
The best morphology sweep covers 76 of 81 source PDF-text lines, but emits
335 detected boxes. That is useful as a recall baseline, but it is not a
production page detector because the candidate count is far above the source
line count.
Detector Training Dataset Export
tools/export_limbu_line_detector_dataset.py now exports supervised text-line
detector labels from the page image plus PDF text-layer geometry artifacts. It
writes:
coco-text-line-detection.json- YOLO labels under
labels/train/andlabels/val/ - copied/augmented page images under
images/train/andimages/val/ yolo-data.yamlline-detector-labels.jsonlwith per-line provenanceline-detector-dataset.jsonwith hashes and source-boundary metadata
The exporter supports repeated --page-manifest and repeated
--pdf-text-context-jsonl arguments, so multiple rendered books can be combined
without hand-concatenating files. It also has an explicit --line-script-filter
for limbu, devanagari, limbu-or-devanagari, or any; this matters because
the page OCR goal covers both Sirijanga/Namdhinggo Limbu and Devanagari-written
Limbu.
The text boundary is strict: PDF text-layer Unicode is provenance/review context only. The exported labels are geometry supervision for a text-line detector, not recognizer gold text.
The first 5-page CDC smoke export is:
uv run python tools/export_limbu_line_detector_dataset.py \
--page-manifest outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/raw-page-manifest/book-pages.jsonl \
--pdf-text-context-jsonl outputs/limbu-book-page-manifests-2026-06-23/cdc-maths-p0011-p0015/full-book-ocr-router-v4-pdf-runner-poppler/pdf-text-context/limbu-pdf-text-context-lines.jsonl \
--out outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23 \
--include-phone-augmentations
Current output:
outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/line-detector-dataset.json
Summary:
- pages:
5 - source context rows:
82 - base line boxes:
81 - exported images:
20 - exported annotations:
324 - train images:
16 - val images:
4 - train pages:
cdc-maths-p0011throughcdc-maths-p0014 - val page:
cdc-maths-p0015 - variants:
original,phone-blur-lowlight,phone-rotate-left,phone-rotate-right
This is the first actual training handoff for replacing the morphology detector. It proved the training path, but it is too small for a real detector.
The current larger CDC Grade 1 source pack is:
outputs/limbu-cdc-grade1-detector-sources-2026-06-23/
It renders the three mirrored CDC curriculum PDFs at 200 DPI and maps their
PDF text-layer boxes onto the rendered page images:
- Maths PDF:
111rendered pages,4352text-layer lines,3915lines with Limbu codepoints after legacy conversion - Science PDF:
89rendered pages,1386text-layer lines,1316lines with Limbu codepoints after legacy conversion - Social PDF:
87rendered pages,1316text-layer lines,1310lines with Limbu codepoints after legacy conversion
The combined detector export command is:
uv run python tools/export_limbu_line_detector_dataset.py \
--page-manifest outputs/limbu-cdc-grade1-detector-sources-2026-06-23/maths/book-pages.jsonl \
--page-manifest outputs/limbu-cdc-grade1-detector-sources-2026-06-23/science/book-pages.jsonl \
--page-manifest outputs/limbu-cdc-grade1-detector-sources-2026-06-23/social/book-pages.jsonl \
--pdf-text-context-jsonl outputs/limbu-cdc-grade1-detector-sources-2026-06-23/maths/pdf-text-context/limbu-pdf-text-context-lines.jsonl \
--pdf-text-context-jsonl outputs/limbu-cdc-grade1-detector-sources-2026-06-23/science/pdf-text-context/limbu-pdf-text-context-lines.jsonl \
--pdf-text-context-jsonl outputs/limbu-cdc-grade1-detector-sources-2026-06-23/social/pdf-text-context/limbu-pdf-text-context-lines.jsonl \
--out outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23 \
--include-phone-augmentations \
--split-strategy hash \
--val-page-ratio 0.15
Current combined dataset:
outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23/line-detector-dataset.json
Summary:
- source page manifests:
3 - source context files:
3 - rendered pages:
287 - source context rows:
7054 - base line boxes:
6540 - skipped non-Limbu/script-filter rows:
513 - skipped invalid bboxes:
1 - exported images:
1148 - exported annotations:
26160 - train images:
976 - val images:
172 - split strategy: deterministic
hash - variants:
original,phone-blur-lowlight,phone-rotate-left,phone-rotate-right
The split remains page-level, not augmentation-level, so augmented copies of the same page do not leak across train and validation.
tools/preflight_limbu_line_detector_training.py validates the exported
detector dataset and writes the exact training command. Current preflight:
uv run python tools/preflight_limbu_line_detector_training.py \
outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/line-detector-dataset.json \
--out outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/training-preflight \
--model yolov8n.pt \
--epochs 100 \
--imgsz 1280 \
--batch 4 \
--project runs/limbu-line-detector \
--name cdc-maths-p0011-p0015-v2
Current result:
outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/training-preflight/line-detector-training-preflight.json
The 5-page smoke dataset preflight passes with 0 issues. The larger all-books
dataset preflight also passes with 0 issues:
outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23/training-preflight/line-detector-training-preflight.json
Local runtime still lacks torch/ultralytics, so detector training runs on the
AWS G6 host.
AWS G6 Detector Smoke Training
The first supervised line-detector smoke run was executed on the existing AWS
playaround G6 instance:
- account:
798006441552 - instance:
i-0fa19dff7b233a353 - name:
ocrtech-g6-training-20260615-v4 - type:
g6.xlarge - runtime path:
/opt/ocrtech - GPU: NVIDIA L4
Remote preflight passed with runtime checks enabled:
cd /opt/ocrtech
.venv/bin/python tools/preflight_limbu_line_detector_training.py \
outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/line-detector-dataset.json \
--out outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/aws-training-preflight-v2 \
--require-runtime \
--model yolov8n.pt \
--epochs 100 \
--imgsz 1280 \
--batch 4 \
--project /opt/ocrtech/runs/limbu-line-detector \
--name cdc-maths-p0011-p0015-v2
Local copy of the AWS preflight:
outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23/aws-training-preflight-v2/line-detector-training-preflight.json
The training run completed:
cd /opt/ocrtech/outputs/limbu-line-detector-dataset-cdc-maths-p0011-p0015-2026-06-23
/opt/ocrtech/.venv/bin/yolo detect train \
data=yolo-data.yaml \
model=/opt/ocrtech/yolov8n.pt \
epochs=100 \
imgsz=1280 \
batch=4 \
project=/opt/ocrtech/runs/limbu-line-detector \
name=cdc-maths-p0011-p0015-v2 \
exist_ok=True
Local copy of the detector run:
outputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/
Important artifacts:
outputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/weights/best.ptoutputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/weights/last.ptoutputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/results.csvoutputs/aws-g6-limbu-line-detector-cdc-maths-p0011-p0015-v2/train.log
Best validation row by mAP50 was epoch 94:
- precision:
0.39716 - recall:
0.31944 - mAP50:
0.22702 - mAP50-95:
0.13800
The final validation of best.pt reported precision 0.399, recall 0.319,
mAP50 0.228, and mAP50-95 0.136 over 4 validation images with 72 line
instances. This checkpoint proves the supervised training path works, but it is
not production-quality. The validation split is one CDC page plus augmentations,
so the metric is a smoke signal only.
The checkpoint was also smoke-tested against the two real PXL prepared page images. Local prediction artifacts:
outputs/aws-g6-limbu-line-detector-phone-predict/
Detector counts:
- cropped pages,
conf=0.05:125,245 - cropped pages,
conf=0.25:21,48 - cropped pages,
conf=0.50:0,0 - auto-detected pages,
conf=0.05:128,228 - auto-detected pages,
conf=0.25:25,44 - auto-detected pages,
conf=0.50:0,1
At conf=0.25, the predicted boxes are sometimes line-sized, but they still
fragment rows, miss lower text, and admit illustration/photo regions. This is
not good enough to replace the current page detector. The right conclusion is:
the training path is alive, but the detector needs substantially more
page-level supervision and likely negative/non-text examples before recognizer
integration.
AWS G6 Detector All-Books Training
The larger CDC Grade 1 all-books detector dataset was synced to the same AWS G6 host and passed remote runtime preflight:
outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23/aws-training-preflight/line-detector-training-preflight.json
Remote training command:
cd /opt/ocrtech/outputs/limbu-line-detector-dataset-cdc-grade1-allbooks-2026-06-23
/opt/ocrtech/.venv/bin/yolo detect train \
data=yolo-data.yaml \
model=/opt/ocrtech/yolov8n.pt \
epochs=80 \
imgsz=1280 \
batch=4 \
project=/opt/ocrtech/runs/limbu-line-detector \
name=cdc-grade1-allbooks-v1 \
exist_ok=True
Launch metadata:
- AWS profile:
playaround - instance:
i-0fa19dff7b233a353 - public IP used for artifact pull and phone prediction:
3.219.242.144 - remote run directory:
/opt/ocrtech/runs/limbu-line-detector/cdc-grade1-allbooks-v1 - remote launch log:
/opt/ocrtech/runs/limbu-line-detector-launch/cdc-grade1-allbooks-v1/train.log - remote PID at launch:
1776
The run completed 80 epochs in 0.720 hours. YOLO scanned 976 train images
and 172 validation images with 0 background, corrupt, or missing-label
issues.
Local copy of the completed detector run:
outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/
Important artifacts:
outputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/weights/best.ptoutputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/weights/last.ptoutputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/results.csvoutputs/aws-g6-limbu-line-detector-cdc-grade1-allbooks-v1/train.log
Best validation row by mAP50 was epoch 79:
- precision:
0.93159 - recall:
0.85800 - mAP50:
0.92478 - mAP50-95:
0.72025
The final epoch 80 row reported precision 0.92947, recall 0.85318, mAP50
0.91988, and mAP50-95 0.71604. The final validation of best.pt reported
precision 0.937, recall 0.845, mAP50 0.921, and mAP50-95 0.724 over
172 validation images and 4148 text-line instances.
This checkpoint is no longer a toy detector on the CDC rendered/augmented validation split. It still does not prove page OCR quality, because the validation set is derived from PDF-rendered curriculum pages, not raw phone photos.
Operational note: on 2026-06-23, the local AWS playaround profile had an
expired SSO session. SSH access to the known instance still worked with
~/.ssh/ocrtech-g6-training-20260615.pem, but EC2 API actions such as
describe-instances and stop-instances could not be run until SSO was
refreshed. The remote host also had no AWS credentials or instance role
available to aws sts get-caller-identity, so it could not stop itself through
the AWS CLI either.
All-Books Checkpoint Phone-Photo Smoke
The completed all-books checkpoint was tested on the same two prepared PXL phone page images used by the failed morphology-detector run:
- cropped prepared pages:
outputs/limbu-phone-photo-ocr-test-2026-06-23-cropped/capture-prep/ - auto-detected prepared pages:
outputs/limbu-phone-photo-auto-detect-capture-2026-06-23/
Remote prediction command pattern:
cd /opt/ocrtech
.venv/bin/yolo detect predict \
model=/opt/ocrtech/runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt \
source=/opt/ocrtech/tmp/limbu-phone-detector-input/<cropped-or-auto> \
imgsz=1280 \
conf=<threshold> \
project=/opt/ocrtech/runs/limbu-line-detector-phone-predict-allbooks \
name=<run-name> \
save=True \
save_txt=True \
save_conf=True \
exist_ok=True
Local prediction artifacts:
outputs/aws-g6-limbu-line-detector-phone-predict-allbooks/
Detector counts:
- cropped pages,
conf=0.25:23,81 - cropped pages,
conf=0.35:23,74 - cropped pages,
conf=0.50:23,67 - auto-detected pages,
conf=0.25:24,78 - auto-detected pages,
conf=0.35:22,72 - auto-detected pages,
conf=0.50:22,64
Geometry audit results:
cropped-conf025: failed,104predictions, blockersame_row_fragmentation_too_high:6>4cropped-conf035: passed,97predictionscropped-conf050: passed,90predictionsauto-conf025: failed,102predictions, blockersame_row_fragmentation_too_high:5>4auto-conf035: failed,94predictions, blockersame_row_fragmentation_too_high:5>4auto-conf050: passed,86predictions
The most useful smoke candidate is cropped-conf050: it passes the geometry
gate and visually produces real line-sized boxes instead of the earlier
word/glyph fragments. It still has false positives on non-page/background text
and small decorative fragments. The detector materializer below now handles
direct checkpoint inference, saved-label ingestion, basic filtering, duplicate
suppression, and explicit two-column reading order. This is still not a finished
end-to-end page OCR benchmark because there is no gold page text/CER for these
phone photos and the raw-photo wrapper still has to combine rectification,
detector inference, recognition, scoring, and audit gates in one reproducible
command.
YOLO Detector Page OCR Materialization
tools/materialize_page_image_line_boxes.py now supports using the trained
detector instead of the old morphology detector:
--line-detector yolo-labels--yolo-labels-dir <dir>--line-detector yolo-model--yolo-model-path <best.pt>- confidence, size, area, and page-region filters
- same-row overlap duplicate suppression for YOLO detections
- explicit wide-column reading orders:
wide_column_left_to_right,wide_column_right_to_left, and split-tall variants - header-preserving wide-column ordering, so page titles above body columns stay before the column body text
- per-page
document.json,document.md, anddocument.body.md
The duplicate suppression is deliberately recorded in the run metadata. It drops a nested same-row fragment when the fragment substantially overlaps a kept row. A much wider row wins over a contained fragment even if the fragment has higher detector confidence; if widths are similar, higher confidence wins. The current defaults are:
- enabled:
true - overlap fraction of smaller box:
0.75 - wider-box preference ratio:
1.12 - y-tolerance: automatic when
0.0
Current PXL eval manifest:
outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-2026-06-23/eval-manifest.jsonl
Saved-label materialization command:
PYTHONPATH=src /tmp/ocrtech-paddle311/bin/python tools/materialize_page_image_line_boxes.py \
--eval-manifest outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-2026-06-23/eval-manifest.jsonl \
--model-config runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json \
--out outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents \
--engine candidate \
--recognition-mode line \
--line-detector yolo-labels \
--yolo-labels-dir outputs/aws-g6-limbu-line-detector-phone-predict-allbooks/cropped-conf050/labels \
--yolo-min-confidence 0.50 \
--yolo-min-width-ratio 0.04 \
--yolo-region-min-y-ratio 0.05 \
--reading-order wide_column_left_to_right \
--crop-padding 8 \
--allow-empty-lines
Saved-label output:
outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/
Run summary after same-row overlap suppression:
- samples:
2 - total materialized OCR lines:
78 - empty recognizer outputs:
0 - detector source:
yolo_line_detector_labels_v1 - reading order:
wide_column_left_to_righton both pages - crop manifest rows:
78 phone-pxl-ocr-1:21lines,0suppressed duplicate detector rowsphone-pxl-ocr-2:57lines,7suppressed duplicate detector rows- high-overlap same-row duplicate pairs after suppression:
0on both pages
Key output files:
outputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-run.jsonoutputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-crops.jsonloutputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/phone-pxl-ocr-1/document.mdoutputs/limbu-phone-photo-yolo-line-ocr-cropped-conf050-ltr-2026-06-23/documents/phone-pxl-ocr-2/document.md
Direct checkpoint materialization was run on the AWS G6 box because the local
machine currently has no combined PaddleOCR plus Ultralytics/Torch runtime and
only about 9.6GiB free disk. The G6 runtime has torch 2.12.1+cu130,
ultralytics 8.4.75, paddle 3.3.1, and paddleocr 3.7.0.
Remote manifest:
/opt/ocrtech/outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/eval-manifest.jsonl
Direct checkpoint materialization command:
cd /opt/ocrtech
PYTHONPATH=src .venv/bin/python tools/materialize_page_image_line_boxes.py \
--eval-manifest outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/eval-manifest.jsonl \
--model-config runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json \
--out outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents \
--engine candidate \
--recognition-mode line \
--line-detector yolo-model \
--yolo-model-path runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt \
--yolo-imgsz 1280 \
--yolo-device 0 \
--yolo-min-confidence 0.50 \
--yolo-min-width-ratio 0.04 \
--yolo-region-min-y-ratio 0.05 \
--reading-order wide_column_left_to_right \
--crop-padding 8 \
--allow-empty-lines
Local copy of direct-checkpoint output:
outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/
Direct-checkpoint run summary:
- samples:
2 - total materialized OCR lines:
78 - empty recognizer outputs:
0 - detector source:
yolo_line_detector_model_v1 - detector source pass count:
yolo-model: 78 - reading order:
wide_column_left_to_righton both pages - crop manifest rows:
78 phone-pxl-ocr-1:21lines,0suppressed duplicate detector rowsphone-pxl-ocr-2:57lines,7suppressed duplicate detector rows- high-overlap same-row duplicate pairs after suppression:
0on both pages
Key direct-checkpoint output files:
outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-run.jsonoutputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-crops.jsonloutputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/phone-pxl-ocr-1/document.mdoutputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/phone-pxl-ocr-2/document.md
The direct checkpoint and saved-label paths produce the same line counts and the same page-1 body Markdown. Page 2 is not byte-identical because the saved labels store rounded normalized box coordinates while direct model inference returns full-precision boxes; small crop-boundary differences change some recognizer outputs. The direct-checkpoint output should be treated as the authoritative current page OCR artifact.
This is the first page-level path that is meaningfully better than the toy morphology OCR: the trained detector checkpoint feeds line crops directly, the recognizer returns Unicode for every kept crop, document JSON/Markdown are emitted, title lines stay above wide-column body text, and duplicated detector fragments are audited and suppressed. It is still not enough to claim "feed in Limbu books and get OCR" because the current entrypoint starts from prepared page images rather than raw captures, has only geometry-based text/non-text filtering, and lacks a gold page transcript/CER benchmark for these real phone photos.
YOLO Prediction Geometry Audit
tools/audit_yolo_line_detector_predictions.py audits saved YOLO prediction
labels before the detector is allowed into phone-page OCR. It does not need text
references. It checks:
- total prediction count per page
- median predicted box width ratio
- narrow predicted box ratio
- same-row fragmentation from normalized y-centers
This targets the real phone-photo failure mode: many confident narrow boxes that look like words/glyph fragments, not text lines. The default blockers are:
- more than
120predictions on a page - median box width below
0.080of page width - more than
0.350of boxes narrower than0.060of page width - more than
4boxes in the same estimated row
Current focused test command:
uv run pytest \
tests/test_audit_yolo_line_detector_predictions.py \
tests/test_export_limbu_line_detector_dataset.py \
tests/test_preflight_limbu_line_detector_training.py \
tests/test_build_limbu_book_page_manifest.py \
tests/test_build_limbu_pdf_text_context.py \
-q
Current result: 24 passed.
Capture Fix Added
src/ocrtech/capture.py now has an automatic page-detection fallback. The
previous implementation only accepted a closed four-corner Canny contour. The
fallback detects the dominant bright, low-saturation page-like region and uses a
minimum-area rectangle when the contour path fails.
On the two real PXL images, auto-detection now completes instead of crashing:
/tmp/ocrtech-paddle311/bin/python tools/prepare_limbu_book_captures.py \
--page-manifest outputs/limbu-phone-photo-ocr-test-2026-06-23/raw-page-manifest/book-pages.jsonl \
--out outputs/limbu-phone-photo-auto-detect-capture-2026-06-23 \
--auto-detect-page \
--auto-deskew \
--allow-failed-capture-audit
The run writes:
outputs/limbu-phone-photo-auto-detect-capture-2026-06-23/
Both pages use method=light_region_min_area_rect and record
touches_image_border=true. The capture audit passes but warns that the source
may be clipped or include background. This is deliberate: the harness should
surface bad capture geometry instead of silently producing confident OCR.
Capture Quality Gates
src/ocrtech/capture.py now records laplacian_variance in
image_quality.source and image_quality.prepared. This is the current
sharpness/blur signal. The existing entropy, luminance_stddev, and
edge_stddev metrics remain in place, and the audit can now enforce all of
these as prepared-image thresholds.
audit_limbu_capture() and tools/prepare_limbu_book_captures.py now expose
phone-capture policy gates for:
- minimum prepared width and height;
- minimum prepared entropy;
- minimum prepared luminance standard deviation;
- minimum prepared edge standard deviation;
- minimum prepared Laplacian variance;
- requiring automatic page detection;
- minimum detected page area ratio;
- optionally failing when auto-detected page corners touch the source image border;
- requiring applied auto-deskew;
- maximum absolute applied deskew angle.
For the current PXL smoke pages, border-touch remains a warning rather than a
failure. Both pages touch the source image border, which means the page may be
clipped or include background, but rejecting them would reject the actual hard
phone photos we need to handle. Use --fail-on-auto-detect-page-border-touch
only for a stricter capture-quality batch.
Local quality-gate replay manifest:
outputs/limbu-phone-capture-quality-gates-2026-06-23/raw-page-manifest/book-pages.jsonl
Replay command:
uv run python tools/prepare_limbu_book_captures.py \
--page-manifest outputs/limbu-phone-capture-quality-gates-2026-06-23/raw-page-manifest/book-pages.jsonl \
--out outputs/limbu-phone-capture-quality-gates-2026-06-23/capture-prep \
--auto-detect-page \
--auto-deskew \
--min-prepared-width 2500 \
--min-prepared-height 3500 \
--min-prepared-entropy 5.0 \
--min-prepared-luminance-stddev 20 \
--min-prepared-edge-stddev 10 \
--min-prepared-laplacian-variance 50 \
--require-auto-detect-page \
--min-auto-detect-page-area-ratio 0.50 \
--require-auto-deskew-applied \
--max-abs-auto-deskew-degrees 4.0
Replay artifact:
outputs/limbu-phone-capture-quality-gates-2026-06-23/capture-prep/
Result:
- pages:
2 - failed capture audits:
0 - both pages warn:
auto_detect_page touches image border
Page quality measurements:
phone-pxl-raw-1: prepared size2945x4059, entropy6.126979, luminance stddev57.978733, edge stddev26.103824, Laplacian variance243.456146, page area ratio0.6772529951108047, deskew1.6degrees.phone-pxl-raw-2: prepared size3013x4165, entropy6.104708, luminance stddev46.921525, edge stddev16.186986, Laplacian variance109.91304, page area ratio0.7292922076056986, deskew1.7degrees.
The raw phone wrapper exposes the same policy switches, so a future end-to-end OCR run can fail before recognition when a photo is too small, too flat, too blurry, lacks page detection, lacks deskew evidence, or is more skewed than the allowed limit.
Compatibility note: capture metadata written before laplacian_variance existed
does not fail replay only because that newer metric is absent. The audit emits a
warning such as image_quality.prepared metrics missing newer replay fields: laplacian_variance. If a caller sets --min-prepared-laplacian-variance, then
legacy metadata without that metric fails because the requested sharpness gate
cannot be evaluated.
Raw Phone-Photo Wrapper Run
tools/run_limbu_phone_page_ocr.py is now the reproducible raw-photo entrypoint.
It does three things in one command:
- Runs
tools/prepare_limbu_book_captures.pyfrom a raw page manifest. - Converts the prepared-page manifest into a materializer eval manifest.
- Runs
tools/materialize_page_image_line_boxes.pywith the trained YOLO line detector and the Limbu recognizer package.
The wrapper preserves the handoff files required by other scripts:
- source raw manifest:
outputs/limbu-phone-page-ocr-raw-yolo-model-auto-2026-06-23/raw-page-manifest/book-pages.jsonl - capture prep run:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/capture-prep/book-capture-prep-run.json - prepared page manifest:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/capture-prep/prepared-page-manifest.jsonl - materializer eval manifest:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/materializer-eval-manifest.jsonl - materializer run:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/documents/image-line-box-run.json - final per-page documents:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/documents/phone-pxl-raw-1/document.mdandoutputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/documents/phone-pxl-raw-2/document.md - wrapper summary with hashes:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/limbu-phone-page-ocr-run.json
Replication-critical provenance rule: every script boundary must preserve the
source page fields, not just the prepared image path. The capture-prep script now
copies source fields from the raw page manifest into
capture-prep/prepared-page-manifest.jsonl, the raw phone wrapper copies them
into materializer-eval-manifest.jsonl under metadata, and the materializer
copies that eval-manifest metadata into each final document.json under
metadata.source_manifest_metadata and
pages[].metadata.source_manifest_metadata. The fields currently preserved are:
page_sequence_numberbook_page_numbersource_filenamesource_document_pathsource_document_sha256source_document_page_numbersource_document_page_indexsource_kindrendered_pdf_dpisource_urlpublication_dateeditionpdf_page_numberlanguagescriptsource_metadata
This is required for replication with other scripts: scoring, review packets, and future corpus exports must be able to trace an OCR line back to the exact source document/page without relying on filenames or memory. Do not strip these fields when creating derived manifests. The refreshed provenance run carries the available phone-capture metadata through all three handoff files. It does not yet carry CDC/PDF source-document fields because the raw PXL manifest itself does not identify the source PDF/book page.
Current provenance-handoff verification:
uv run pytest \
tests/test_prepare_limbu_book_captures.py \
tests/test_run_limbu_phone_page_ocr.py \
tests/test_materialize_page_image_line_boxes.py \
-q
Result: 40 passed. The broader focused page-OCR suite, including the central
page scorer, page-output audit tests, and page eval-pack metadata tests, also
passed: 139 passed, with only the existing SWIG deprecation warnings from PDF
rendering imports.
The raw source manifest rows point at the two uploaded PXL JPEGs on the AWS G6 box:
incoming/limbu-phone-pxl-raw-2026-06-23/PXL_20260623_132115733.jpgincoming/limbu-phone-pxl-raw-2026-06-23/PXL_20260623_132120681.jpg
AWS G6 command:
cd /opt/ocrtech
PYTHONPATH=src .venv/bin/python tools/run_limbu_phone_page_ocr.py \
--page-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-auto-2026-06-23/raw-page-manifest/book-pages.jsonl \
--out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23 \
--model-config runs/limbu-real-heavy-gork-cdc-replay-v1-lr1e6-8ep-normedit-g6-l4-ppocrv5-2026-06-23/package-best/model-card.json \
--engine candidate \
--recognition-mode line \
--line-detector yolo-model \
--yolo-model-path runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt \
--yolo-imgsz 1280 \
--yolo-device 0 \
--yolo-min-confidence 0.50 \
--yolo-min-width-ratio 0.04 \
--yolo-header-min-width-ratio 0.015 \
--yolo-header-max-y-ratio 0.30 \
--yolo-header-min-confidence 0.75 \
--yolo-region-min-y-ratio 0.05 \
--reading-order auto_layout_left_to_right \
--crop-padding 8 \
--allow-empty-lines
Wrapper summary:
- pages:
2 - OCR lines:
79 - empty OCR lines:
0 - capture failed audit count:
0 - materializer detector:
yolo_line_detector_model_v1 - detector source pass count:
yolo-model: 79 - requested reading order:
auto_layout_left_to_right - reading-order selection counts:
top_to_bottom: 1,wide_column_left_to_right: 1 - detector checkpoint:
runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt
Capture operations are recorded per page in
capture-prep/book-capture-prep-run.json under pages[].capture_operations.
Both pages used the bright-region fallback, then perspective rectification,
deskew, grayscale, and autocontrast:
phone-pxl-raw-1:light_region_min_area_rect, area ratio0.6772529951108047,touches_image_border=true, rectified size2833x3981, deskew angle1.6, prepared size2945x4059phone-pxl-raw-2:light_region_min_area_rect, area ratio0.7292922076056986,touches_image_border=true, rectified size2893x4079, deskew angle1.7, prepared size3013x4165
Materializer page summary:
phone-pxl-raw-1:21OCR lines,0empty lines,0same-row duplicate suppressions, reading ordertop_to_bottomphone-pxl-raw-2:58OCR lines,0empty lines,4same-row duplicate suppressions, reading orderwide_column_left_to_right
This raw-wrapper run is useful but not yet good enough to call done. The
manually prepared direct-checkpoint run materialized 21 + 57 lines. The first
raw auto-wrapper run materialized 20 + 58 lines because page rectification and
region filtering altered the detector input. The current header-rescue run
materializes 21 + 58: it fixes the missing top marker on page 1, but page 2 is
still one line over the manually prepared baseline.
A previous run forced wide_column_left_to_right, which was wrong for the first
page and placed the title around line 14. The corrected
auto_layout_left_to_right logic now selects top_to_bottom for
phone-pxl-raw-1 and wide_column_left_to_right for phone-pxl-raw-2. The
first raw page starts with the rescued narrow top marker:
ᤍᤁ
ᤈᤠ᤺ᤵᤐᤠ ᤛᤒᤠ
The rescued marker crop is very small and the recognized Unicode text is poor. That is expected for now. The important change is that the detector inventory is no longer dropping the page-1 header/page-marker line. The next real page-understanding bug is page-2 line inventory, not title order.
The current YOLO filter policy has a narrow-header rescue:
- normal minimum detector width is still controlled by
--yolo-min-width-ratio 0.04 - a narrow detection can pass only near the top of the image when all three
header flags allow it:
--yolo-header-min-width-ratio 0.015,--yolo-header-max-y-ratio 0.30, and--yolo-header-min-confidence 0.75 - the rescue still obeys the normal confidence, region, height, and border filters, so bottom fragments do not reopen
OCR Line Filter / Text-Nontext Gate
tools/filter_page_ocr_documents.py is the current non-oracle post-materializer
line filter. It reads existing page-eval-compatible document.json files,
removes lines/blocks using only live OCR attributes, writes filtered
document.json files, records one JSONL decision row per source line, and can
be replay-audited by tools/audit_page_ocr_line_filter.py.
This is not a learned text/non-text classifier yet. It is a reproducible gate for script, confidence, geometry, and structural-role decisions. The filter now supports:
--drop-empty--require-script sirijonga|devanagari|limbu_or_devanagari- confidence thresholds with
--min-confidence - absolute and page-relative width/height thresholds
--drop-structural-roleand--keep-structural-role--infer-margin-page-markers, which derivespage_markerfor tiny centered short boxes near the top or bottom page margin when older documents only recordedstructural_role: text
The materializer also marks the same small centered margin marker as
structural_role=page_marker on new runs. The filter-side inference exists so
older artifacts can be filtered without rerunning YOLO and OCR. This matters on
the local machine because the recognizer model card exists, but the current YOLO
detector weights are not present at
runs/limbu-line-detector/cdc-grade1-allbooks-v1/weights/best.pt; the recorded
full wrapper command is an AWS G6 command.
Current replay command on the two-page PXL artifact:
uv run python tools/filter_page_ocr_documents.py \
--eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/materializer-eval-manifest.jsonl \
--documents-dir outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/documents \
--out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers \
--drop-empty \
--require-script sirijonga \
--min-confidence 0.50 \
--infer-margin-page-markers \
--drop-structural-role page_marker
Audit command:
uv run python tools/audit_page_ocr_line_filter.py \
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers
Filter artifact:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers/
Audit result: passed with issues=0.
Current result:
- source lines:
79 - kept lines:
78 - removed lines:
1 - removal counts:
structural_role_dropped: 1,low_confidence: 1
The removed line is phone-pxl-raw-1/img-line-0001. The crop is visually the
page marker 52, but the recognizer read it as ᤍᤁ with confidence
0.36888888478279114. Its detected box is
[1345.6043701171875, 954.5286254882812, 54.516357421875, 66.0487060546875]
on a 2945x4059 prepared page, so the filter marks it
structural_role=page_marker,
structural_role_reason=centered_margin_short_small_box, and
structural_role_source=filter_inference.
Replication notes for downstream scripts:
- Use the filtered directory as
--documents-dironly when you explicitly want non-text/page-marker suppression. - Use the unfiltered
documents/directory for detector recall, missed-line, and reference-review work, otherwise the review packet can hide detector output that still needs adjudication. ocr-line-filter-run.jsonrecords the exact source manifest, source documents directory, filter parameters, counts, and output hashes.ocr-line-filter-lines.jsonlis the per-line decision table; it is the file to inspect when a later script disagrees with the filtered line count.ocr-line-filter-audit.jsonand.mdare the replay proof. A filtered directory without a passing audit is not claim-ready.
Page Output Audit For Scoring
tools/audit_page_ocr_documents.py audits page-eval-compatible
document.json output directories and writes the per-sample
limbu-output-audit.json files required by
tools/run_page_ocr_eval.py --require-output-audit. This is separate from
ocrtech audit-limbu-output, which expects the older
limbu-output-manifest.json pipeline-output shape and does not directly fit
the page materializer/filter directories.
The page document audit checks:
document.jsonparses as the sharedDocumentschema;document.source_pathis readable;document.source_pathmatches the eval-manifest input by resolved path or by SHA-256, so copied prepared images are allowed only when byte-identical;- page width/height and line bounding boxes are positive and inside the page;
- line IDs are present and unique per page;
- block
line_idspoint at real lines; - expected
ocr_output_kindand detector source pass match policy; - line confidence is present when required;
- line crop paths exist and
crop_sha256matches when crop checking is enabled.
Current command on the active filtered phone documents:
uv run python tools/audit_page_ocr_documents.py \
--eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \
--documents-dir outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers \
--expected-ocr-output-kind live_detector \
--expected-detector-source-pass yolo-model \
--require-line-crops \
--require-line-confidence \
--min-line-count 1
Audit artifacts:
- aggregate:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers/page-ocr-documents-audit.json - per page:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers/phone-pxl-raw-1/limbu-output-audit.json - per page:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers/phone-pxl-raw-2/limbu-output-audit.json
Current result:
- samples:
2 - passed samples:
2 - failed samples:
0 - audited lines:
78 - empty lines:
0 - source binding for both pages:
same_sha256 - crop hashes checked:
20onphone-pxl-raw-1,58onphone-pxl-raw-2 - detector source pass counts:
yolo-model: 78
After this audit and the scorer SHA-256 source-binding fix, the current
claim-safe scorer dry run against reference-draft/eval-manifest.jsonl still
fails, as it should, but it no longer reports either:
page output audits are not all passingdocument source_path mismatches detected
Against the original reference-draft/eval-manifest.jsonl, the remaining
scorer blockers are now the real benchmark blockers plus missing scorer
sidecars: draft references, non-frozen pack metadata, missing eval-pack
datasheet/script inventory/dictionary/leakage evidence, only 2 scoring pages,
and no layout-detection reference coverage. For a reproducible scorer dry run,
first build the scorer sidecars described in the next section and use that
rewritten manifest.
Page Eval Pack Metadata Sidecars
tools/prepare_page_ocr_eval_pack_metadata.py prepares the central scorer
sidecars for an existing page-level OCR eval manifest. It writes a rewritten
manifest with scorer-visible source provenance, a summary.json with the exact
page-pack pack_content_sha256, and a script-inventory.json derived from the
loaded reference text. It does not certify labels: draft references remain
draft, claim_evidence_eligible=false remains false, and --require-claim-safe
still fails until the references are human verified.
This tool exists because the first phone reference draft had good reference
files but did not expose whole-page bbox and source_image fields in the
manifest keys that tools/run_page_ocr_eval.py audits. The phone draft writer
now emits those fields for future runs, but this sidecar tool is the
replication path for existing drafts and other page-manifest builders.
Current sidecar-generation command:
uv run python tools/prepare_page_ocr_eval_pack_metadata.py \
--eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \
--out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack \
--eval-pack-version limbu-phone-page-reference-draft-v1 \
--force
Sidecar output:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack/
Key files:
eval-manifest.jsonlsummary.jsonscript-inventory.jsonpage-ocr-eval-pack-metadata.jsonREADME.md
Current sidecar result:
- samples:
2 - reference lines:
79 - claim-ready references:
0/2 pack_content_sha256:b20abcbbe4c6f38e39be055d9d618d29d0ecdc3751a5eb38d50384d89fabed8e- metadata coverage under the scorer:
missing_source_document=0,missing_page=0,missing_bbox=0,missing_claim_eligibility=0,missing_language=0,missing_script=0,missing_domain=0,missing_gt_status=0 - script inventory coverage under the scorer:
missing_required_codepoints=[],missing_required_graphemes=[]
Scorer dry-run command using the sidecars and the filtered phone OCR documents:
uv run python tools/run_page_ocr_eval.py \
--eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack/eval-manifest.jsonl \
--documents-dir outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/ocr-line-filter-sirijonga-conf050-margin-markers \
--ocr-output-kind live_detector \
--require-output-audit \
--out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/page-eval-scorer-pack-dry-run \
--summary-file outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack/summary.json \
--script-inventory-file outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/scorer-pack/script-inventory.json \
--system-name limbu-phone-page-ocr \
--model-name phone-page-yolo-recognizer-filtered-draft \
--language limbu \
--script Limbu \
--domain real_print \
--gt-status draft_machine_ocr_needs_human_review \
--eval-pack-version limbu-phone-page-reference-draft-v1 \
--selected-checkpoint-on-test \
--force
Dry-run scorer output:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/page-eval-scorer-pack-dry-run/
Current dry-run metrics:
- scored pages:
2/2 - page text codepoint CER:
0.000975 - page text grapheme CER:
0.001976 - text exact pages:
1/2 - line reference/prediction/match counts:
79/78/78 - line detection precision/recall/F1 micro:
1.000000/0.987342/0.993631 - matched-line text CER:
0 - matched-line reading-order pair accuracy:
1.000000 - full reading-order pair accuracy:
0.952381
The sidecar dry run removes the avoidable scorer warnings:
- no
eval pack summary/datasheet missing - no
eval pack summary missing pack_content_sha256 - no
pack content digest mismatch - no
script inventory file missing - no
missing_source_document - no
missing_bbox - no
page output audits are not all passing - no
document source_path mismatches detected
The remaining warnings are honest blockers, not metadata bugs:
- checkpoint/config/dictionary artifacts were not supplied as readable local files;
- eval pack is not frozen and not unseen for selection;
- selected checkpoint on this draft pack is declared;
- eval pack and rows are not claim eligible;
- references are not claim-ready:
0/2; - sample size is only
2, below the30page claim threshold; - GT status is
draft_machine_ocr_needs_human_review; - no layout detection references are present;
- no train/eval leakage manifest was supplied.
One metric caveat matters: the filtered OCR output has 78 lines, while the
draft references still have 79 lines. The unmatched reference line is the page
marker removed by the text/non-text filter. That is useful signal, not a
failure to hide; review/promotion must either mark that reference line as
structural/non-text or remove it from the claim-facing reference.
The raw-wrapper run now has an explicit audit gate comparing it against the
manually prepared direct-checkpoint baseline. The audit reads the wrapper
summary, resolves materializer_run, reads each document.json using the real
pages[].text_lines[] structure, checks detector and reading-order invariants,
and blocks when the per-page line inventory differs from baseline.
Raw-vs-baseline inventory audit command:
uv run python tools/audit_limbu_phone_page_ocr_run.py \
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/limbu-phone-page-ocr-run.json \
--out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/phone-page-ocr-audit \
--baseline-materializer-run outputs/limbu-phone-photo-yolo-model-ocr-cropped-conf050-ltr-2026-06-23/documents/image-line-box-run.json \
--baseline-page-map phone-pxl-raw-1=phone-pxl-ocr-1 \
--baseline-page-map phone-pxl-raw-2=phone-pxl-ocr-2 \
--required-detector yolo_line_detector_model_v1 \
--expected-reading-order phone-pxl-raw-1=top_to_bottom \
--expected-reading-order phone-pxl-raw-2=wide_column_left_to_right \
--max-baseline-line-count-delta 0 \
--exit-zero
Audit artifact:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/phone-page-ocr-audit/phone-page-ocr-audit.md
Current raw-vs-baseline audit result: failed with one blocker.
phone-pxl-raw-2: candidate58lines, baseline57, delta+1
The audit now also records physical row-band diagnostics from the actual
document.json line boxes. This matters because a line-count delta can mean
either a new physical row or a different decomposition of the same row into
column boxes.
phone-pxl-raw-1: candidate21lines, baseline21, candidate21row bands, baseline20row bands. The difference is the rescued tiny top marker being separated rather than grouped with the title/header band.phone-pxl-raw-2: candidate58lines, baseline57, but both have33physical row bands. The mismatch is a box-composition issue inside the same row structure, not an extra physical text row.
The audit report now names the row-band composition disagreements rather than
only reporting the bare +1 count:
phone-pxl-raw-1: band1differs because the raw run separates the narrow top marker/header behavior differently than the manually prepared baseline.phone-pxl-raw-2: band18is candidate1box vs baseline2; band27is candidate2boxes vs baseline1; band31is candidate2boxes vs baseline1.
Visual diagnostics for the lower page-2 disagreement are:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/diagnostics/candidate-page2-lower-lines.pngoutputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/diagnostics/page2-suspicious-crops-candidate-vs-baseline.png
There is also a reviewer-facing row-composition packet:
uv run python tools/prepare_limbu_phone_row_composition_review.py \
--audit-json outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/phone-page-ocr-audit/phone-page-ocr-audit.json \
--out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/row-composition-review
Packet artifact:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/row-composition-review/phone-row-composition-review.html
The packet has 4 pending rows and copied crop images for every candidate and
baseline side; an HTML link check found 12 images, 12 links, and 0 missing
targets.
Those diagnostics show the disagreement is over real printed lower-page text, not keyboard/table background or illustration content. Therefore do not solve this by blindly suppressing the extra candidate box. The stricter line-count gate is useful as a warning against unnoticed drift, but the manually prepared baseline is not a human gold reference. The next benchmark needs a reviewed phone-page reference that can decide whether these lower rows should be merged or split in final reading order.
The current diagnosis is:
phone-pxl-raw-1no longer has a title-order problem and no longer has a line-count problem. The first line is the rescued narrow marker, but OCR on that tiny crop is low quality.phone-pxl-raw-2now uses the right two-column reading order, but it still keeps one extra line relative to the manually prepared direct-checkpoint baseline. The new row-band and visual evidence shows this is a lower-page row-composition disagreement on real text, not a reading-order change and not a text/non-text false positive.
Replication note: --max-baseline-row-band-count-delta exists as an optional
audit gate, but it is intentionally not part of the default strict command
above. The row-band comparison is diagnostic unless a caller explicitly chooses
to block on it, because page 1 shows that row-band grouping can differ even when
the line inventory is correct.
This gate is intentionally strict, but it is still a pseudo-baseline gate. For these same phone photos, raw rectification and filtering must preserve physical row inventory and keep row-composition disagreements explicit before we can trust page-level OCR metrics or move on to claiming full book OCR quality. Exact box-count equality against the manually prepared page is not a substitute for a reviewed phone-page reference.
Phone Benchmark Reference Draft
tools/prepare_limbu_phone_page_benchmark_reference_draft.py turns the current
raw phone OCR artifact into a reviewable reference scaffold. It deliberately
does not create gold data. It copies the raw and prepared page images, writes
one .ref.json file per page using the current OCR text and line boxes, writes
an eval manifest that points at those draft references, and writes a browser
HTML page with line-box overlays plus a textarea view of the draft text.
Replication command:
uv run python tools/prepare_limbu_phone_page_benchmark_reference_draft.py \
--run-json outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/limbu-phone-page-ocr-run.json \
--out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft
Reference-draft output:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/
Key files:
limbu-phone-page-reference-draft.jsoneval-manifest.jsonllimbu-phone-page-reference-draft.htmlreferences/phone-pxl-raw-1.ref.jsonreferences/phone-pxl-raw-2.ref.jsonimages/phone-pxl-raw-1-prepared.pngimages/phone-pxl-raw-1-raw.jpgimages/phone-pxl-raw-2-prepared.pngimages/phone-pxl-raw-2-raw.jpg
Generated summary:
- samples:
2 - draft lines:
79 phone-pxl-raw-1:21draft reference linesphone-pxl-raw-2:58draft reference lines- claim boundary:
machine OCR draft references only; not a gold benchmark until reviewed
Every draft reference sets:
metadata.reference_status:draft_machine_ocr_needs_human_reviewmetadata.claim_evidence_eligible:falsemetadata.text_ground_truth_certified:falsemetadata.requires_human_review:true
This is required for replication: downstream scripts can use the manifest and reference paths, but no metric from this draft is claim evidence. A human review must correct text, decide the row-composition disagreements, and then change the reference metadata to verified/claim-eligible before page CER or reading-order claims are allowed.
Structural reference audit command:
uv run ocrtech audit-references \
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \
--out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/reference-audit
Audit result:
- passed:
true - samples:
2 - parsed references:
2 - text references:
2 - reading-order references:
2 - claim-ready references:
0 - issues:
0 - warnings:
0
Path verification after generation found all copied raw images, prepared images,
and reference JSON files present. The HTML review page had 4 local links and
0 missing targets.
After the draft is corrected and marked claim-ready, score the saved page OCR
documents with tools/run_page_ocr_eval.py using
reference-draft/eval-manifest.jsonl as --eval-manifest and the raw-wrapper
documents/ directory as --documents-dir. For a claim-grade run, first require:
uv run ocrtech audit-references ... --require-claim-ready- a passing page-output audit for every scored page output directory
--require-claim-safeintools/run_page_ocr_eval.py
The central scorer now has its own reference claim-readiness gate. It records
metrics.reference_claim_readiness with:
ready_countscored_pagesstatus_countsclaim_eligible_countsnot_ready_countnot_ready_sample_ids
Under --require-claim-safe, tools/run_page_ocr_eval.py now rejects any page
whose loaded reference file is not both reference_status=verified and
claim_evidence_eligible=true. This protects the harness from accidentally
scoring reference-draft/ as if it were a phone-photo benchmark.
Current fail-closed check against the active draft references and filtered phone documents:
failed_as_expected=true
contains_reference_gate=true
references are not all claim-ready: 0/2 have reference_status=verified and claim_evidence_eligible=true
This check is intentionally not a benchmark result. It proves that the current
draft cannot be promoted by mistake. A real phone-page benchmark still needs
reference-verified/eval-manifest.jsonl from the apply step below, plus a
claim-safe page eval run that writes page-eval-run.json, metrics.json,
metrics.md, claim-safety.md, page-metrics.csv, line-alignment.csv, and
the review artifacts.
Phone Reference Review And Promotion Gate
tools/prepare_limbu_phone_page_reference_review.py converts the draft
references into a line-level review packet. This is the missing bridge between
machine OCR output and a real phone-photo benchmark. It copies every line crop
into the packet, writes a CSV that can be edited/exported from the browser, and
writes an HTML review surface with per-line crop, OCR draft text, review text,
review status, reviewer, and notes.
Replication command:
uv run python tools/prepare_limbu_phone_page_reference_review.py \
--eval-manifest outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-draft/eval-manifest.jsonl \
--out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-review
Review packet output:
outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-review/
Key files:
phone-page-reference-review.jsonphone-page-reference-review.csvphone-page-reference-review.jsonlphone-page-reference-review.htmlcrops/phone-pxl-raw-1/*.pngcrops/phone-pxl-raw-2/*.png
Generated summary:
- samples:
2 - review rows:
79 - copied crop images:
79 - missing crop images:
0 - initial status counts:
pending: 79 - HTML local image references:
79 - missing HTML targets:
0
The review status contract is:
accept: keep OCR text after human verification.corrected: keepreview_text;review_textmust be non-empty.drop: omit a non-text/noise/duplicate line from the verified reference.pending,needs_context,bad_segmentation, andneeds_resegmentationblock promotion.
Every accepted, corrected, or dropped row must carry a reviewer. The HTML export keeps the same CSV schema, so a reviewed CSV can be applied without hand conversion.
tools/apply_limbu_phone_page_reference_review.py is the only path that should
promote this packet into claim-ready references. It refuses to write verified
references unless:
- every draft line has exactly one review row;
- every row has a final non-blocking status;
- kept rows have non-empty text;
- kept rows satisfy any requested target codepoint range, for the current
Sirijonga pages use
--required-codepoint-range U+1900..U+194F; - a reviewer is present, either per row or through
--reviewer; --tables-reviewed,--figures-reviewed, and--captions-reviewedare explicitly supplied.
Apply command after human review:
uv run python tools/apply_limbu_phone_page_reference_review.py \
--packet-dir outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-review \
--out outputs/limbu-phone-page-ocr-raw-yolo-model-autolayout-v2-header-rescue-provenance-2026-06-23/reference-verified \
--reviewer <human-reviewer-id> \
--required-codepoint-range U+1900..U+194F \
--tables-reviewed \
--figures-reviewed \
--captions-reviewed \
--dataset limbu-phone-page-verified \
--split eval
The same apply command was intentionally run against the untouched packet and failed closed because all rows are still pending:
review_status pending blocks verification
No verified phone benchmark exists yet. The output directory
reference-verified/ is reserved for the future completed review result.
Focused verification for this work:
uv run pytest \
tests/test_limbu_phone_page_reference_review.py \
tests/test_prepare_limbu_phone_page_benchmark_reference_draft.py \
tests/test_prepare_page_ocr_eval_pack_metadata.py \
tests/test_prepare_limbu_phone_row_composition_review.py \
tests/test_audit_limbu_phone_page_ocr_run.py \
tests/test_audit_page_ocr_documents.py \
tests/test_run_page_ocr_eval.py \
tests/test_materialize_page_image_line_boxes.py \
tests/test_filter_page_ocr_documents.py \
tests/test_run_limbu_phone_page_ocr.py \
tests/test_prepare_limbu_book_captures.py \
tests/test_capture.py \
tests/test_audit_yolo_line_detector_predictions.py \
tests/test_export_limbu_line_detector_dataset.py \
tests/test_preflight_limbu_line_detector_training.py \
tests/test_build_limbu_book_page_manifest.py \
tests/test_build_limbu_pdf_text_context.py \
-q
Result: 139 passed, with only the existing SWIG deprecation warnings from PDF
rendering imports.
What Is Still Missing
The missing part is not mainly recognition. It is page understanding.
Required next work:
- Harden the raw-phone-photo path so automatic rectification and detector filtering preserve physical text rows and make row-composition decisions auditable. The current wrapper exists, the corrected auto-order run fixes the title ordering, and the header rescue fixes page 1 line inventory. Page 2 still has a lower-page row-composition disagreement relative to the manually prepared direct-checkpoint run.
- Package or document a stable combined runtime. The direct checkpoint path runs on the AWS G6 environment, but the local machine currently lacks a combined PaddleOCR plus Ultralytics/Torch runtime.
- Replace the current heuristic OCR-line filter with trained/evaluated text vs non-text classification. The replayable filter now handles script, confidence, geometry, and page-marker suppression, but images, headers, keyboard/table artifacts, decorative marks, and borderline marginalia still need scored classifier behavior before this becomes a general book pipeline.
- Keep hardening reading-order logic for single-column book pages and two-column newspaper pages. Wide-column LTR/RTL ordering and header preservation exist, but they are not yet scored against gold page order.
- Turn the phone-reference draft into a frozen phone-photo benchmark with raw image, rectified page image, reviewed text-line boxes, reviewed reading order, and final Unicode text.
- Report page-level metrics separately from recognizer oracle-crop CER: line recall, line precision, reading-order accuracy, page CER, review burden, and failure flags.
Until those are done, we can say: the page OCR harness can run raw phone JPEGs through page prep, the trained line detector checkpoint, and the recognizer, but the result is not yet a scored, production-ready "feed in book pages and get OCR" system.