File size: 2,197 Bytes
fd4e724 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | # Reproducible OCR benchmarks
The generator creates CC0 synthetic public DE/EN printed pages (100/200/300 DPI),
text-layer/scanned PDFs, blur/contrast/rotation, ruled tables, columns, a receipt,
blank/damaged input and mixed 20-/100-page PDFs. No real personal data or private
source documents are used. DejaVu Sans is a prerequisite, provided in the worker
image. Ground truth is rendered source text; independent human review, genuine
handwriting and annotated layout/table ground truth remain outstanding.
```sh
OCR_MODELS_DIR=/path/to/verified/models OMP_NUM_THREADS=2 OPENBLAS_NUM_THREADS=1 \
python -m ocr.benchmark --runs 3 --fixtures /tmp/ocr-fixtures \
--output /tmp/ocr-report.json
```
The report records revision/runtime source digest, cgroup limits, platform,
engine versions, model-init and warm latency percentiles, CER/WER, per-fixture
failures/flags/geometry bounds, batches/pages/min, CPU time and peak process RSS.
It does not persist documents or OCR text. CPU-only mixed batches use both embedded
text and CPU OCR; they are not proof of CPU/GPU overflow speedup.
For the two-CPU reproduction, build the reviewed image from repository root, with
an optional ephemeral proxy CA mount in a managed environment, then:
```sh
docker run --rm --network none --cpus 2 --memory 2g --read-only \
--tmpfs /tmp:rw,noexec,nosuid,size=512m hybrid-ocr-cpu:REVIEWED_REVISION \
sh -c 'python -m ocr.benchmark --runs 3 --fixtures /tmp/fixtures --output /tmp/report.json > /tmp/summary.json && cat /tmp/report.json'
```
No host ports are published. The benchmark overrides the worker server command;
the image's inherited liveness healthcheck may report unhealthy because no server
is expected in that container. Its state is not an OCR outcome or readiness result.
A separate actual-server smoke test validates readiness/auth/OCR. Limits do not
make this an actual Hugging Face CPU Basic deployment, nor an exclusive-core test.
See `docs/ocr-benchmarks/` and `docs/OCR_RELEASE_READINESS.md` for retained measurements,
accuracy limitations, exact source revision and production qualification gaps.
Never enable overflow based on these CPU-only measurements or mocked alias timing.
|