README.md

---
license: cc-by-nc-4.0
library_name: onnx
pipeline_tag: image-classification
tags:
  - onnx
  - ocr
  - document-ai
  - document-layout
  - image-quality
  - image-classification
datasets:
  - allenai/olmOCR-mix-0225
---

# OCR Crop Quality Classifier (ONNX)

> **A newer version is available:** [`Inceptive/document-gate-keeper-v0.5`](https://huggingface.co/Inceptive/document-gate-keeper-v0.5),
> released under Apache 2.0. Its preprocessing differs from this model's, so follow its own model
> card rather than the code below.

A lightweight binary classifier that tells whether a **document region crop** is legible enough to
be sent to an OCR model. It labels each crop `good` or `bad`.

It is meant to sit between a layout detector and a (vision-language) OCR model. Crops are produced
by a detector such as [`Inceptive/PP-DocLayoutV3-ONNX`](https://huggingface.co/Inceptive/PP-DocLayoutV3-ONNX),
and this model flags the ones that are too blurry, pixelated or compression-damaged. The pipeline can
then skip those regions instead of running expensive inference on them, since unreadable regions
are a common source of hallucinated OCR output.

| | |
|---|---|
| **Task** | Binary image classification (`bad` / `good`) |
| **Format** | ONNX, opset 17, single file (no external-data sidecar) |
| **Size** | 1.5 MB, 390,370 parameters |
| **Input** | `input_IMAGE`, `float32[N, 3, 640, 640]`, NCHW, RGB, normalized |
| **Output** | `output_0`, `float32[N, 2]`, softmax probabilities, order `[bad, good]` |
| **Runtime** | Any ONNX Runtime execution provider (CPU is fast enough for per-crop use) |

## Intended use

- **In scope:** crops of printed or scanned document pages (text blocks, titles, tables, form
  fields, figures with text), as produced by a document layout detector. Use it to decide whether a
  region should be OCR'd or skipped / flagged for review.
- **Out of scope:** natural-scene photos, general-purpose image quality assessment (IQA) or
  aesthetic scoring, and full-resolution pages when you need a per-region decision. Also out of
  scope: any decision about a document's *content*. The model only judges visual legibility.

## Model inputs and outputs

### Input: `input_IMAGE`

`float32` tensor of shape `[N, 3, 640, 640]`, with a dynamic batch dimension `N`. The image must be
preprocessed **exactly** as during training:

1. **Convert to RGB.** If you load images with OpenCV, convert BGR → RGB first.
2. **Downscale only if needed.** If the longest side is larger than 640 px, resize so that the
   longest side is exactly 640 px, **keeping the aspect ratio**. Images that already fit are
   **never upscaled**.
3. **Center and zero-pad** the image onto a 640 × 640 canvas, filling every channel with `0`
   (black). Padding happens in pixel space, *before* normalization.
4. **Scale** pixel values to `[0, 1]` (divide by 255).
5. **Normalize** with the ImageNet statistics: `mean = [0.485, 0.456, 0.406]`,
   `std = [0.229, 0.224, 0.225]`, per channel.
6. **Transpose** HWC → CHW and add the batch dimension.

> Do not stretch crops to 640 × 640. A plain resize distorts the aspect ratio and upsamples small
> crops, which the model never saw during training, and shifts its scores.

### Output: `output_0`

`float32` tensor of shape `[N, 2]`, holding class probabilities in the order `["bad", "good"]`. The
graph already ends with a `Softmax`, so do **not** apply another one.

The decision rule we use is `P(good) < 0.5 → low quality`. Raise the threshold to skip more regions
(fewer hallucinations, more missed text), or lower it to keep more.

## How to use

Install the dependencies:

```bash
pip install onnxruntime numpy pillow huggingface_hub
```

Classify one crop:

```python
import json

import numpy as np
import onnxruntime as ort
from huggingface_hub import hf_hub_download
from PIL import Image

REPO_ID = "Inceptive/document-gate-keeper-v0"

# config.json holds the preprocessing contract (it mirrors the ONNX metadata_props).
with open(hf_hub_download(repo_id=REPO_ID, filename="config.json")) as f:
    config = json.load(f)

SIZE = config["input_size"][0]
MEAN = np.array(config["normalization_mean"], dtype=np.float32)
STD = np.array(config["normalization_std"], dtype=np.float32)
CLASSES = tuple(config["classes"])


def preprocess(image: Image.Image) -> np.ndarray:
    """PIL image -> float32 [1, 3, 640, 640] tensor, as used during training."""
    image = image.convert("RGB")

    # Downscale (keeping the aspect ratio) only if the image is larger than 640 px; never upscale.
    w, h = image.size
    scale = SIZE / max(w, h)
    if scale < 1.0:
        image = image.resize((max(1, round(w * scale)), max(1, round(h * scale))), Image.BILINEAR)

    # Center the image on a black 640x640 canvas: zero padding on all channels, before normalization.
    canvas = Image.new("RGB", (SIZE, SIZE), (0, 0, 0))
    canvas.paste(image, ((SIZE - image.width) // 2, (SIZE - image.height) // 2))

    x = np.asarray(canvas, dtype=np.float32) / 255.0  # HWC, [0, 1]
    x = (x - MEAN) / STD                              # ImageNet normalization
    return x.transpose(2, 0, 1)[np.newaxis]           # NCHW


model_path = hf_hub_download(repo_id=REPO_ID, filename=config["model_file"])
session = ort.InferenceSession(
    model_path,
    providers=["CUDAExecutionProvider", "CPUExecutionProvider"],  # falls back to CPU
)

image = Image.open("crop.png")
probs = session.run(["output_0"], {"input_IMAGE": preprocess(image)})[0][0]  # already softmaxed

p_bad, p_good = float(probs[0]), float(probs[1])
print(f"P(bad)={p_bad:.3f}  P(good)={p_good:.3f}  ->  {CLASSES[int(probs.argmax())]}")

GOOD_QUALITY_THRESHOLD = config["good_threshold"]
if p_good < GOOD_QUALITY_THRESHOLD:
    print("Low-quality crop: skip OCR or flag it for review.")
```

Classify several crops in one call (the batch dimension is dynamic):

```python
crops = [Image.open(p) for p in ["region_0.png", "region_1.png", "region_2.png"]]
batch = np.concatenate([preprocess(c) for c in crops])           # [N, 3, 640, 640]
probs = session.run(["output_0"], {"input_IMAGE": batch})[0]     # [N, 2]

for path, (p_bad, p_good) in zip(["region_0.png", "region_1.png", "region_2.png"], probs):
    print(path, "good" if p_good >= GOOD_QUALITY_THRESHOLD else "bad", f"(P(good)={p_good:.3f})")
```

Each crop is padded to 640 × 640, so a batch of `N` crops costs `N × 3 × 640 × 640 × 4` bytes of
input (≈ 4.9 MB per crop). Batch in moderate chunks.

**With a layout detector:** run [`Inceptive/PP-DocLayoutV3-ONNX`](https://huggingface.co/Inceptive/PP-DocLayoutV3-ONNX)
on the page, crop each detected box from the page image at its original resolution, and pass each
crop through `preprocess` above. Do not run the classifier on the detector's own resized input.

## Model architecture

A small convolutional network:

| Stage | Layers | Output channels |
|---|---|---|
| Block 1 | Conv 3×3 (stride 1, same padding) → BatchNorm → ReLU → MaxPool 2×2 | 32 |
| Block 2 | Conv 3×3 → BatchNorm → ReLU → MaxPool 2×2 | 64 |
| Block 3 | Conv 3×3 → BatchNorm → ReLU → MaxPool 2×2 | 128 |
| Block 4 | Conv 3×3 → BatchNorm → ReLU → MaxPool 2×2 | 256 |
| Head | Global average pooling → Flatten → Dropout → Linear (256 → 2) → Softmax | 2 |

The preprocessing contract is also embedded in the ONNX file's `metadata_props` (`input_size`,
`input_layout`, `input_scale`, `normalization_mean`, `normalization_std`, `classes`):

```python
import onnx
print({p.key: p.value for p in onnx.load(model_path).metadata_props})
```

## Training data

The training set is synthetic and was built from two public document datasets:

- [**olmOCR**](https://huggingface.co/datasets/allenai/olmOCR-mix-0225) (Allen Institute for AI),
  a large and diverse collection of PDF pages.
- [**FUNSD**](https://guillaumejaume.github.io/FUNSD/), noisy scanned forms.

Construction:

1. **Region cropping.** Every page was run through
   [`Inceptive/PP-DocLayoutV3-ONNX`](https://huggingface.co/Inceptive/PP-DocLayoutV3-ONNX), and each
   detected layout region was cropped. The classifier therefore sees the same kind of input it gets
   at inference time in an OCR pipeline.
2. **Quality degradation.** Each crop was degraded by applying **successive upscales and
   downscales** (resampling artifacts and loss of fine detail) and **JPEG compression** (blocking
   and ringing artifacts).
3. **Labelling.** Original, untouched crops are labelled `good`. Degraded crops are labelled `bad`.


## Training procedure

- **Epochs:** 10 full passes over the generated dataset.
- **Input pipeline:** the preprocessing described above (RGB, downscale-only resize, centered zero
  padding to 640 × 640, ImageNet normalization).

## Limitations and biases

- **Synthetic degradations only.** "Bad" means *resampled and/or JPEG-compressed*. The model was
  not trained on motion blur, defocus, noise, low contrast, skew, show-through, stains or
  handwriting quality, and it may misjudge real-world scans that fail in those ways.
- **Domain.** The source pages come from olmOCR's PDF mix and FUNSD's English forms. Other
  scripts, unusual layouts or very different rendering resolutions are less well covered.
- **Mostly blank or extreme crops.** A crop that is mostly empty margin, or very thin and elongated
  (and so mostly padding after letterboxing), carries little signal and may be scored `bad` even
  when it is clean. Use it on tight layout-detector crops, not on arbitrary rectangles.
- **Small crops are not enlarged.** A tiny crop occupies only a small part of the 640 × 640 canvas.
  That matches training, but the model's confidence on very small regions is lower.
- **Probabilities are not calibrated.** Tune the `P(good)` threshold on your own data if the
  cost of skipping a region differs from the cost of OCR'ing a bad one.

## License

This model is released under **[CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/)**:
non-commercial use only. This follows the terms of its training data:

- **FUNSD** is available for non-commercial, research and educational purposes only.
- **olmOCR-mix** is released by AI2 under ODC-BY 1.0, which requires attribution (see Citation).

## Future work

Future versions will improve the model's performance and generalization, and will be published under a more permissive license, thanks to a change in dataset composition.
## Citation

If you use this model, please also cite the datasets it was trained on:

```bibtex
@misc{poznanski2025olmocr,
  title         = {olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models},
  author        = {Jake Poznanski and Jon Borchardt and Jason Dunkelberger and Regan Huff and Daniel Lin and Aman Rangapur and Christopher Wilhelm and Kyle Lo and Luca Soldaini},
  year          = {2025},
  eprint        = {2502.18443},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL}
}

@inproceedings{jaume2019funsd,
  title     = {FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents},
  author    = {Guillaume Jaume and Hazim Kemal Ekenel and Jean-Philippe Thiran},
  booktitle = {2019 International Conference on Document Analysis and Recognition Workshops (ICDARW)},
  year      = {2019}
}
```