| --- |
| license: mit |
| library_name: onnx |
| tags: |
| - onnx |
| - pdf |
| - pdf-repair |
| - tabular-classification |
| - scikit-learn |
| datasets: |
| - utilvo-com/pdf-corruption-lab |
| pipeline_tag: tabular-classification |
| metrics: |
| - accuracy |
| model-index: |
| - name: pdf-damage-classifier |
| results: |
| - task: |
| type: tabular-classification |
| name: PDF damage type classification |
| dataset: |
| type: utilvo-com/pdf-corruption-lab |
| name: PDF Corruption Lab (synthetic, 25% stratified hold-out) |
| metrics: |
| - type: accuracy |
| value: 1.0 |
| name: Accuracy (synthetic hold-out, 300 files) |
| --- |
| |
| # pdf-damage-classifier |
|
|
| A tiny (75 KB) ONNX model that looks at the **raw bytes** of a PDF and predicts |
| what kind of structural damage it has, so a repair tool can choose the right |
| recovery strategy. It does not need a PDF parser, so it works on files that are |
| too broken to open. |
|
|
| Trained from scratch (scikit-learn `RandomForestClassifier`, 60 trees, depth 8, |
| exported with `skl2onnx`) on the synthetic |
| [`utilvo-com/pdf-corruption-lab`](https://huggingface.co/datasets/utilvo-com/pdf-corruption-lab) |
| dataset. No third-party models or documents are used. |
|
|
| ## Classes |
|
|
| `valid`, `missing_eof`, `invalid_startxref`, `scrambled_xref`, |
| `bad_root_reference`, `truncated` |
|
|
| | Class | Typical repair strategy | |
| |---|---| |
| | `valid` | nothing to do | |
| | `missing_eof` | append the `%%EOF` marker | |
| | `invalid_startxref` | locate the `xref` table and rewrite `startxref` | |
| | `scrambled_xref` | rebuild the xref table by scanning for `N G obj` | |
| | `bad_root_reference` | find the `/Type /Catalog` object and fix the trailer | |
| | `truncated` | scan surviving objects and rebuild page tree, xref and trailer | |
|
|
| ## Input |
|
|
| A float32 vector of 12 structural features computed by |
| [`pdf_features.py`](pdf_features.py). The order is defined in `labels.json`. |
|
|
| | Feature | Meaning | |
| |---|---| |
| | `has_pdf_header` | file starts with `%PDF-` | |
| | `eof_in_last_1k` / `ends_with_eof` | `%%EOF` marker near / at the end | |
| | `startxref_present` / `startxref_numeric` | `startxref` exists and is followed by a number | |
| | `startxref_points_to_xref` | that number is an offset where `xref` really starts | |
| | `xref_valid_fraction` | share of xref entries that point to a real `N G obj` | |
| | `has_trailer`, `root_object_exists` | trailer present; `/Root` target object exists | |
| | `obj_endobj_gap`, `stream_endstream_gap` | relative imbalance of `obj`/`endobj` and `stream`/`endstream` | |
| | `log10_size` | log10 of file size in bytes | |
|
|
| ## Usage |
|
|
| ```bash |
| pip install -r requirements.txt |
| python predict.py broken.pdf |
| # broken.pdf: scrambled_xref (p=0.99) |
| ``` |
|
|
| Or directly with `onnxruntime`: |
|
|
| ```python |
| import json, numpy as np, onnxruntime as ort |
| from pdf_features import extract_features |
| |
| meta = json.load(open("labels.json")) |
| sess = ort.InferenceSession("pdf_damage_classifier.onnx") |
| x = np.array([extract_features(open("broken.pdf", "rb").read())], dtype=np.float32) |
| label_idx, probs = sess.run(None, {sess.get_inputs()[0].name: x}) |
| print(meta["classes"][int(label_idx[0])]) |
| ``` |
|
|
| ## Evaluation and limitations (please read) |
|
|
| - Accuracy on the 25 % stratified hold-out split (300 synthetic files) is 100 %, |
| and the 7 reference files in `output/` of the dataset repo are all classified |
| correctly. |
| - **This number is not evidence of real-world accuracy.** Damage types are |
| generated by simple rules and are largely separable by the features, so the |
| task is easy by construction. Training and test files come from the same |
| generator. |
| - Training files are produced by `pdf-lib` with classic xref tables (no object |
| streams / xref streams) and are small. PDFs from other software, encrypted |
| files, incremental updates, and real-world corruption (bit flips, partial |
| overwrites, mixed damage) are **not covered** and will likely be misclassified. |
| - Use it as a quick triage hint, not as a validator. |
|
|
| ## Reproduce |
|
|
| ```bash |
| git clone https://huggingface.co/datasets/utilvo-com/pdf-corruption-lab |
| python train.py pdf-corruption-lab/training . |
| ``` |
|
|
| ## Intended use |
|
|
| Triage for PDF repair pipelines, teaching material for ONNX export of |
| scikit-learn models, and as a baseline to beat with better data. |
|
|
| ## Maintainers |
|
|
| Maintained by the Utilvo team. |
|
|
| ## License |
|
|
| MIT. |
|
|
