File size: 4,281 Bytes
f954ee9 121733a f954ee9 15781f1 121733a 15781f1 f954ee9 15781f1 121733a 15781f1 f954ee9 15781f1 f954ee9 15781f1 f954ee9 15781f1 f954ee9 15781f1 f954ee9 15781f1 f954ee9 121733a 15781f1 f954ee9 15781f1 f954ee9 15781f1 f954ee9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 | ---
license: mit
library_name: onnx
tags:
- onnx
- pdf
- pdf-repair
- tabular-classification
- scikit-learn
datasets:
- utilvo-com/pdf-corruption-lab
pipeline_tag: tabular-classification
metrics:
- accuracy
model-index:
- name: pdf-damage-classifier
results:
- task:
type: tabular-classification
name: PDF damage type classification
dataset:
type: utilvo-com/pdf-corruption-lab
name: PDF Corruption Lab (synthetic, 25% stratified hold-out)
metrics:
- type: accuracy
value: 1.0
name: Accuracy (synthetic hold-out, 300 files)
---
# pdf-damage-classifier
A tiny (75 KB) ONNX model that looks at the **raw bytes** of a PDF and predicts
what kind of structural damage it has, so a repair tool can choose the right
recovery strategy. It does not need a PDF parser, so it works on files that are
too broken to open.
Trained from scratch (scikit-learn `RandomForestClassifier`, 60 trees, depth 8,
exported with `skl2onnx`) on the synthetic
[`utilvo-com/pdf-corruption-lab`](https://huggingface.co/datasets/utilvo-com/pdf-corruption-lab)
dataset. No third-party models or documents are used.
## Classes
`valid`, `missing_eof`, `invalid_startxref`, `scrambled_xref`,
`bad_root_reference`, `truncated`
| Class | Typical repair strategy |
|---|---|
| `valid` | nothing to do |
| `missing_eof` | append the `%%EOF` marker |
| `invalid_startxref` | locate the `xref` table and rewrite `startxref` |
| `scrambled_xref` | rebuild the xref table by scanning for `N G obj` |
| `bad_root_reference` | find the `/Type /Catalog` object and fix the trailer |
| `truncated` | scan surviving objects and rebuild page tree, xref and trailer |
## Input
A float32 vector of 12 structural features computed by
[`pdf_features.py`](pdf_features.py). The order is defined in `labels.json`.
| Feature | Meaning |
|---|---|
| `has_pdf_header` | file starts with `%PDF-` |
| `eof_in_last_1k` / `ends_with_eof` | `%%EOF` marker near / at the end |
| `startxref_present` / `startxref_numeric` | `startxref` exists and is followed by a number |
| `startxref_points_to_xref` | that number is an offset where `xref` really starts |
| `xref_valid_fraction` | share of xref entries that point to a real `N G obj` |
| `has_trailer`, `root_object_exists` | trailer present; `/Root` target object exists |
| `obj_endobj_gap`, `stream_endstream_gap` | relative imbalance of `obj`/`endobj` and `stream`/`endstream` |
| `log10_size` | log10 of file size in bytes |
## Usage
```bash
pip install -r requirements.txt
python predict.py broken.pdf
# broken.pdf: scrambled_xref (p=0.99)
```
Or directly with `onnxruntime`:
```python
import json, numpy as np, onnxruntime as ort
from pdf_features import extract_features
meta = json.load(open("labels.json"))
sess = ort.InferenceSession("pdf_damage_classifier.onnx")
x = np.array([extract_features(open("broken.pdf", "rb").read())], dtype=np.float32)
label_idx, probs = sess.run(None, {sess.get_inputs()[0].name: x})
print(meta["classes"][int(label_idx[0])])
```
## Evaluation and limitations (please read)
- Accuracy on the 25 % stratified hold-out split (300 synthetic files) is 100 %,
and the 7 reference files in `output/` of the dataset repo are all classified
correctly.
- **This number is not evidence of real-world accuracy.** Damage types are
generated by simple rules and are largely separable by the features, so the
task is easy by construction. Training and test files come from the same
generator.
- Training files are produced by `pdf-lib` with classic xref tables (no object
streams / xref streams) and are small. PDFs from other software, encrypted
files, incremental updates, and real-world corruption (bit flips, partial
overwrites, mixed damage) are **not covered** and will likely be misclassified.
- Use it as a quick triage hint, not as a validator.
## Reproduce
```bash
git clone https://huggingface.co/datasets/utilvo-com/pdf-corruption-lab
python train.py pdf-corruption-lab/training .
```
## Intended use
Triage for PDF repair pipelines, teaching material for ONNX export of
scikit-learn models, and as a baseline to beat with better data.
## Maintainers
Maintained by the Utilvo team.
## License
MIT.
|