Eroux's picture
Update eval numbers to post-crop Hidden Trespass at validation-selected operating point
ce0e254 verified
|
Raw History Blame Contribute Delete
5.01 kB
---
license: apache-2.0
tags:
- object-detection
- document-layout-analysis
- tibetan
- pp-doclayout
- paddleocr
- tibla
pipeline_tag: object-detection
datasets:
- BDRC/TiBLAD
---
# TiBLA-PP-DocLayout-L
**Permissive (Apache-2.0) alternative in TiBLA (Tibetan Book Layout Analysis)** β€”
a PP-DocLayout-L detector for the page layout of modern Tibetan books (headers,
text area, footers, footnotes). It matches the primary RT-DETR-l checkpoint at an
Apache-2.0 license.
- **Base model / provenance:** [PP-DocLayout-L](https://github.com/PaddlePaddle/PaddleOCR)
(PaddleOCR / PaddleDetection, an RT-DETR-L detector), fine-tuned on the
leak-free **v4** `tam2col` split of TiBLAD.
- **License:** Apache-2.0.
- **Dataset:** [BDRC/TiBLAD](https://huggingface.co/datasets/BDRC/TiBLAD)
- **Paper:** [buda-base/papers](https://github.com/buda-base/papers) (`papers/2026-tibetan-book-layout`) β€” *arXiv link forthcoming*
- **Code:** [github.com/buda-base/tibla](https://github.com/buda-base/tibla)
## Task
A **4-class** detector β€” `header`, `text-area`, `footer`, `footnote` β€” kept as
four classes at training time. Evaluation folds them into a **3-class canonical
scheme**: `header`+`footer` are combined into one `header-footer` class (matched
individually, merged losslessly afterwards), `text-area` is merged to a single
page/column envelope as a post-processing step (two boxes only on genuine
two-column pages), and `footnote` is left as-is. All numbers below are in that
canonical space, on the leak-free TiBLAD **v4** 833-page test set, unified scorer
(pycocotools bbox mAP 0.50:0.05:0.95; F1 by greedy IoUβ‰₯0.5 at the best-mean-F1
operating point).
## Files
- `inference/` β€” the exported PaddlePaddle inference model (`inference.pdiparams`,
`inference.yml`, `inference.json`), re-exported for Paddle 3.0.0.
- `best_model.pdparams` β€” the trainable fine-tuned weights.
- `infer.py` β€” batch inference helper (YOLO-format output).
## Inference
```python
# pip install paddlepaddle==3.0.0 paddlex
from paddlex import create_model
model = create_model(model_name="PP-DocLayout-L", model_dir="inference")
for res in model.predict("page.jpg", threshold=0.61): # recommended global conf
res.print() # boxes with label in {header, text-area, footnote, footer}
res.save_to_img("out/")
```
A ready-made `infer.py` (batch, YOLO-format output) is included in this repo.
Recommended global operating confidence: **0.61** (the validation-selected
best-mean-F1 point).
## Evaluation (TiBLAD v4, 833-page test)
| metric | TiBLA-RTDETR | **TiBLA-PP-DocLayout-L** | TiBLA-RFDETR |
|---|---|---|---|
| license | AGPL-3.0 | **Apache-2.0** | Apache-2.0 |
| base model | RT-DETR-l (Ultralytics) | **PP-DocLayout-L (PaddleOCR, RT-DETR-L)** | RF-DETR-L (Roboflow) |
| mean F1 (canonical 3-class) | 0.952 | **0.955** | 0.921 |
|   header-footer F1 | 0.954 | **0.953** | 0.947 |
|   text-area F1 | 0.999 | **0.998** | 0.996 |
|   footnote F1 | 0.902 | **0.914** | 0.821 |
| mean AP@0.50 | 0.974 | **0.959** | 0.925 |
| mean AP@[0.50:0.95] | 0.786 | **0.781** | 0.667 |
| shared-class mAP@[.50:.95] (DocLayNet-aligned) | 0.650 | **0.641** | 0.604 |
| Hidden Trespass β€” header/footer | 0.009 | **0.004** | 0.021 |
| Hidden Trespass β€” footnote | 0.043 | **0.037** | 0.178 |
| COTe (Trespass) | 0.975 (0.001) | **0.978 (0.000)** | 0.974 (0.002) |
| operating confidence | 0.64 | **0.61** | 0.47 |
*"operating confidence" is the single global best-mean-F1 confidence, selected on
the leak-free validation split and frozen for test (no test-set tuning). COCO AP
rows are threshold-free (all detections above the fixed 0.05 floor).*
**Hidden Trespass** = peripheral (header/footer/footnote) ground-truth **area**
that survives in the *actual OCR body crop* `C = E \ P`, where `E` is the predicted
`text-area` envelope and `P` is the union of the predicted peripheral boxes the
pipeline subtracts; area-based, micro-averaged over the test set. Lower is better
(less peripheral text bled into the OCR region). Formal definition in the
[paper](https://github.com/buda-base/papers).
## Which checkpoint to pick
| checkpoint | license | mean F1 | shared mAP | footnote HT |
|---|---|---|---|---|
| TiBLA-RTDETR (primary) | AGPL-3.0 | 0.952 | 0.650 | 0.043 |
| **TiBLA-PP-DocLayout-L** | Apache-2.0 | 0.955 | 0.641 | 0.037 |
| TiBLA-RFDETR | Apache-2.0 | 0.921 | 0.604 | 0.178 |
RT-DETR-l leads on mAP, shared-class mAP and the 5-seed mean F1 (0.961 Β± 0.009),
but its weights are AGPL-3.0 (Ultralytics). If you need a permissive license,
PP-DocLayout-L is an Apache-2.0 match (statistically on par on F1); RF-DETR is a
lighter PyTorch-native Apache-2.0 option.
## Citation
```bibtex
@misc{tibla2026,
title = {TiBLA: Tibetan Book Layout Analysis},
author = {Buddhist Digital Resource Center (BDRC)},
year = {2026},
howpublished = {\url{https://github.com/buda-base/tibla}},
note = {arXiv link forthcoming}
}
```