Rewrite card: PDF to Markdown positioning, quick start, licence in plain words
Browse files
README.md
CHANGED
|
@@ -15,30 +15,29 @@ tags:
|
|
| 15 |
- lightgbm
|
| 16 |
- rag
|
| 17 |
- pdf-to-markdown
|
|
|
|
| 18 |
---
|
| 19 |
|
| 20 |
-
#
|
| 21 |
|
| 22 |
-
|
| 23 |
|
| 24 |
-
|
| 25 |
|
| 26 |
-
##
|
| 27 |
-
|
| 28 |
-
With the PDFluent SDK (Python, SDK 1.3.0 or newer), the model is compiled in and on by default:
|
| 29 |
|
| 30 |
```bash
|
| 31 |
pip install pdfluent
|
| 32 |
```
|
| 33 |
|
| 34 |
```python
|
| 35 |
-
import pdfluent
|
| 36 |
-
|
| 37 |
-
doc = pdfluent.open("report.pdf")
|
| 38 |
-
markdown = doc.extract_markdown() # headings come out as #, ## and ###
|
| 39 |
-
print(markdown)
|
| 40 |
```
|
| 41 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
To inspect the weights without the SDK:
|
| 43 |
|
| 44 |
```python
|
|
@@ -52,6 +51,13 @@ print(model["feature_names"]) # the features the SDK computes per heading bl
|
|
| 52 |
print(model["num_class"], len(model["tree_info"]))
|
| 53 |
```
|
| 54 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
## Inputs and outputs
|
| 56 |
|
| 57 |
The model takes a feature row per heading block from the PDFluent SDK. Features include font size, position and neighbouring blocks. It outputs a level per heading. It is compiled into the PDFluent SDK since version 1.3.0 and runs by default there. No separate install is needed. The SDK handles heading detection and feature extraction.
|
|
@@ -71,11 +77,13 @@ The CC BY attribution list is [attribution.json](attribution.json); [PMC-AUTHORS
|
|
| 71 |
|
| 72 |
Validated only on a held-out part of the same development data, not on a separate sealed benchmark. Synthetic layouts did not improve performance. Behaviour on unrelated PDFs is not established. Only assigns levels, does not detect headings. Requires the PDFluent SDK's heading detection and feature extraction to be useful.
|
| 73 |
|
| 74 |
-
## Licence
|
|
|
|
|
|
|
| 75 |
|
| 76 |
-
|
| 77 |
|
| 78 |
-
|
| 79 |
|
| 80 |
## Citation
|
| 81 |
|
|
@@ -94,9 +102,11 @@ Licence texts: [LICENSE](LICENSE) (AGPL-3.0) and [LICENSE-COMMERCIAL](LICENSE-CO
|
|
| 94 |
|
| 95 |
- Website: https://pdfluent.com
|
| 96 |
- Documentation: https://pdfluent.com/docs
|
| 97 |
-
-
|
| 98 |
- SDK source and releases: https://github.com/pdfluent/pdfluent-sdk
|
|
|
|
| 99 |
- Organisation: https://huggingface.co/PDFluent
|
| 100 |
- Formula recognition model: https://huggingface.co/PDFluent/formula-recognition-onnx
|
| 101 |
- Heading level classifier: https://huggingface.co/PDFluent/pdf-heading-level-classifier
|
| 102 |
-
- Layout ground-truth
|
|
|
|
|
|
| 15 |
- lightgbm
|
| 16 |
- rag
|
| 17 |
- pdf-to-markdown
|
| 18 |
+
- document-parsing
|
| 19 |
---
|
| 20 |
|
| 21 |
+
# PDF Heading Level Classifier: heading detection for PDF to Markdown
|
| 22 |
|
| 23 |
+
Assigns heading level 1, 2 or 3 to every heading block detected in PDFs. A small LightGBM model for #, ## and ### structure in PDF to Markdown output, compiled into PDFluent SDK 1.3.0 and on by default with no separate install. PDFluent SDK 1.3.0 scores the highest overall on opendataloader-bench of the engines on its public leaderboard*: 0.9168 against 0.907 for opendataloader [hybrid], 0.885 for nutrient and 0.882 for docling.
|
| 24 |
|
| 25 |
+
\* Measured with the benchmark's own harness and evaluator on its 200 documents, PDFluent SDK 1.3.0, `quality="auto"` with `keep_furniture=True`, on Linux (CPU only), 29 October 2026: 0.9168 overall against 0.907 for opendataloader [hybrid], 0.885 for nutrient and 0.882 for docling. Competitor figures are taken from the benchmark's public README (read 2 November 2026), not re-run by us (our own re-run of opendataloader [hybrid] on 28 October 2026 gave 0.9068). PDFluent is not on that leaderboard; the results are submitted in a pull request: https://github.com/opendataloader-project/opendataloader-bench/pull/37. Development was optimised for this benchmark among its targets. The scores are measured on the benchmark's own 200 documents. The 19-measure result on our own held-out half and the separate checks are on pdfluent.com/benchmarks. The scores come from a release candidate wheel of 1.3.0 that reported version 1.2.0. Speed is not published. The run keeps page headers and footers, because the benchmark's ground truth counts them as content. The difference is one run without a confidence interval.
|
| 26 |
|
| 27 |
+
## Use it in two lines
|
|
|
|
|
|
|
| 28 |
|
| 29 |
```bash
|
| 30 |
pip install pdfluent
|
| 31 |
```
|
| 32 |
|
| 33 |
```python
|
| 34 |
+
import pdfluent; print(pdfluent.open("report.pdf").extract_markdown()) # headings come out as #, ## and ###
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
```
|
| 36 |
|
| 37 |
+
The model is compiled into SDK 1.3.0 and on by default.
|
| 38 |
+
|
| 39 |
+
## Inspect the weights without the SDK
|
| 40 |
+
|
| 41 |
To inspect the weights without the SDK:
|
| 42 |
|
| 43 |
```python
|
|
|
|
| 51 |
print(model["num_class"], len(model["tree_info"]))
|
| 52 |
```
|
| 53 |
|
| 54 |
+
## What you get
|
| 55 |
+
|
| 56 |
+
- Assign heading level 1, 2 or 3 to every heading block the SDK detects
|
| 57 |
+
- Trained on arXiv (CC BY 4.0, CC0), PubMed Central (CC BY) and synthetic pages
|
| 58 |
+
- One feature row per heading block in, class scores for level 1, 2 and 3 out
|
| 59 |
+
- Weights published here for inspection, with the attribution list for every training paper
|
| 60 |
+
|
| 61 |
## Inputs and outputs
|
| 62 |
|
| 63 |
The model takes a feature row per heading block from the PDFluent SDK. Features include font size, position and neighbouring blocks. It outputs a level per heading. It is compiled into the PDFluent SDK since version 1.3.0 and runs by default there. No separate install is needed. The SDK handles heading detection and feature extraction.
|
|
|
|
| 77 |
|
| 78 |
Validated only on a held-out part of the same development data, not on a separate sealed benchmark. Synthetic layouts did not improve performance. Behaviour on unrelated PDFs is not established. Only assigns levels, does not detect headings. Requires the PDFluent SDK's heading detection and feature extraction to be useful.
|
| 79 |
|
| 80 |
+
## Licence in plain words
|
| 81 |
+
|
| 82 |
+
The heading level classifier is part of the PDFluent SDK, licensed under AGPL-3.0-only or the PDFluent Commercial Licence at your option. The model weights are available under these licences. See https://pdfluent.com/license for terms.
|
| 83 |
|
| 84 |
+
## Try PDFluent
|
| 85 |
|
| 86 |
+
Run pip install pdfluent to process your PDFs into structured Markdown. The docs at pdfluent.com/docs show how to use extract_markdown with the classifier.
|
| 87 |
|
| 88 |
## Citation
|
| 89 |
|
|
|
|
| 102 |
|
| 103 |
- Website: https://pdfluent.com
|
| 104 |
- Documentation: https://pdfluent.com/docs
|
| 105 |
+
- SDK on PyPI: https://pypi.org/project/pdfluent/
|
| 106 |
- SDK source and releases: https://github.com/pdfluent/pdfluent-sdk
|
| 107 |
+
- Licence: https://pdfluent.com/license
|
| 108 |
- Organisation: https://huggingface.co/PDFluent
|
| 109 |
- Formula recognition model: https://huggingface.co/PDFluent/formula-recognition-onnx
|
| 110 |
- Heading level classifier: https://huggingface.co/PDFluent/pdf-heading-level-classifier
|
| 111 |
+
- Layout ground-truth dataset: https://huggingface.co/datasets/PDFluent/pdf-document-layout-ground-truth
|
| 112 |
+
- Collection: https://huggingface.co/collections/PDFluent/pdfluent-document-ai-6ae99736adab8c6d90781ab5
|