jasperdew commited on
Commit
36382d8
·
verified ·
1 Parent(s): f9d08ab

Rewrite card: PDF to Markdown positioning, quick start, licence in plain words

Browse files
Files changed (1) hide show
  1. README.md +26 -16
README.md CHANGED
@@ -15,30 +15,29 @@ tags:
15
  - lightgbm
16
  - rag
17
  - pdf-to-markdown
 
18
  ---
19
 
20
- # PDFluent heading level classifier: heading levels 1 to 3 for PDF documents
21
 
22
- A small LightGBM model that assigns heading level 1, 2 or 3 to heading blocks detected by the PDFluent SDK in PDFs, for developers converting PDFs to structured formats.
23
 
24
- Model name: `heading_level_979`. The weights are `model.json.gz`, a compressed LightGBM tree dump. This is not an ONNX model.
25
 
26
- ## Quick start
27
-
28
- With the PDFluent SDK (Python, SDK 1.3.0 or newer), the model is compiled in and on by default:
29
 
30
  ```bash
31
  pip install pdfluent
32
  ```
33
 
34
  ```python
35
- import pdfluent
36
-
37
- doc = pdfluent.open("report.pdf")
38
- markdown = doc.extract_markdown() # headings come out as #, ## and ###
39
- print(markdown)
40
  ```
41
 
 
 
 
 
42
  To inspect the weights without the SDK:
43
 
44
  ```python
@@ -52,6 +51,13 @@ print(model["feature_names"]) # the features the SDK computes per heading bl
52
  print(model["num_class"], len(model["tree_info"]))
53
  ```
54
 
 
 
 
 
 
 
 
55
  ## Inputs and outputs
56
 
57
  The model takes a feature row per heading block from the PDFluent SDK. Features include font size, position and neighbouring blocks. It outputs a level per heading. It is compiled into the PDFluent SDK since version 1.3.0 and runs by default there. No separate install is needed. The SDK handles heading detection and feature extraction.
@@ -71,11 +77,13 @@ The CC BY attribution list is [attribution.json](attribution.json); [PMC-AUTHORS
71
 
72
  Validated only on a held-out part of the same development data, not on a separate sealed benchmark. Synthetic layouts did not improve performance. Behaviour on unrelated PDFs is not established. Only assigns levels, does not detect headings. Requires the PDFluent SDK's heading detection and feature extraction to be useful.
73
 
74
- ## Licence
 
 
75
 
76
- Weights are offered under AGPL-3.0-only or the PDFluent Commercial Licence. Commercial use without AGPL obligations requires the commercial licence: https://pdfluent.com/license. Copyright 2026 Innovation Trigger B.V.
77
 
78
- Licence texts: [LICENSE](LICENSE) (AGPL-3.0) and [LICENSE-COMMERCIAL](LICENSE-COMMERCIAL).
79
 
80
  ## Citation
81
 
@@ -94,9 +102,11 @@ Licence texts: [LICENSE](LICENSE) (AGPL-3.0) and [LICENSE-COMMERCIAL](LICENSE-CO
94
 
95
  - Website: https://pdfluent.com
96
  - Documentation: https://pdfluent.com/docs
97
- - Licence: https://pdfluent.com/license
98
  - SDK source and releases: https://github.com/pdfluent/pdfluent-sdk
 
99
  - Organisation: https://huggingface.co/PDFluent
100
  - Formula recognition model: https://huggingface.co/PDFluent/formula-recognition-onnx
101
  - Heading level classifier: https://huggingface.co/PDFluent/pdf-heading-level-classifier
102
- - Layout ground-truth sample: https://huggingface.co/datasets/PDFluent/pdf-document-layout-ground-truth
 
 
15
  - lightgbm
16
  - rag
17
  - pdf-to-markdown
18
+ - document-parsing
19
  ---
20
 
21
+ # PDF Heading Level Classifier: heading detection for PDF to Markdown
22
 
23
+ Assigns heading level 1, 2 or 3 to every heading block detected in PDFs. A small LightGBM model for #, ## and ### structure in PDF to Markdown output, compiled into PDFluent SDK 1.3.0 and on by default with no separate install. PDFluent SDK 1.3.0 scores the highest overall on opendataloader-bench of the engines on its public leaderboard*: 0.9168 against 0.907 for opendataloader [hybrid], 0.885 for nutrient and 0.882 for docling.
24
 
25
+ \* Measured with the benchmark's own harness and evaluator on its 200 documents, PDFluent SDK 1.3.0, `quality="auto"` with `keep_furniture=True`, on Linux (CPU only), 29 October 2026: 0.9168 overall against 0.907 for opendataloader [hybrid], 0.885 for nutrient and 0.882 for docling. Competitor figures are taken from the benchmark's public README (read 2 November 2026), not re-run by us (our own re-run of opendataloader [hybrid] on 28 October 2026 gave 0.9068). PDFluent is not on that leaderboard; the results are submitted in a pull request: https://github.com/opendataloader-project/opendataloader-bench/pull/37. Development was optimised for this benchmark among its targets. The scores are measured on the benchmark's own 200 documents. The 19-measure result on our own held-out half and the separate checks are on pdfluent.com/benchmarks. The scores come from a release candidate wheel of 1.3.0 that reported version 1.2.0. Speed is not published. The run keeps page headers and footers, because the benchmark's ground truth counts them as content. The difference is one run without a confidence interval.
26
 
27
+ ## Use it in two lines
 
 
28
 
29
  ```bash
30
  pip install pdfluent
31
  ```
32
 
33
  ```python
34
+ import pdfluent; print(pdfluent.open("report.pdf").extract_markdown()) # headings come out as #, ## and ###
 
 
 
 
35
  ```
36
 
37
+ The model is compiled into SDK 1.3.0 and on by default.
38
+
39
+ ## Inspect the weights without the SDK
40
+
41
  To inspect the weights without the SDK:
42
 
43
  ```python
 
51
  print(model["num_class"], len(model["tree_info"]))
52
  ```
53
 
54
+ ## What you get
55
+
56
+ - Assign heading level 1, 2 or 3 to every heading block the SDK detects
57
+ - Trained on arXiv (CC BY 4.0, CC0), PubMed Central (CC BY) and synthetic pages
58
+ - One feature row per heading block in, class scores for level 1, 2 and 3 out
59
+ - Weights published here for inspection, with the attribution list for every training paper
60
+
61
  ## Inputs and outputs
62
 
63
  The model takes a feature row per heading block from the PDFluent SDK. Features include font size, position and neighbouring blocks. It outputs a level per heading. It is compiled into the PDFluent SDK since version 1.3.0 and runs by default there. No separate install is needed. The SDK handles heading detection and feature extraction.
 
77
 
78
  Validated only on a held-out part of the same development data, not on a separate sealed benchmark. Synthetic layouts did not improve performance. Behaviour on unrelated PDFs is not established. Only assigns levels, does not detect headings. Requires the PDFluent SDK's heading detection and feature extraction to be useful.
79
 
80
+ ## Licence in plain words
81
+
82
+ The heading level classifier is part of the PDFluent SDK, licensed under AGPL-3.0-only or the PDFluent Commercial Licence at your option. The model weights are available under these licences. See https://pdfluent.com/license for terms.
83
 
84
+ ## Try PDFluent
85
 
86
+ Run pip install pdfluent to process your PDFs into structured Markdown. The docs at pdfluent.com/docs show how to use extract_markdown with the classifier.
87
 
88
  ## Citation
89
 
 
102
 
103
  - Website: https://pdfluent.com
104
  - Documentation: https://pdfluent.com/docs
105
+ - SDK on PyPI: https://pypi.org/project/pdfluent/
106
  - SDK source and releases: https://github.com/pdfluent/pdfluent-sdk
107
+ - Licence: https://pdfluent.com/license
108
  - Organisation: https://huggingface.co/PDFluent
109
  - Formula recognition model: https://huggingface.co/PDFluent/formula-recognition-onnx
110
  - Heading level classifier: https://huggingface.co/PDFluent/pdf-heading-level-classifier
111
+ - Layout ground-truth dataset: https://huggingface.co/datasets/PDFluent/pdf-document-layout-ground-truth
112
+ - Collection: https://huggingface.co/collections/PDFluent/pdfluent-document-ai-6ae99736adab8c6d90781ab5