Commit Β·
61cd31a
0
Parent(s):
Release snapshot
Browse files- .gitattributes +35 -0
- README.md +75 -0
.gitattributes
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
|
@@ -0,0 +1,75 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: nutrient-commercial
|
| 4 |
+
pipeline_tag: image-text-to-text
|
| 5 |
+
language: [multilingual]
|
| 6 |
+
tags:
|
| 7 |
+
- page-stream-segmentation
|
| 8 |
+
- document-boundary-detection
|
| 9 |
+
- document-ai
|
| 10 |
+
- document-splitting
|
| 11 |
+
datasets:
|
| 12 |
+
- nutrientdocs/doc-split-benchmark
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# doc-split-v2 Β· _commercial_
|
| 16 |
+
|
| 17 |
+
**One model that splits any document stream β short or long, English or not, forms or prose.** The
|
| 18 |
+
high-accuracy flagship for page-stream segmentation: hand it a merged/scanned PDF and it marks where each
|
| 19 |
+
new document begins. **Weights are commercial** (not downloadable here); this page is a spec + scorecard.
|
| 20 |
+
|
| 21 |
+
It is the document specialist in a field of generalists β cloud VLMs and OpenPSS's own single-slice
|
| 22 |
+
specialists each fall down somewhere; this one does short *and* long streams with one model. For the
|
| 23 |
+
open-weight companion, see [doc-split-v1](https://huggingface.co/nutrientdocs/doc-split-v1).
|
| 24 |
+
|
| 25 |
+
- π― **Try it:** [doc-split-demo](https://huggingface.co/spaces/nutrientdocs/doc-split-demo?model=flagship)
|
| 26 |
+
- π **Leaderboard:** [doc-split-leaderboard](https://huggingface.co/spaces/nutrientdocs/doc-split-leaderboard)
|
| 27 |
+
- π **Benchmark:** [doc-split-benchmark](https://huggingface.co/datasets/nutrientdocs/doc-split-benchmark)
|
| 28 |
+
- π’ **Open weights:** [doc-split-v1](https://huggingface.co/nutrientdocs/doc-split-v1)
|
| 29 |
+
|
| 30 |
+
## Results β boundary F1 (ΞΊ)
|
| 31 |
+
|
| 32 |
+
**This model** vs the open `doc-split-v1`, the strongest cloud VLM, and prior work (bold = best releasable single model).
|
| 33 |
+
|
| 34 |
+
| Cut | **doc-split-v2** | doc-split-v1 (open) | best cloud VLM | OpenPSS specialist |
|
| 35 |
+
|---|---|---|---|---|
|
| 36 |
+
| OpenPSS-short (sparse) | **0.652** (.60) | 0.585 | 0.598 | 0.76 (short-spec) |
|
| 37 |
+
| OpenPSS-long | **0.891** (.86) | 0.859 | 0.244 | 0.83 (long-spec) |
|
| 38 |
+
| our-200 (synthetic) | **0.944** (.79) | 0.936 | 0.942 | β |
|
| 39 |
+
| TABME++ test | **0.943** (.91) | 0.704 | β | β |
|
| 40 |
+
| Tobacco800 test | **0.969** (.93) | 0.820 | β | β |
|
| 41 |
+
| val (real-doc) | **0.917** (.86) | 0.918 | β | β |
|
| 42 |
+
|
| 43 |
+
**One balanced model, not two specialists.** OpenPSS needs a separate short- and long-specialist (each
|
| 44 |
+
craters on the other slice); the flagship does both with one model, and its OpenPSS-**long** (0.891) tops
|
| 45 |
+
even OpenPSS's own long-specialist (0.83) and every cloud VLM (best 0.244). It also dominates the modern
|
| 46 |
+
TABME++/Tobacco800 benchmarks. See the [leaderboard](https://huggingface.co/spaces/nutrientdocs/doc-split-leaderboard)
|
| 47 |
+
for the full field.
|
| 48 |
+
|
| 49 |
+
## Intended use & limits
|
| 50 |
+
|
| 51 |
+
- **Use it for:** segmenting a stream of page images (a merged/scanned PDF) into its constituent documents β
|
| 52 |
+
short or long, English or not, forms or prose. One model handles both sparse and dense boundary regimes;
|
| 53 |
+
the embedded text layer sharpens boundaries when present, and scanned pages fall back to vision.
|
| 54 |
+
- **Limits:** optimized for **document** page streams; confidence is calibrated on our-domain data, so very
|
| 55 |
+
out-of-distribution scans are approximate. Boundaries only (not document *type*).
|
| 56 |
+
|
| 57 |
+
## License & data
|
| 58 |
+
|
| 59 |
+
The model **weights** are offered under a commercial Nutrient license β deployed on-prem, so your documents
|
| 60 |
+
never leave your infrastructure. The training set is not redistributed; evaluation runs on the held-out
|
| 61 |
+
[doc-split-benchmark](https://huggingface.co/datasets/nutrientdocs/doc-split-benchmark).
|
| 62 |
+
|
| 63 |
+
> ### π© Get access
|
| 64 |
+
>
|
| 65 |
+
> `doc-split-v2` is commercial and its weights are not downloadable here. To run it on-prem β
|
| 66 |
+
> one model for any stream, calibrated, private β **contact Nutrient:
|
| 67 |
+
> [nutrient.io/contact-sales](https://www.nutrient.io/contact-sales/).**
|
| 68 |
+
|
| 69 |
+
## About the author
|
| 70 |
+
|
| 71 |
+
<a href="https://nutrient.io/">
|
| 72 |
+
<img src="https://avatars2.githubusercontent.com/u/1527679?v=3&s=200" height="80" />
|
| 73 |
+
</a>
|
| 74 |
+
|
| 75 |
+
This project is maintained and funded by [Nutrient](https://nutrient.io/) - The deterministic document infrastructure enterprises run their highest-stakes workflows on: replayable output, clear exceptions, and full audit trails on the messy, regulated documents where AI alone breaks.
|