| --- |
| license: other |
| license_name: apache-2.0-and-cc-by-nc-sa-4.0 |
| license_link: LICENSE |
| base_model: |
| - numind/NuExtract3 |
| tags: |
| - nnef |
| - tract |
| - document-understanding |
| - vision-language |
| - layout-analysis |
| library_name: tract |
| --- |
| |
| # einkpub — model weights for `einkpdf` |
|
|
| NNEF exports of the two models [`einkpdf`](https://github.com/epi/einkpdf) uses to |
| turn research PDFs into EPUBs sized for a 7" e-ink reader. Both run under |
| [tract](https://github.com/sonos/tract) on CPU — no GPU, and no Python at |
| inference time. |
|
|
| A page is rasterized, the **layout** model finds the labelled regions and their |
| reading order, and the **transcriber** turns the page into structured markdown. |
| The layout half exists because the transcriber emits no coordinates at all, so |
| there would otherwise be nothing to crop a figure from. |
|
|
| ## ⚠️ This repository holds two models under two different licences |
|
|
| HuggingFace has one licence field per repository, so this one is tagged `other`. |
| **The licence depends on which files you use.** |
|
|
| | Path | Model | Licence | |
| |---|---|---| |
| | `/` | NuExtract3 transcriber | **Apache-2.0** — see `LICENSE`, `NOTICE` | |
| | `/layout` | surya layout predictor | **CC-BY-NC-SA-4.0** — see `layout/LICENSE`, `layout/NOTICE` | |
|
|
| If you take only the transcriber, Apache-2.0 governs and there is no |
| non-commercial restriction. If you take anything under `layout/`, the |
| CC-BY-NC-SA-4.0 terms apply to those files, including the non-commercial and |
| share-alike conditions. |
|
|
| surya's author waives the non-commercial term for organizations under $5M USD in |
| gross revenue over the most recent 12 months **and** under $5M in lifetime |
| VC/angel funding, that are not competitive with the |
| [Datalab API](https://www.datalab.to/). That waiver is theirs to grant, not this |
| repository's to pass on — commercial licensing is available from Datalab. |
|
|
| ## Files |
|
|
| ### Transcriber — Apache-2.0 |
|
|
| | File | Size | Purpose | |
| |---|---|---| |
| | `vision.nnef.tgz` | 1.3 GB | Vision tower | |
| | `decoder.nnef.tgz` | 2.3 GB | Text decoder | |
| | `tokenizer.json` | 20 MB | HuggingFace tokenizer | |
| | `einkpdf.json` | 4 KB | Runtime config consumed by `einkpdf` | |
|
|
| ### Layout — CC-BY-NC-SA-4.0 |
|
|
| | File | Size | Purpose | |
| |---|---|---| |
| | `layout/layout_predictor_encoder.nnef.tgz` | 299 MB | Vision encoder | |
| | `layout/layout_predictor_decoder_prefill.nnef.tgz` | 219 MB | Decoder, prefill pass | |
| | `layout/layout_predictor_decoder_iter.nnef.tgz` | 199 MB | Decoder, iterative pass | |
| | `layout/layout_config.json` | 1 KB | Slicer and preprocessor config | |
| | `layout/model_config.json` | 6 KB | Decoder token config | |
|
|
| ## Changes from the base models |
|
|
| **Transcriber**, from [`numind/NuExtract3`](https://huggingface.co/numind/NuExtract3): |
|
|
| - Converted from safetensors to NNEF for tract. |
| - Weights at f16; the decoder additionally quantized to Q4_0, cutting peak RSS |
| from ~21 GB to ~6 GB and running ~1.76x faster on CPU. |
| - No fine-tuning. |
| |
| **Layout**, from [surya](https://github.com/VikParuchuri/surya) (checkpoint |
| `2025_02_18`): |
| |
| - Converted from PyTorch to NNEF, split into encoder, decoder-prefill and |
| decoder-iterative graphs. |
| - No fine-tuning. |
| |
| Neither conversion changes model architecture, and both are intended to match |
| their originals. The layout preprocessor is checked against golden arrays from |
| surya's Python implementation. |
| |
| ## A note on operator names |
| |
| The layout graphs were exported against tract 0.21 and still name |
| `tract_onnx_is_nan`, which tract 0.23 moved into core as `tract_core_is_nan`. |
| `einkpdf` re-registers the old name at load time rather than requiring a |
| re-export; anything else loading these graphs on tract ≥ 0.23 will need to do the |
| same. |
| |
