msbayindir/rag-chunker-models

rag-chunker models

ONNX models that rag-chunker downloads on first use for local page layout analysis, OCR and table structure recognition. They are unmodified exports of PaddleOCR / PP-StructureV3 models (Apache-2.0); this repository only keeps them at a stable address with their checksums, so the tool keeps working whatever happens to the original mirrors.

rag-chunker verifies the sha256 of every download against the catalogue in its source (src/local/models.ts), so a changed file is rejected.

File	What it is	Size	Licence	Original	sha256
texify2/config.json	Formula recognition (texify2): model configuration	0.0 MB	CC-BY-SA-4.0 (vikp/texify2)	source	7133e06cbdbb942886cd5595b860596d08a8dd49c0e91ae7e14283f18fe8b127
texify2/generation_config.json	Formula recognition (texify2): generation settings	0.0 MB	CC-BY-SA-4.0 (vikp/texify2)	source	0fb8980236d664d2851ed5939b435ecc6fe0e32ee08b83e75cb1743db0cfe10e
texify2/preprocessor_config.json	Formula recognition (texify2): image preprocessing settings	0.0 MB	CC-BY-SA-4.0 (vikp/texify2)	source	b2fc6ff0098ac8e6af6e688bfc28188db883c99a48afa60e8555faeedc0c45e4
texify2/tokenizer.json	Formula recognition (texify2): tokenizer	2.0 MB	CC-BY-SA-4.0 (vikp/texify2)	source	02c318d9cfa95bf323371762b8f838a82709530274d36dba6eca880f0add6cc4
texify2/tokenizer_config.json	Formula recognition (texify2): tokenizer settings	0.0 MB	CC-BY-SA-4.0 (vikp/texify2)	source	fd4d94f8b9dbb7deeb3a3ef084ca0e16c43d45774a180730b7e9cfd6359a074b
texify2/onnx/encoder_model_quantized.onnx	Formula recognition (texify2): image encoder, 8-bit	75.6 MB	CC-BY-SA-4.0 (vikp/texify2)	source	6ef545e9b9c451951077ee8f54282fcc77156380df7c3ce5c14852102c921397
texify2/onnx/decoder_model_merged_quantized.onnx	Formula recognition (texify2): LaTeX decoder, 8-bit	228.8 MB	CC-BY-SA-4.0 (vikp/texify2)	source	c9d9f2b65d8c0ea9ff1b3c3ba54a98c1d93f91c690ac10877a395bb7a8cd407a

Usage

Nothing to do by hand: npx @msbayindir/rag-chunker process paper.pdf --layout --ocr pp-ocr fetches what it needs into ~/.rag-chunker/models (RAG_CHUNKER_MODELS_DIR changes the folder). To use the files directly, load them with onnxruntime; the pre- and post-processing rag-chunker applies (input sizes, normalisation, dictionaries, decoding) is in src/local/layout.ts, src/local/ocr.ts and src/local/table.ts.

Credits and licences

The models were trained and released by the PaddlePaddle / PaddleOCR team under the Apache License 2.0 (see LICENSE and NOTICE). The ONNX exports come from https://huggingface.co/Xenova/texify2/resolve/main and https://huggingface.co/Xenova/texify2/resolve/main/onnx. This mirror adds nothing of its own.