MiniCPM-OCR-1.0 as single CMF files for cortiq (Rust, no Python)
#12
by infosave - opened
We packaged MiniCPM-OCR-1.0 as single CMF files that run with cortiq, a Rust inference engine with no Python: the SigLIP encoder with the slicing processor and the language model, on Vulkan/DX12/Metal or CPU. cortiq serve gives an OpenAI-compatible endpoint with image parts. These files cover the recognition step; layout detection and reading order stay in your SDK.
Files: https://huggingface.co/infosave/MiniCPM-OCR-1.0-cmf — q8_2f (1.7 GB, 300 greedy tokens of a page identical to Transformers float32) and q4tp (1.4 GB).
On one A4 page at 144 dpi an RTX PRO 4500 decodes at 254 and 267 tok/s, with 2 s from the image to the first token. An Apple M4 (10-core GPU, Metal) decodes at 73 and 102 tok/s, about 8 s to the first token.
cargo install cortiq-cli
cortiq run minicpm-ocr-1.0-q8_2f.cmf --image page.png --prompt "Text Recognition:" --max-tokens 2048
(cortiq 0.8.54)