Instructions to use litert-community/MiniCPM-OCR-1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use litert-community/MiniCPM-OCR-1.0 with LiteRT-LM:

# LiteRT-LM runs on various platforms (Android, iOS, Windows, Linux, macOS, IoT, Web/WASM)
# and supports many APIs (C++, Python, Kotlin, Swift, JavaScript, Flutter).
# For platform-specific integration guides, please refer to the official developer website:
# https://ai.google.dev/edge/litert-lm

# To try LiteRT-LM, the easiest way is to use our CLI tool.
# 1. Install the LiteRT-LM CLI tool:
pip install -U litert-lm

# 2. Download and run this model locally:
# See: https://ai.google.dev/edge/litert-lm/cli
# A single .litertlm file in the repo is picked automatically; otherwise the CLI asks which one to run
# (or pass its name right after the repo id).
litert-lm run \
  --from-huggingface-repo=litert-community/MiniCPM-OCR-1.0 \
  --prompt="Write me a poem"

LiteRT

How to use litert-community/MiniCPM-OCR-1.0 with LiteRT:

# No code snippets available yet for this library.

# To use this model, check the repository files and the library's documentation.

# Want to help? PRs adding snippets are welcome at:
# https://github.com/huggingface/huggingface.js

MiniCPM-OCR

How to use litert-community/MiniCPM-OCR-1.0 with MiniCPM-OCR:

pip install "minicpm-ocr[all]" huggingface_hub

# Deploy litert-community/MiniCPM-OCR-1.0 as a recognition service before running the Python example:
# https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/vllm_deploy_en.md
# vLLM: load the model above; use --port 8080,
# --served-model-name minicpm-ocr and --max-model-len 40960.
# Layout detection uses cuda:0; recognition runs on the deployed service.

from importlib.resources import files
from pathlib import Path

from minicpmvocr import load_config, pipelined_run
from huggingface_hub import snapshot_download

layout_model = snapshot_download("PaddlePaddle/PP-DocLayoutV3")
cfg = load_config(
    str(files("minicpmvocr").joinpath("configs/minicpmvocr.yaml")),
    overrides={
        "input": {"file_path": str(Path("demo.png").resolve())},
        "engine": {
            "type": "vllm",
            "server_url": "http://127.0.0.1:8080",
            "model_name": "minicpm-ocr",
            "max_model_len": 40960,
        },
        "layout": {"model_dir": layout_model, "device": "cuda:0"},
        # Render PDFs in-process so this example also works without a main guard.
        "pipeline": {"overlap_layout": True, "render_workers": 1},
    },
)
pipelined_run(cfg)

Notebooks
Google Colab
Kaggle

MiniCPM-OCR-1.0

Commit History

Nov 9 4 days ago

12:09
litertlm_manifest.json: MiniCPM-OCR-1.0-VL_int8vis.litertlm's sha256, size and sections after the move to external weights; min runtime 0.19.0 and the Android GPU known issue (LiteRT-LM#3986)

mlboydaisuke

Verified signature
e09f8aa
12:09
MiniCPM-OCR-1.0-VL_int8vis.litertlm: weights moved to TFLiteWeights sections at the end of the file (LiteRT external weights); Android GPU needs LiteRT-LM 0.19.0 or later

mlboydaisuke

Verified signature
7ac3219
11:40
litertlm_manifest.json: MiniCPM-OCR-1.0-VL_int8.litertlm's sha256, size and sections after the move to external weights; min runtime 0.19.0 and the Android GPU known issue (LiteRT-LM#3986)

mlboydaisuke

Verified signature
0faf630
11:40
MiniCPM-OCR-1.0-VL_int8.litertlm: weights moved to TFLiteWeights sections at the end of the file (LiteRT external weights); Android GPU needs LiteRT-LM 0.19.0 or later

mlboydaisuke

Verified signature
27a9d61

Nov 8 5 days ago

07:33
Card: a conversation stops after a message that does not fit in the context window

mlboydaisuke

Verified signature
15a73f1

Nov 7 6 days ago

07:04
Engine-per-conversation note: add the fix status (LiteRT-LM main e1f86b05, no release yet)

mlboydaisuke

Verified signature
8e2b020

Nov 3 10 days ago

17:23
Card: link the math_to_latex sample in google-ai-edge/litert-samples; Mac GPU long-output note

mlboydaisuke

Verified signature
22f7833

Oct 30 14 days ago

14:40
Recommend MiniCPM-OCR-1.0-VL_int8vis.litertlm on phone CPUs⁠

mlboydaisuke

Verified signature
b3110ea

Oct 29 15 days ago

11:40
MiniCPM-OCR-1.0 for LiteRT-LM: int8 decoder + fp16 vision, regions and tiled pages

mlboydaisuke

Verified signature
c818533
11:34
initial commit

mlboydaisuke

Verified signature
28563f2

Commit History

Nov 9 4 days ago

litertlm_manifest.json: MiniCPM-OCR-1.0-VL_int8vis.litertlm's sha256, size and sections after the move to external weights; min runtime 0.19.0 and the Android GPU known issue (LiteRT-LM#3986)

MiniCPM-OCR-1.0-VL_int8vis.litertlm: weights moved to TFLiteWeights sections at the end of the file (LiteRT external weights); Android GPU needs LiteRT-LM 0.19.0 or later

litertlm_manifest.json: MiniCPM-OCR-1.0-VL_int8.litertlm's sha256, size and sections after the move to external weights; min runtime 0.19.0 and the Android GPU known issue (LiteRT-LM#3986)

MiniCPM-OCR-1.0-VL_int8.litertlm: weights moved to TFLiteWeights sections at the end of the file (LiteRT external weights); Android GPU needs LiteRT-LM 0.19.0 or later

Nov 8 5 days ago

Card: a conversation stops after a message that does not fit in the context window

Nov 7 6 days ago

Engine-per-conversation note: add the fix status (LiteRT-LM main e1f86b05, no release yet)

Nov 3 10 days ago

Card: link the math_to_latex sample in google-ai-edge/litert-samples; Mac GPU long-output note

Oct 30 14 days ago

Recommend MiniCPM-OCR-1.0-VL_int8vis.litertlm on phone CPUs⁠

Oct 29 15 days ago

MiniCPM-OCR-1.0 for LiteRT-LM: int8 decoder + fp16 vision, regions and tiled pages

initial commit