Jeremy Park

jeremyparkphd

jeremyipark

AI & ML interests

Computer vision, VLMs

Recent Activity

reacted to spillai's post with 🔥 29 days ago

We're excited to introduce VLM Run Gateway - a new unified OpenAI-compatible API for running open-weight VLMs, OCR VLMs and ViT-based vision models. https://vlm.run/gateway Full model catalog: https://vlm.run/gateway/models Blog post announcement: https://www.vlm.run/blog/introducing-gateway Try different models on the gateway simply by updating the model name. Free to use and no sign-up required for now (in alpha). $ uvx vlmrun gw models $ uvx vlmrun gw chat <doc>.pdf -m glm-ocr $ uvx vlmrun gw chat <doc>.pdf -m deepseek-ocr-2 $ uvx vlmrun gw chat <doc>.pdf -m pp-ocrv6 $ uvx vlmrun gw chat <img>.jpg -m qwen/qwen3.5-0.8b -p "describe the image" $ uvx vlmrun gw chat <vid>.mp4 -m qwen/qwen3.5-0.8b -p "describe the video"

upvoted an article 29 days ago

VLM Run Gateway: Run open-weight OCR, VLM and vision models behind one API

upvoted an article about 2 months ago

VLM Run Gateway: Run GLM-OCR, DeepSeek-OCR-2, dots.mocr with an OpenAI Compatible API

View all activity

Organizations

None yet

reacted to spillai's post with 🔥 29 days ago

Post

3207

We're excited to introduce VLM Run Gateway - a new unified OpenAI-compatible API for running open-weight VLMs, OCR VLMs and ViT-based vision models.

https://vlm.run/gateway
Full model catalog: https://vlm.run/gateway/models
Blog post announcement: https://www.vlm.run/blog/introducing-gateway

Try different models on the gateway simply by updating the model name. Free to use and no sign-up required for now (in alpha).

$ uvx vlmrun gw models
$ uvx vlmrun gw chat <doc>.pdf -m glm-ocr
$ uvx vlmrun gw chat <doc>.pdf -m deepseek-ocr-2
$ uvx vlmrun gw chat <doc>.pdf -m pp-ocrv6
$ uvx vlmrun gw chat <img>.jpg -m qwen/qwen3.5-0.8b -p "describe the image"
$ uvx vlmrun gw chat <vid>.mp4 -m qwen/qwen3.5-0.8b -p "describe the video"

upvoted an article 29 days ago

Article

VLM Run Gateway: Run open-weight OCR, VLM and vision models behind one API

vlm-run

•

29 days ago

• 6

upvoted an article about 2 months ago

Article

VLM Run Gateway: Run GLM-OCR, DeepSeek-OCR-2, dots.mocr with an OpenAI Compatible API

vlm-run

•

Aug 19

• 6

reacted to nwaughachukwuma's post with 🔥 about 2 months ago

Post

3587

# VLM Run Gateway: Run GLM-OCR, DeepSeek-OCR-2, and Dots.mocr with an OpenAI Compatible API

Open-weight OCR VLMs have advanced significantly over the past year, yet most teams still rely on frontier VLMs for document parsing because researching, evaluating, and deploying the right models remains challenging.

So we built VLM Run Gateway: one OpenAI-compatible endpoint for open-weight OCR and VLM models.

If you’re using frontier VLMs primarily for OCR/document parsing, open-weight OCR models can be dramatically cheaper and often very accurate. With a one-line change, you can switch between open-weight OCR VLMs (DeepSeek OCR 2, GLM-OCR, dots.mocr, Paddle OCR VL, PP-OCRv6, etc.) and process 100K+ pages for under $60.

Try it out quickly via the CLI:

uvx vlmrun gw models
uvx vlmrun config set --api-key '<VLMRUN_API_KEY>' # anon-user, rate-limited
uvx vlmrun gw chat <doc>.pdf -m zai-org/glm-ocr

uvx vlmrun gw chat <doc>.pdf -m zai-org/glm-ocr --json-mode
uvx vlmrun gw chat <doc>.pdf -m deepseek-ai/deepseek-ocr-2
uvx vlmrun gw chat <doc>.pdf -m rednote-hilab/dots.mocr
uvx vlmrun gw chat <doc>.pdf -m paddleocr/pp-ocrv6

OpenAI SDK:

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="rednote-hilab/dots.mocr",
    messages=[{
      "role": "user",
		  "content": [{
            "type": "document_url",
            "document_url": {"url": "https://.../invoice.pdf"},
      }],
    }],
    extra_body={"document_dpi": 72},
)

Docs: https://docs.vlm.run/gateway

Catalog: https://docs.vlm.run/gateway/models

MCP: https://docs.vlm.run/gateway/mcp-server

Colab Quickstart: https://colab.research.google.com/drive/1RkuVIyuc5Po-UlcSlFyJCam5tjCm9IHM?usp=sharing

Read the full post here: https://huggingface.co/blog/vlm-run/intro-to-vlmrun-gateway

reacted to nwaughachukwuma's post with ❤️ 2 months ago

Post

747

# One API for Every Visual & OCR Models.

The VLM Run Gateway is a fully compatible API for OpenAI chat completions for visual intelligence. If you’re building document extraction or visual understanding, the Gateway exposes OCR, VQA, and detection behind a single interface you already know.

Read the docs: https://docs.vlm.run/gateway/introduction.

We actively support the following recent OCR and VQA models, which you can try today at no cost:

* zai-org/glm-ocr
* rednote-hilab/dots.mocr
* paddleocr/pp-ocrv6
* qwen/qwen3.5-0.8b

## Quickstart

### CLI

uvx vlmrun gw models
uvx vlmrun config set --api-key '<VLMRUN_API_KEY>' # anon-user, rate-limited
uvx vlmrun gw chat <doc>.pdf -m zai-org/glm-ocr

uvx vlmrun gw chat <doc>.pdf -m zai-org/glm-ocr --json-mode
uvx vlmrun gw chat <doc>.pdf -m deepseek-ai/deepseek-ocr-2
uvx vlmrun gw chat <doc>.pdf -m rednote-hilab/dots.mocr
uvx vlmrun gw chat <doc>.pdf -m paddleocr/pp-ocrv6

### OpenAI SDK

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)


response = client.chat.completions.create(
    model="zai-org/glm-ocr",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "document_url",
                    "document_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/finance.sec-filings/tsla-8k.pdf"
                    },
                },
            ],
        }
    ],
    extra_body={"method": "markdown", "document_dpi": 150},
)

print(response.choices[0].message.content)

## Auth and limits

Anonymous auth is enabled, so you can omit the authorization header entirely, or send Bearer "" or Bearer vlmrun. Rate limits are 60 req/min and 1000 req/hr.

reacted to spillai's post with 🔥 5 months ago

Post

8800

mm-ctx – fast, multimodal context for agents.

LLM-based agents handle text incredibly well, but images, videos, or PDFs with visual content are hard to interpret. mm-ctx gives your CLI agent multi-modal skills.

Try it interactively in Spaces: vlm-run/mm-ctx

Readme: https://vlm-run.github.io/mm/
PyPI: https://pypi.org/project/mm-ctx
SKILL.md: https://github.com/vlm-run/skills/blob/main/skills/mm-cli-skill/SKILL.md

mm-ctx is meant to feel familiar: the UNIX tools we already love (find/cat/grep/wc), rebuilt for file types LLMs can't read natively and designed to work with agents via the CLI.
- mm grep "invoice #1234" ~/Downloads searches across PDFs and returns line-numbered matches
- mm cat <document>.pdf returns a metadata description of the file
- mm cat <photo>.jpg returns a caption of the photo
- mm cat <video>.mp4 returns a caption of the video

A few things we obsessed over:
⚡ Speed: Rust core for the hot paths
🏠 Local-first, BYO model: Uses any OpenAI-compatible endpoint: Ollama, vLLM/SGLang, LMStudio with any multimodal LLM (Gemma4, Qwen3.5, GLM-4.6V).
🔗 Composable: stdin + structured outputs
🤖 Drops into any agent via mm-cli-skills: Claude Code, Codex, Gemini CLI, OpenClaw.

We’d love to hear your feedback! Especially on the CLI and what file types and workflows you would like to see next.

2 replies

Jeremy Park

AI & ML interests

Recent Activity

Organizations

jeremyparkphd's activity

VLM Run Gateway: Run open-weight OCR, VLM and vision models behind one API

VLM Run Gateway: Run GLM-OCR, DeepSeek-OCR-2, dots.mocr with an OpenAI Compatible API