Instructions to use KORMo-VL/KORMo-Web-4B-mlx-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use KORMo-VL/KORMo-Web-4B-mlx-4bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("KORMo-VL/KORMo-Web-4B-mlx-4bit") config = load_config("KORMo-VL/KORMo-Web-4B-mlx-4bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use KORMo-VL/KORMo-Web-4B-mlx-4bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "KORMo-VL/KORMo-Web-4B-mlx-4bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "KORMo-VL/KORMo-Web-4B-mlx-4bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use KORMo-VL/KORMo-Web-4B-mlx-4bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "KORMo-VL/KORMo-Web-4B-mlx-4bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default KORMo-VL/KORMo-Web-4B-mlx-4bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use KORMo-VL/KORMo-Web-4B-mlx-4bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "KORMo-VL/KORMo-Web-4B-mlx-4bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "KORMo-VL/KORMo-Web-4B-mlx-4bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
KORMo-Web-4B
KORMo-Web-4B is a 4B vision-language agent that completes real browser tasks from screenshots alone. Give it a request in Korean or English; it plans, clicks, types and moves between sites and tabs until the task is done. In the same live-web evaluation, it beats every open-weight web agent we compared, by +9.6 points in English and +22.1 points in Korean.
On Online-Mind2Web it reaches 42.3% in English and 42.3% in Korean, which is +22.3 / +28.8 points over the base Qwen3.5-4B.
What Makes KORMo-Web Different?
- 10M+ Atomic-Level Web Pretraining Samples. Large-scale Korean and English web understanding pretraining teaches the model what individual UI elements are, where they are, and what they doโcovering text, location, appearance, layout, function, and state.
- 100K+ Web Interaction Trajectories. Bilingual browser demonstrations teach the model to translate fine-grained web understanding into real-world, multi-step actions.
- Open Models and Data. Releasing model weights, large-scale bilingual web pretraining data, interaction trajectories, and evaluation benchmarks to support reproducible web agent research.
Performance
All models below ran the same 612 live-website tasks (300 English, 312 Korean), each with its own prompt and action format, and were scored by the same LLM judge.
| Model | Size | Online-Mind2Web EN | Online-Mind2Web KO |
|---|---|---|---|
| Qwen3.5-4B (base, no fine-tuning) | 4B | 20.0 | 13.5 |
| Fara-1.5-4B | 4B | 20.7 | 7.7 |
| UI-TARS-1.5-7B | 7B | 22.3 | 20.2 |
| OpenWebRL-4B | 4B | 32.7 | 15.4 |
| KORMo-Web-4B (ours) | 4B | 42.3 | 42.3 |
| KORMo-Web-9B (ours) | 9B | 51.0 | 48.1 |
Success rate (%). The denominator is always the full task count, so crashed or empty runs count as failures.
KORMo-Web-4B outperforms open web agents of similar size in both English and Korean.
Why KORMo-Web
- Korean and English, natural requests. Handles short commands as well as long, conversational instructions in either language.
- Screenshot-only. It needs no DOM or accessibility tree; it acts on what is on screen, like a person, and checks each action's result on the next screenshot.
- Thinks, then acts. A short reasoning step precedes every action, so its behaviour is easy to inspect and debug.
- Multi-site, multi-tab workflows. It can search on one site, verify on another and write the result into a third, managing tabs on its own.
- Open and ready to run. Apache-2.0 weights, a reference browser harness, the tool schema and the exact inference settings are all included.
Quickstart
This repository holds the 4-bit MLX weights of KORMo-Web-4B for Apple Silicon Macs. Serve it with vllm-mlx (0.6.0 or later), which exposes the same OpenAI-compatible API as vLLM.
Run the agent
1. Install and download
pip install -U "vllm-mlx>=0.6.0" openai playwright huggingface_hub
playwright install chromium
huggingface-cli download KORMo-VL/KORMo-Web-4B-mlx-4bit --local-dir KORMo-Web-4B-mlx-4bit
# system prompt, tool schema and reference harness (shared with the full-precision repo)
huggingface-cli download KORMo-VL/KORMo-Web-4B system_prompt.md tools.json HARNESS.md examples/agent_playwright.py \
--local-dir KORMo-Web-4B-mlx-4bit
2. Start the model server
vllm-mlx serve ./KORMo-Web-4B-mlx-4bit --served-model-name KORMo-Web-4B --mllm --port 8000
Do not enable --enable-auto-tool-choice or --reasoning-parser: the harness parses the raw <think> and <tool_call> text from the message content.
3. Run a browser task
python KORMo-Web-4B-mlx-4bit/examples/agent_playwright.py --model KORMo-Web-4B \
--endpoint http://localhost:8000/v1 \
--task "๋ค๋ฐฉ์์ ์ํ์ญ ๊ทผ์ฒ ๋ณด์ฆ๊ธ ์ฒ๋ง์ ์ดํ๋ก ์ค์ ํ๋ค ์๋ฃธ 3๊ฐ์ ๋ ์ฐพ์์ ์ดํด๋ณด๊ณ ๋คํ๋ฉด ์ฐพ์ ๋ด์ฉ์ ์นด์ด์คํธ ๋ฉ์ผ๋ก mlplab@kaist.ac.kr ํํ
์ ๋ฆฌํด์ ๋ณด๋ด์ค"
The agent opens Chromium, prints each action as it goes, and finishes with the model's answer.
Inference example (one step)
Send one screenshot with the request and get back one action:
import base64, json
from openai import OpenAI
SYSTEM = open("KORMo-Web-4B-mlx-4bit/system_prompt.md").read().strip()
TOOLS = json.load(open("KORMo-Web-4B-mlx-4bit/tools.json"))
shot = base64.b64encode(open("screen.png", "rb").read()).decode()
obs = ("<observation>\nscreen size: 1280 x 720\nCurrent tab: 0\nAvailable tabs:\n"
"- Tab 0 (active): https://www.google.com/\n\nurl: https://www.google.com/\nscreenshot:\n")
messages = [
{"role": "system", "content": SYSTEM},
{"role": "user", "content": [
{"type": "text", "text": "<user_request>\n๊ตฌ๊ธ์์ KAIST ๋ฅผ ๊ฒ์ํด์ ๊ณต์ ํํ์ด์ง๋ฅผ ์ด์ด์ค.\n</user_request>\n" + obs},
{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{shot}"}},
{"type": "text", "text": "\n</observation>"},
]},
]
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") # vllm-mlx server
out = client.chat.completions.create(
model="KORMo-Web-4B", messages=messages, tools=TOOLS,
temperature=0.6, top_p=0.95, max_tokens=16384,
extra_body={"top_k": 20, "chat_template_kwargs": {"enable_thinking": True}},
)
print(out.choices[0].message.content)
The output looks like this (illustrative):
<think>
I can see the Google home page with an empty search box in the center. I have not searched yet.
My plan: 1. Click the search box. 2. Type "KAIST". 3. Press Enter and open the official site.
</think>
<tool_call>
<function=click>
<parameter=x>
500
</parameter>
<parameter=y>
427
</parameter>
</function>
</tool_call>
Coordinates in the tool call are normalized to 0โ1000. Convert them to pixels with x / 1000 ร width and y / 1000 ร height.
How It Works
user <user_request> task </user_request> + observation (tabs, url, screenshot)
assistant <think> what I see ยท what is done ยท plan </think> <tool_call> one action </tool_call>
tool action result + new observation (newest screenshot only)
โฆ repeat until finished(answer)
- Action space: 22 browser tools: click, type, hotkey, scroll, drag, goto, tab control,
finished, and others (tools.json). - Memory: the full text history of thoughts, actions and observations is kept. Only the current screenshot is sent.
- Recommended settings: temperature 0.6, top_p 0.95, top_k 20, thinking on, up to 16k tokens per step and 50 steps. These are already set in
generation_config.json.
The exact message format, parsing rules and feedback strings are in HARNESS.md. Follow it if you build your own harness.
Evaluation Details
| Tasks | Online-Mind2Web: 300 English, 312 Korean |
| Judge | GPT-6 |
Files
| File | Description |
|---|---|
chat_template.jinja |
Chat template (Qwen3.5 XML tool calls, thinking on) |
system_prompt.md ยท tools.json |
System prompt and 22-tool schema, verbatim from training |
HARNESS.md |
Full harness specification |
examples/agent_playwright.py |
Reference browser agent (Playwright + OpenAI-compatible API). It does not execute upload. |
config.json ยท generation_config.json ยท processor_config.json ยท tokenizer*.json |
Model, sampling and processor configs |
ํ๊ตญ์ด ์์ฝ
KORMo-Web-4B๋ ์ฌ๋์ฒ๋ผ ์น ํ๋ฉด์ ๋ณด๊ณ ํด๋ฆญยท์ ๋ ฅยทํ์์ ์ํํ๋ 4B ๊ท๋ชจ์ ํ์ ์๊ฐ ๊ธฐ๋ฐ ์น ์์ด์ ํธ์ ๋๋ค.
- ๋๊ท๋ชจ ์น ์ฌ์ ํ์ต: 1,000๋ง ๊ฑด ์ด์์ ํ์ ์น ํ์ต ์ํ๋ก UI ์์์ ์์นยท์๋ฏธยท๊ตฌ์กฐยท๊ธฐ๋ฅ ๋ฑ์ ์ธ๋ฐํ๊ฒ ํ์ต(Atomic-Level Pretraining)
- ์ค์ ์น ํ๋ ํ์ต: 10๋ง ๊ฑด ์ด์์ ํ๊ตญ์ธ ํ์ ์น ์ฌ์ฉ ๊ถค์ ์ ํ์ตํด ์ฌ๋ฌ ์น์ฌ์ดํธ์ ํญ์ ๋๋๋๋ ๋ณต์กํ ์์ ์ํ
- ๋ฐ์ด๋ ์ฑ๋ฅ: Online-Mind2Web์์ ์์ดยทํ๊ตญ์ด ๋ชจ๋ 42.3% ์ฑ๊ณต๋ฅ , ๋น๊ตํ ๊ณต๊ฐ ๋ชจ๋ธ ์ค ์ต๊ณ ์ฑ๋ฅ(๊ธฐ์กด ์ต๊ณ ๋๋น +9.6%p / +22.1%p)
- ์๊ฐ ์ค์ฌ ์น ์กฐ์: ๋ณ๋์ DOM ์ ๋ณด ์์ด ์คํฌ๋ฆฐ์ท๋ง์ผ๋ก ์น์ ์ดํดํ๊ณ , ํ๋ ๊ฒฐ๊ณผ๋ฅผ ํ์ธํ๋ฉฐ ๋ค์ ์์ ์ํ
- ๋ชจ๋ธยท๋ฐ์ดํฐ ๊ณต๊ฐ: Apache-2.0 ๋ชจ๋ธ ๊ฐ์ค์น, ๋๊ท๋ชจ ํ์ต ๋ฐ์ดํฐ, ์น ์ฌ์ฉ ๊ถค์ , ํ๊ฐ ๋ฒค์น๋งํฌ ๋ฐ ์คํ ์ฝ๋ ๊ณต๊ฐ
Acknowledgements
Built on Qwen3.5-4B. Evaluated on Online-Mind2Web and its Korean counterpart.
- Downloads last month
- -
4-bit