Instructions to use KORMo-VL/KORMo-Web-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use KORMo-VL/KORMo-Web-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="KORMo-VL/KORMo-Web-4B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("KORMo-VL/KORMo-Web-4B")
model = AutoModelForMultimodalLM.from_pretrained("KORMo-VL/KORMo-Web-4B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use KORMo-VL/KORMo-Web-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "KORMo-VL/KORMo-Web-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KORMo-VL/KORMo-Web-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/KORMo-VL/KORMo-Web-4B

SGLang

How to use KORMo-VL/KORMo-Web-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "KORMo-VL/KORMo-Web-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KORMo-VL/KORMo-Web-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "KORMo-VL/KORMo-Web-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KORMo-VL/KORMo-Web-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use KORMo-VL/KORMo-Web-4B with Docker Model Runner:
```
docker model run hf.co/KORMo-VL/KORMo-Web-4B
```

KORMo-Web-4B

KORMo-Web-4B is a 4B vision-language agent that completes real browser tasks from screenshots alone. Give it a request in Korean or English; it plans, clicks, types and moves between sites and tabs until the task is done. In the same live-web evaluation, it beats every open-weight web agent we compared, by +9.6 points in English and +22.1 points in Korean.

On Online-Mind2Web it reaches 42.3% in English and 42.3% in Korean, which is +22.3 / +28.8 points over the base Qwen3.5-4B.

Performance

All models below ran the same 612 live-website tasks (300 English, 312 Korean), each with its own prompt and action format, and were scored by the same LLM judge.

Model	Size	Online-Mind2Web EN	Online-Mind2Web KO
Qwen3.5-4B (base, no fine-tuning)	4B	20.0	13.5
Fara-1.5-4B	4B	20.7	7.7
UI-TARS-1.5-7B	7B	22.3	20.2
OpenWebRL-4B	4B	32.7	15.4
KORMo-Web-4B (ours)	4B	42.3	42.3
KORMo-Web-9B (ours)	9B	51.0	48.1

Success rate (%). The denominator is always the full task count, so crashed or empty runs count as failures.

KORMo-Web-4B outperforms open web agents of similar size in both English and Korean.

Why KORMo-Web

Korean and English, natural requests. Handles short commands as well as long, conversational instructions in either language.
Screenshot-only. It needs no DOM or accessibility tree; it acts on what is on screen, like a person, and checks each action's result on the next screenshot.
Thinks, then acts. A short reasoning step precedes every action, so its behaviour is easy to inspect and debug.
Multi-site, multi-tab workflows. It can search on one site, verify on another and write the result into a third, managing tabs on its own.
Open and ready to run. Apache-2.0 weights, a reference browser harness, the tool schema and the exact inference settings are all included.

Quickstart

Tested with vLLM 0.23.0 and transformers 5.12.1. The model runs on a single GPU (TP=1).

Run the agent

1. Install and download

pip install -U "vllm>=0.23.0" openai playwright huggingface_hub
playwright install chromium
huggingface-cli download KORMo-VL/KORMo-Web-4B --local-dir KORMo-Web-4B

2. Start the model server

vllm serve ./KORMo-Web-4B --served-model-name KORMo-Web-4B \
  --max-model-len 131072 --limit-mm-per-prompt '{"image":1,"video":0}'

3. Run a browser task

python KORMo-Web-4B/examples/agent_playwright.py --model KORMo-Web-4B \
  --task "다나와에서 50만 원 이하, 10인치 이상 태블릿 3개를 찾아서 가격을 알려줘"

The agent opens Chromium, prints each action as it goes, and finishes with the model's answer.

Inference example (one step)

Send one screenshot with the request and get back one action:

import base64, json
from openai import OpenAI

SYSTEM = open("KORMo-Web-4B/system_prompt.md").read().strip()
TOOLS = json.load(open("KORMo-Web-4B/tools.json"))
shot = base64.b64encode(open("screen.png", "rb").read()).decode()

obs = ("<observation>\nscreen size: 1280 x 720\nCurrent tab: 0\nAvailable tabs:\n"
       "- Tab 0 (active): https://www.google.com/\n\nurl: https://www.google.com/\nscreenshot:\n")
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": [
        {"type": "text", "text": "<user_request>\n구글에서 KAIST 를 검색해서 공식 홈페이지를 열어줘.\n</user_request>\n" + obs},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{shot}"}},
        {"type": "text", "text": "\n</observation>"},
    ]},
]
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
out = client.chat.completions.create(
    model="KORMo-Web-4B", messages=messages, tools=TOOLS,
    temperature=0.6, top_p=0.95, max_tokens=16384,
    extra_body={"top_k": 20, "chat_template_kwargs": {"enable_thinking": True}},
)
print(out.choices[0].message.content)

The output looks like this (illustrative):

<think>
I can see the Google home page with an empty search box in the center. I have not searched yet.
My plan: 1. Click the search box. 2. Type "KAIST". 3. Press Enter and open the official site.
</think>

<tool_call>
<function=click>
<parameter=x>
500
</parameter>
<parameter=y>
427
</parameter>
</function>
</tool_call>

Coordinates in the tool call are normalized to 0–1000. Convert them to pixels with x / 1000 × width and y / 1000 × height.

How It Works

user       <user_request> task </user_request>  + observation (tabs, url, screenshot)
assistant  <think> what I see · what is done · plan </think>  <tool_call> one action </tool_call>
tool       action result  + new observation (newest screenshot only)
…          repeat until finished(answer)

Action space: 22 browser tools: click, type, hotkey, scroll, drag, goto, tab control, finished, and others (tools.json).
Memory: the full text history of thoughts, actions and observations is kept. Only the current screenshot is sent.
Recommended settings: temperature 0.6, top_p 0.95, top_k 20, thinking on, up to 16k tokens per step and 50 steps. These are already set in generation_config.json.

The exact message format, parsing rules and feedback strings are in HARNESS.md. Follow it if you build your own harness.

Evaluation Details


Tasks	Online-Mind2Web: 300 English, 312 Korean
Judge	GPT-6

Files

File	Description
`chat_template.jinja`	Chat template (Qwen3.5 XML tool calls, thinking on)
`system_prompt.md` · `tools.json`	System prompt and 22-tool schema, verbatim from training
`HARNESS.md`	Full harness specification
`examples/agent_playwright.py`	Reference browser agent (Playwright + OpenAI-compatible API). It does not execute `upload`.
`config.json` · `generation_config.json` · `processor_config.json` · `tokenizer*.json`	Model, sampling and processor configs