Instructions to use KORMo-VL/KORMo-Web-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use KORMo-VL/KORMo-Web-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="KORMo-VL/KORMo-Web-4B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("KORMo-VL/KORMo-Web-4B")
model = AutoModelForMultimodalLM.from_pretrained("KORMo-VL/KORMo-Web-4B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use KORMo-VL/KORMo-Web-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "KORMo-VL/KORMo-Web-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KORMo-VL/KORMo-Web-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/KORMo-VL/KORMo-Web-4B

SGLang

How to use KORMo-VL/KORMo-Web-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "KORMo-VL/KORMo-Web-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KORMo-VL/KORMo-Web-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "KORMo-VL/KORMo-Web-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KORMo-VL/KORMo-Web-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use KORMo-VL/KORMo-Web-4B with Docker Model Runner:
```
docker model run hf.co/KORMo-VL/KORMo-Web-4B
```

KORMo-Web-4B / README.md

JLee0

Show demo video at the top of the card

2da3ed9 verified about 2 hours ago

preview code

Raw History Blame Contribute Delete

8.7 kB

---
license: apache-2.0
base_model:
- Qwen/Qwen3.5-4B
language:
- ko
- en
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- gui-agent
- web-agent
- computer-use
- browser
- korean
---
<style>
#kormo-web-card { font-size: 16px; line-height: 1.6; overflow-wrap: break-word; }
#kormo-web-card h1 { margin: 0 0 12px; font-size: 30px; line-height: 1.25; letter-spacing: -0.018em; }
#kormo-web-card h2 { margin: 30px 0 10px; font-size: 22px; line-height: 1.3; letter-spacing: -0.012em; }
#kormo-web-card h3 { margin: 22px 0 8px; font-size: 17px; font-weight: 700; line-height: 1.4; }
#kormo-web-card h2 + h3 { margin-top: 14px; }
#kormo-web-card p { margin: 0 0 10px; }
#kormo-web-card ul, #kormo-web-card ol { margin: 6px 0 12px; padding-left: 22px; }
#kormo-web-card li { margin: 4px 0; line-height: 1.55; }
#kormo-web-card table { width: 100%; margin: 6px 0 14px; border-collapse: collapse; font-size: 14px; line-height: 1.45; font-variant-numeric: tabular-nums; }
#kormo-web-card th, #kormo-web-card td { padding: 7px 10px; vertical-align: middle; }
#kormo-web-card blockquote { margin: 12px 0 16px; padding: 0 0 0 14px; font-size: 15px; }
#kormo-web-card blockquote p { margin: 0; }
#kormo-web-card pre { margin: 10px 0 14px; padding: 14px 16px; overflow-x: auto; line-height: 1.5; }
#kormo-web-card pre code { font-size: 13px; }
#kormo-web-card > p:has(> em:only-child) { margin: 4px 0 14px; font-size: 13px; }
#kormo-web-card .pills { margin: 4px 0 18px; }
#kormo-web-card .pill { display: inline-block; margin: 0 6px 6px 0; padding: 3px 10px; border-radius: 999px; border: 1px solid rgba(127,127,127,.35); font-size: 13px; }
@media (max-width: 640px) {
  #kormo-web-card h1 { font-size: 26px; }
  #kormo-web-card table { font-size: 13px; }
  #kormo-web-card th, #kormo-web-card td { padding: 7px 6px; }
}
</style>

<div id="kormo-web-card">

<video src="https://huggingface.co/KORMo-VL/KORMo-Web-4B/resolve/main/assets/demo.mp4" controls autoplay muted loop playsinline width="100%"></video>

# KORMo-Web-4B

**KORMo-Web-4B** is a 4B vision-language agent that completes real browser tasks **from screenshots alone**. Give it a request in Korean or English; it plans, clicks, types and moves between sites and tabs until the task is done. In the same live-web evaluation, it beats every open-weight web agent we compared, **by +9.6 points in English and +22.1 points in Korean**.

On **Online-Mind2Web** it reaches **42.3% in English** and **42.3% in Korean**, which is **+22.3 / +28.8 points** over the base Qwen3.5-4B.

## Performance

All models below ran **the same 612 live-website tasks** (300 English, 312 Korean), each with its own prompt and action format, and were scored by **the same LLM judge**.

| Model | Size | Online-Mind2Web EN | Online-Mind2Web KO |
|---|---:|---:|---:|
| Qwen3.5-4B (base, no fine-tuning) | 4B | 20.0 | 13.5 |
| Fara-1.5-4B | 4B | 20.7 | 7.7 |
| UI-TARS-1.5-7B | 7B | 22.3 | 20.2 |
| OpenWebRL-4B | 4B | 32.7 | 15.4 |
| **KORMo-Web-4B (ours)** | **4B** | **42.3** | **42.3** |
| KORMo-Web-9B (ours) | 9B | 51.0 | 48.1 |


Success rate (%). The denominator is always the full task count, so crashed or empty runs count as failures.

KORMo-Web-4B outperforms open web agents of similar size in both English and Korean.

## Why KORMo-Web

- **Korean and English, natural requests.** Handles short commands as well as long, conversational instructions in either language.
- **Screenshot-only.** It needs no DOM or accessibility tree; it acts on what is on screen, like a person, and checks each action's result on the next screenshot.
- **Thinks, then acts.** A short reasoning step precedes every action, so its behaviour is easy to inspect and debug.
- **Multi-site, multi-tab workflows.** It can search on one site, verify on another and write the result into a third, managing tabs on its own.
- **Open and ready to run.** Apache-2.0 weights, a reference browser harness, the tool schema and the exact inference settings are all included.

## Quickstart

Tested with **vLLM 0.23.0** and **transformers 5.12.1**. The model runs on a single GPU (TP=1).

### Run the agent

**1. Install and download**

```bash
pip install -U "vllm>=0.23.0" openai playwright huggingface_hub
playwright install chromium
huggingface-cli download KORMo-VL/KORMo-Web-4B --local-dir KORMo-Web-4B
```

**2. Start the model server**

```bash
vllm serve ./KORMo-Web-4B --served-model-name KORMo-Web-4B \
  --max-model-len 131072 --limit-mm-per-prompt '{"image":1,"video":0}'
```

**3. Run a browser task**

```bash
python KORMo-Web-4B/examples/agent_playwright.py --model KORMo-Web-4B \
  --task "다나와에서 50만 원 이하, 10인치 이상 태블릿 3개를 찾아서 가격을 알려줘"
```

The agent opens Chromium, prints each action as it goes, and finishes with the model's answer.

### Inference example (one step)

Send one screenshot with the request and get back one action:

```python
import base64, json
from openai import OpenAI
SYSTEM = open("KORMo-Web-4B/system_prompt.md").read().strip()
TOOLS = json.load(open("KORMo-Web-4B/tools.json"))
shot = base64.b64encode(open("screen.png", "rb").read()).decode()
obs = ("<observation>\nscreen size: 1280 x 720\nCurrent tab: 0\nAvailable tabs:\n"
       "- Tab 0 (active): https://www.google.com/\n\nurl: https://www.google.com/\nscreenshot:\n")
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": [
        {"type": "text", "text": "<user_request>\n구글에서 KAIST 를 검색해서 공식 홈페이지를 열어줘.\n</user_request>\n" + obs},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{shot}"}},
        {"type": "text", "text": "\n</observation>"},
    ]},
]
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
out = client.chat.completions.create(
    model="KORMo-Web-4B", messages=messages, tools=TOOLS,
    temperature=0.6, top_p=0.95, max_tokens=16384,
    extra_body={"top_k": 20, "chat_template_kwargs": {"enable_thinking": True}},
)
print(out.choices[0].message.content)
```

The output looks like this (illustrative):

```text
<think>
I can see the Google home page with an empty search box in the center. I have not searched yet.
My plan: 1. Click the search box. 2. Type "KAIST". 3. Press Enter and open the official site.
</think>
<tool_call>
<function=click>
<parameter=x>
500
</parameter>
<parameter=y>
427
</parameter>
</function>
</tool_call>
```

Coordinates in the tool call are normalized to 0–1000. Convert them to pixels with `x / 1000 × width` and `y / 1000 × height`.

## How It Works

```
user       <user_request> task </user_request>  + observation (tabs, url, screenshot)
assistant  <think> what I see · what is done · plan </think>  <tool_call> one action </tool_call>
tool       action result  + new observation (newest screenshot only)
…          repeat until finished(answer)
```

- **Action space:** 22 browser tools: click, type, hotkey, scroll, drag, goto, tab control, `finished`, and others ([`tools.json`](tools.json)).
- **Memory:** the full text history of thoughts, actions and observations is kept. Only the current screenshot is sent.
- **Recommended settings:** temperature 0.6, top_p 0.95, top_k 20, thinking on, up to 16k tokens per step and 50 steps. These are already set in `generation_config.json`.

The exact message format, parsing rules and feedback strings are in **[HARNESS.md](HARNESS.md)**. Follow it if you build your own harness.

## Evaluation Details

| | |
|---|---|
| Tasks | Online-Mind2Web: 300 English, 312 Korean |
| Judge | GPT-6 |

## Files

| File | Description |
|---|---|
| `chat_template.jinja` | Chat template (Qwen3.5 XML tool calls, thinking on) |
| `system_prompt.md` · `tools.json` | System prompt and 22-tool schema, verbatim from training |
| `HARNESS.md` | Full harness specification |
| `examples/agent_playwright.py` | Reference browser agent (Playwright + OpenAI-compatible API). It does not execute `upload`. |
| `config.json` · `generation_config.json` · `processor_config.json` · `tokenizer*.json` | Model, sampling and processor configs |

## 한국어 요약

**KORMo-Web-4B**는 스크린샷만 보고 한국어·영어 웹 작업을 수행하는 4B 공개 웹 에이전트입니다. 같은 조건의 실제 웹 평가에서 비교한 공개 모델 중 가장 높은 점수를 받았습니다.

- **Online-Mind2Web:** EN 42.3, KO 42.3
- 비슷한 크기의 공개 웹 에이전트보다 영어·한국어 모두에서 높은 성능을 보입니다.

## Acknowledgements

Built on **[Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B)**. Evaluated on **Online-Mind2Web** and its Korean counterpart.

</div>

	---
	license: apache-2.0
	base_model:
	- Qwen/Qwen3.5-4B
	language:
	- ko
	- en
	pipeline_tag: image-text-to-text
	library_name: transformers
	tags:
	- gui-agent
	- web-agent
	- computer-use
	- browser
	- korean
	---

	<style>
	#kormo-web-card { font-size: 16px; line-height: 1.6; overflow-wrap: break-word; }
	#kormo-web-card h1 { margin: 0 0 12px; font-size: 30px; line-height: 1.25; letter-spacing: -0.018em; }
	#kormo-web-card h2 { margin: 30px 0 10px; font-size: 22px; line-height: 1.3; letter-spacing: -0.012em; }
	#kormo-web-card h3 { margin: 22px 0 8px; font-size: 17px; font-weight: 700; line-height: 1.4; }
	#kormo-web-card h2 + h3 { margin-top: 14px; }
	#kormo-web-card p { margin: 0 0 10px; }
	#kormo-web-card ul, #kormo-web-card ol { margin: 6px 0 12px; padding-left: 22px; }
	#kormo-web-card li { margin: 4px 0; line-height: 1.55; }
	#kormo-web-card table { width: 100%; margin: 6px 0 14px; border-collapse: collapse; font-size: 14px; line-height: 1.45; font-variant-numeric: tabular-nums; }
	#kormo-web-card th, #kormo-web-card td { padding: 7px 10px; vertical-align: middle; }
	#kormo-web-card blockquote { margin: 12px 0 16px; padding: 0 0 0 14px; font-size: 15px; }
	#kormo-web-card blockquote p { margin: 0; }
	#kormo-web-card pre { margin: 10px 0 14px; padding: 14px 16px; overflow-x: auto; line-height: 1.5; }
	#kormo-web-card pre code { font-size: 13px; }
	#kormo-web-card > p:has(> em:only-child) { margin: 4px 0 14px; font-size: 13px; }
	#kormo-web-card .pills { margin: 4px 0 18px; }
	#kormo-web-card .pill { display: inline-block; margin: 0 6px 6px 0; padding: 3px 10px; border-radius: 999px; border: 1px solid rgba(127,127,127,.35); font-size: 13px; }
	@media (max-width: 640px) {
	#kormo-web-card h1 { font-size: 26px; }
	#kormo-web-card table { font-size: 13px; }
	#kormo-web-card th, #kormo-web-card td { padding: 7px 6px; }
	}
	</style>

	<div id="kormo-web-card">

	<video src="https://huggingface.co/KORMo-VL/KORMo-Web-4B/resolve/main/assets/demo.mp4" controls autoplay muted loop playsinline width="100%"></video>

	# KORMo-Web-4B

	KORMo-Web-4B is a 4B vision-language agent that completes real browser tasks from screenshots alone. Give it a request in Korean or English; it plans, clicks, types and moves between sites and tabs until the task is done. In the same live-web evaluation, it beats every open-weight web agent we compared, by +9.6 points in English and +22.1 points in Korean.

	On Online-Mind2Web it reaches 42.3% in English and 42.3% in Korean, which is +22.3 / +28.8 points over the base Qwen3.5-4B.

	## Performance

	All models below ran the same 612 live-website tasks (300 English, 312 Korean), each with its own prompt and action format, and were scored by the same LLM judge.

	\| Model \| Size \| Online-Mind2Web EN \| Online-Mind2Web KO \|
	\|---\|---:\|---:\|---:\|
	\| Qwen3.5-4B (base, no fine-tuning) \| 4B \| 20.0 \| 13.5 \|
	\| Fara-1.5-4B \| 4B \| 20.7 \| 7.7 \|
	\| UI-TARS-1.5-7B \| 7B \| 22.3 \| 20.2 \|
	\| OpenWebRL-4B \| 4B \| 32.7 \| 15.4 \|
	\| KORMo-Web-4B (ours) \| 4B \| 42.3 \| 42.3 \|
	\| KORMo-Web-9B (ours) \| 9B \| 51.0 \| 48.1 \|


	Success rate (%). The denominator is always the full task count, so crashed or empty runs count as failures.

	KORMo-Web-4B outperforms open web agents of similar size in both English and Korean.

	## Why KORMo-Web

	- Korean and English, natural requests. Handles short commands as well as long, conversational instructions in either language.
	- Screenshot-only. It needs no DOM or accessibility tree; it acts on what is on screen, like a person, and checks each action's result on the next screenshot.
	- Thinks, then acts. A short reasoning step precedes every action, so its behaviour is easy to inspect and debug.
	- Multi-site, multi-tab workflows. It can search on one site, verify on another and write the result into a third, managing tabs on its own.
	- Open and ready to run. Apache-2.0 weights, a reference browser harness, the tool schema and the exact inference settings are all included.

	## Quickstart

	Tested with vLLM 0.23.0 and transformers 5.12.1. The model runs on a single GPU (TP=1).

	### Run the agent

	1. Install and download

	```bash
	pip install -U "vllm>=0.23.0" openai playwright huggingface_hub
	playwright install chromium
	huggingface-cli download KORMo-VL/KORMo-Web-4B --local-dir KORMo-Web-4B
	```

	2. Start the model server

	```bash
	vllm serve ./KORMo-Web-4B --served-model-name KORMo-Web-4B \
	--max-model-len 131072 --limit-mm-per-prompt '{"image":1,"video":0}'
	```

	3. Run a browser task

	```bash
	python KORMo-Web-4B/examples/agent_playwright.py --model KORMo-Web-4B \
	--task "다나와에서 50만 원 이하, 10인치 이상 태블릿 3개를 찾아서 가격을 알려줘"
	```

	The agent opens Chromium, prints each action as it goes, and finishes with the model's answer.

	### Inference example (one step)

	Send one screenshot with the request and get back one action:

	```python
	import base64, json
	from openai import OpenAI

	SYSTEM = open("KORMo-Web-4B/system_prompt.md").read().strip()
	TOOLS = json.load(open("KORMo-Web-4B/tools.json"))
	shot = base64.b64encode(open("screen.png", "rb").read()).decode()

	obs = ("<observation>\nscreen size: 1280 x 720\nCurrent tab: 0\nAvailable tabs:\n"
	"- Tab 0 (active): https://www.google.com/\n\nurl: https://www.google.com/\nscreenshot:\n")
	messages = [
	{"role": "system", "content": SYSTEM},
	{"role": "user", "content": [
	{"type": "text", "text": "<user_request>\n구글에서 KAIST 를 검색해서 공식 홈페이지를 열어줘.\n</user_request>\n" + obs},
	{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{shot}"}},
	{"type": "text", "text": "\n</observation>"},
	]},
	]
	client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
	out = client.chat.completions.create(
	model="KORMo-Web-4B", messages=messages, tools=TOOLS,
	temperature=0.6, top_p=0.95, max_tokens=16384,
	extra_body={"top_k": 20, "chat_template_kwargs": {"enable_thinking": True}},
	)
	print(out.choices[0].message.content)
	```

	The output looks like this (illustrative):

	```text
	<think>
	I can see the Google home page with an empty search box in the center. I have not searched yet.
	My plan: 1. Click the search box. 2. Type "KAIST". 3. Press Enter and open the official site.
	</think>

	<tool_call>
	<function=click>
	<parameter=x>
	500
	</parameter>
	<parameter=y>
	427
	</parameter>
	</function>
	</tool_call>
	```

	Coordinates in the tool call are normalized to 0–1000. Convert them to pixels with `x / 1000 × width` and `y / 1000 × height`.

	## How It Works

	```
	user <user_request> task </user_request> + observation (tabs, url, screenshot)
	assistant <think> what I see · what is done · plan </think> <tool_call> one action </tool_call>
	tool action result + new observation (newest screenshot only)
	… repeat until finished(answer)
	```

	- Action space: 22 browser tools: click, type, hotkey, scroll, drag, goto, tab control, `finished`, and others ([`tools.json`](tools.json)).
	- Memory: the full text history of thoughts, actions and observations is kept. Only the current screenshot is sent.
	- Recommended settings: temperature 0.6, top_p 0.95, top_k 20, thinking on, up to 16k tokens per step and 50 steps. These are already set in `generation_config.json`.

	The exact message format, parsing rules and feedback strings are in [HARNESS.md](HARNESS.md). Follow it if you build your own harness.

	## Evaluation Details

	\| \| \|
	\|---\|---\|
	\| Tasks \| Online-Mind2Web: 300 English, 312 Korean \|
	\| Judge \| GPT-6 \|

	## Files

	\| File \| Description \|
	\|---\|---\|
	\| `chat_template.jinja` \| Chat template (Qwen3.5 XML tool calls, thinking on) \|
	\| `system_prompt.md` · `tools.json` \| System prompt and 22-tool schema, verbatim from training \|
	\| `HARNESS.md` \| Full harness specification \|
	\| `examples/agent_playwright.py` \| Reference browser agent (Playwright + OpenAI-compatible API). It does not execute `upload`. \|
	\| `config.json` · `generation_config.json` · `processor_config.json` · `tokenizer*.json` \| Model, sampling and processor configs \|

	## 한국어 요약

	KORMo-Web-4B는 스크린샷만 보고 한국어·영어 웹 작업을 수행하는 4B 공개 웹 에이전트입니다. 같은 조건의 실제 웹 평가에서 비교한 공개 모델 중 가장 높은 점수를 받았습니다.

	- Online-Mind2Web: EN 42.3, KO 42.3
	- 비슷한 크기의 공개 웹 에이전트보다 영어·한국어 모두에서 높은 성능을 보입니다.

	## Acknowledgements

	Built on [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B). Evaluated on Online-Mind2Web and its Korean counterpart.

	</div>