---
license: apache-2.0
base_model:
- KORMo-VL/KORMo-Web-4B
base_model_relation: quantized
language:
- ko
- en
pipeline_tag: image-text-to-text
library_name: mlx
tags:
- mlx
- gui-agent
- web-agent
- computer-use
- browser
- korean
---
# KORMo-Web-4B
**KORMo-Web-4B** is a 4B vision-language agent that completes real browser tasks **from screenshots alone**. Give it a request in Korean or English; it plans, clicks, types and moves between sites and tabs until the task is done. In the same live-web evaluation, it beats every open-weight web agent we compared, **by +9.6 points in English and +22.1 points in Korean**.
On **Online-Mind2Web** it reaches **42.3% in English** and **42.3% in Korean**, which is **+22.3 / +28.8 points** over the base Qwen3.5-4B.
## What Makes KORMo-Web Different?
- **10M+ Atomic-Level Web Pretraining Samples.** Large-scale Korean and English web understanding pretraining teaches the model what individual UI elements **are, where they are, and what they do**—covering text, location, appearance, layout, function, and state.
- **100K+ Web Interaction Trajectories.** Bilingual browser demonstrations teach the model to translate fine-grained web understanding into real-world, multi-step actions.
- **Open Models and Data.** Releasing model weights, large-scale bilingual web pretraining data, interaction trajectories, and evaluation benchmarks to support reproducible web agent research.
## Performance
All models below ran **the same 612 live-website tasks** (300 English, 312 Korean), each with its own prompt and action format, and were scored by **the same LLM judge**.
| Model | Size | Online-Mind2Web EN | Online-Mind2Web KO |
|---|---:|---:|---:|
| Qwen3.5-4B (base, no fine-tuning) | 4B | 20.0 | 13.5 |
| Fara-1.5-4B | 4B | 20.7 | 7.7 |
| UI-TARS-1.5-7B | 7B | 22.3 | 20.2 |
| OpenWebRL-4B | 4B | 32.7 | 15.4 |
| **KORMo-Web-4B (ours)** | **4B** | **42.3** | **42.3** |
| KORMo-Web-9B (ours) | 9B | 51.0 | 48.1 |
Success rate (%). The denominator is always the full task count, so crashed or empty runs count as failures.
KORMo-Web-4B outperforms open web agents of similar size in both English and Korean.
## Why KORMo-Web
- **Korean and English, natural requests.** Handles short commands as well as long, conversational instructions in either language.
- **Screenshot-only.** It needs no DOM or accessibility tree; it acts on what is on screen, like a person, and checks each action's result on the next screenshot.
- **Thinks, then acts.** A short reasoning step precedes every action, so its behaviour is easy to inspect and debug.
- **Multi-site, multi-tab workflows.** It can search on one site, verify on another and write the result into a third, managing tabs on its own.
- **Open and ready to run.** Apache-2.0 weights, a reference browser harness, the tool schema and the exact inference settings are all included.
## Quickstart
This repository holds the **4-bit MLX** weights of KORMo-Web-4B for Apple Silicon Macs. Serve it with **[vllm-mlx](https://github.com/waybarrios/vllm-mlx)** (0.6.0 or later), which exposes the same OpenAI-compatible API as vLLM.
### Run the agent
**1. Install and download**
```bash
pip install -U "vllm-mlx>=0.6.0" openai playwright huggingface_hub
playwright install chromium
huggingface-cli download KORMo-VL/KORMo-Web-4B-mlx-4bit --local-dir KORMo-Web-4B-mlx-4bit
# system prompt, tool schema and reference harness (shared with the full-precision repo)
huggingface-cli download KORMo-VL/KORMo-Web-4B system_prompt.md tools.json HARNESS.md examples/agent_playwright.py \
--local-dir KORMo-Web-4B-mlx-4bit
```
**2. Start the model server**
```bash
vllm-mlx serve ./KORMo-Web-4B-mlx-4bit --served-model-name KORMo-Web-4B --mllm --port 8000
```
Do not enable `--enable-auto-tool-choice` or `--reasoning-parser`: the harness parses the raw `
` and `` text from the message content.
**3. Run a browser task**
```bash
python KORMo-Web-4B-mlx-4bit/examples/agent_playwright.py --model KORMo-Web-4B \
--endpoint http://localhost:8000/v1 \
--task "다방에서 월평역 근처 보증금 천만원 이하로 설정한뒤 원룸 3개정도 찾아서 살펴보고 다하면 찾은 내용을 카이스트 메일로 mlplab@kaist.ac.kr 한테 정리해서 보내줘"
```
The agent opens Chromium, prints each action as it goes, and finishes with the model's answer.
### Inference example (one step)
Send one screenshot with the request and get back one action:
```python
import base64, json
from openai import OpenAI
SYSTEM = open("KORMo-Web-4B-mlx-4bit/system_prompt.md").read().strip()
TOOLS = json.load(open("KORMo-Web-4B-mlx-4bit/tools.json"))
shot = base64.b64encode(open("screen.png", "rb").read()).decode()
obs = ("\nscreen size: 1280 x 720\nCurrent tab: 0\nAvailable tabs:\n"
"- Tab 0 (active): https://www.google.com/\n\nurl: https://www.google.com/\nscreenshot:\n")
messages = [
{"role": "system", "content": SYSTEM},
{"role": "user", "content": [
{"type": "text", "text": "\n구글에서 KAIST 를 검색해서 공식 홈페이지를 열어줘.\n\n" + obs},
{"type": "image_url", "image_url": {"url": f"data:image/png;base64,{shot}"}},
{"type": "text", "text": "\n"},
]},
]
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY") # vllm-mlx server
out = client.chat.completions.create(
model="KORMo-Web-4B", messages=messages, tools=TOOLS,
temperature=0.6, top_p=0.95, max_tokens=16384,
extra_body={"top_k": 20, "chat_template_kwargs": {"enable_thinking": True}},
)
print(out.choices[0].message.content)
```
The output looks like this (illustrative):
```text
I can see the Google home page with an empty search box in the center. I have not searched yet.
My plan: 1. Click the search box. 2. Type "KAIST". 3. Press Enter and open the official site.
500
427
```
Coordinates in the tool call are normalized to 0–1000. Convert them to pixels with `x / 1000 × width` and `y / 1000 × height`.
## How It Works
```
user task + observation (tabs, url, screenshot)
assistant what I see · what is done · plan one action
tool action result + new observation (newest screenshot only)
… repeat until finished(answer)
```
- **Action space:** 22 browser tools: click, type, hotkey, scroll, drag, goto, tab control, `finished`, and others ([`tools.json`](tools.json)).
- **Memory:** the full text history of thoughts, actions and observations is kept. Only the current screenshot is sent.
- **Recommended settings:** temperature 0.6, top_p 0.95, top_k 20, thinking on, up to 16k tokens per step and 50 steps. These are already set in `generation_config.json`.
The exact message format, parsing rules and feedback strings are in **[HARNESS.md](HARNESS.md)**. Follow it if you build your own harness.
## Evaluation Details
| | |
|---|---|
| Tasks | Online-Mind2Web: 300 English, 312 Korean |
| Judge | GPT-6 |
## Files
| File | Description |
|---|---|
| `chat_template.jinja` | Chat template (Qwen3.5 XML tool calls, thinking on) |
| `system_prompt.md` · `tools.json` | System prompt and 22-tool schema, verbatim from training |
| `HARNESS.md` | Full harness specification |
| `examples/agent_playwright.py` | Reference browser agent (Playwright + OpenAI-compatible API). It does not execute `upload`. |
| `config.json` · `generation_config.json` · `processor_config.json` · `tokenizer*.json` | Model, sampling and processor configs |
## 한국어 요약
**KORMo-Web-4B**는 사람처럼 웹 화면을 보고 클릭·입력·탐색을 수행하는 **4B 규모의 한영 시각 기반 웹 에이전트**입니다.
- **대규모 웹 사전학습:** 1,000만 건 이상의 한영 웹 학습 샘플로 UI 요소의 위치·의미·구조·기능 등을 세밀하게 학습(Atomic-Level Pretraining)
- **실제 웹 행동 학습:** 10만 건 이상의 한국인 한영 웹 사용 궤적을 학습해 여러 웹사이트와 탭을 넘나드는 복잡한 작업 수행
- **뛰어난 성능:** Online-Mind2Web에서 영어·한국어 모두 **42.3% 성공률**, 비교한 공개 모델 중 최고 성능(기존 최고 대비 +9.6%p / +22.1%p)
- **시각 중심 웹 조작:** 별도의 DOM 정보 없이 스크린샷만으로 웹을 이해하고, 행동 결과를 확인하며 다음 작업 수행
- **모델·데이터 공개:** Apache-2.0 모델 가중치, 대규모 학습 데이터, 웹 사용 궤적, 평가 벤치마크 및 실행 코드 공개
## Acknowledgements
Built on **[Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B)**. Evaluated on **Online-Mind2Web** and its Korean counterpart.