KORMo-Web-4B

KORMo-Web-4B is a 4B vision-language agent that completes real browser tasks from screenshots alone. Give it a request in Korean or English; it plans, clicks, types and moves between sites and tabs until the task is done. In the same live-web evaluation, it beats every open-weight web agent we compared, by +9.6 points in English and +22.1 points in Korean.

On Online-Mind2Web it reaches 42.3% in English and 42.3% in Korean, which is +22.3 / +28.8 points over the base Qwen3.5-4B.

What Makes KORMo-Web Different?

  • 10M+ Atomic-Level Web Pretraining Samples. Large-scale Korean and English web understanding pretraining teaches the model what individual UI elements are, where they are, and what they doโ€”covering text, location, appearance, layout, function, and state.
  • 100K+ Web Interaction Trajectories. Bilingual browser demonstrations teach the model to translate fine-grained web understanding into real-world, multi-step actions.
  • Open Models and Data. Releasing model weights, large-scale bilingual web pretraining data, interaction trajectories, and evaluation benchmarks to support reproducible web agent research.

Performance

All models below ran the same 612 live-website tasks (300 English, 312 Korean), each with its own prompt and action format, and were scored by the same LLM judge.

Model Size Online-Mind2Web EN Online-Mind2Web KO
Qwen3.5-4B (base, no fine-tuning) 4B 20.0 13.5
Fara-1.5-4B 4B 20.7 7.7
UI-TARS-1.5-7B 7B 22.3 20.2
OpenWebRL-4B 4B 32.7 15.4
KORMo-Web-4B (ours) 4B 42.3 42.3
KORMo-Web-9B (ours) 9B 51.0 48.1

Success rate (%). The denominator is always the full task count, so crashed or empty runs count as failures.

KORMo-Web-4B outperforms open web agents of similar size in both English and Korean.

Why KORMo-Web

  • Korean and English, natural requests. Handles short commands as well as long, conversational instructions in either language.
  • Screenshot-only. It needs no DOM or accessibility tree; it acts on what is on screen, like a person, and checks each action's result on the next screenshot.
  • Thinks, then acts. A short reasoning step precedes every action, so its behaviour is easy to inspect and debug.
  • Multi-site, multi-tab workflows. It can search on one site, verify on another and write the result into a third, managing tabs on its own.
  • Open and ready to run. Apache-2.0 weights, a reference browser harness, the tool schema and the exact inference settings are all included.

Quickstart

This repository holds the 4-bit MLX weights of KORMo-Web-4B for Apple Silicon Macs. Serve it with vllm-mlx (0.6.0 or later), which exposes the same OpenAI-compatible API as vLLM.

Run the agent

1. Install and download

pip install -U "vllm-mlx>=0.6.0" openai playwright huggingface_hub
playwright install chromium
huggingface-cli download KORMo-VL/KORMo-Web-4B-mlx-4bit --local-dir KORMo-Web-4B-mlx-4bit
# system prompt, tool schema and reference harness (shared with the full-precision repo)
huggingface-cli download KORMo-VL/KORMo-Web-4B system_prompt.md tools.json HARNESS.md examples/agent_playwright.py \
  --local-dir KORMo-Web-4B-mlx-4bit

2. Start the model server

vllm-mlx serve ./KORMo-Web-4B-mlx-4bit --served-model-name KORMo-Web-4B --mllm --port 8000

Do not enable --enable-auto-tool-choice or --reasoning-parser: the harness parses the raw <think> and <tool_call> text from the message content.

3. Run a browser task

python KORMo-Web-4B-mlx-4bit/examples/agent_playwright.py --model KORMo-Web-4B \
  --endpoint http://localhost:8000/v1 \
  --task "๋‹ค๋ฐฉ์—์„œ ์›”ํ‰์—ญ ๊ทผ์ฒ˜ ๋ณด์ฆ๊ธˆ ์ฒœ๋งŒ์› ์ดํ•˜๋กœ ์„ค์ •ํ•œ๋’ค ์›๋ฃธ 3๊ฐœ์ •๋„ ์ฐพ์•„์„œ ์‚ดํŽด๋ณด๊ณ  ๋‹คํ•˜๋ฉด ์ฐพ์€ ๋‚ด์šฉ์„ ์นด์ด์ŠคํŠธ ๋ฉ”์ผ๋กœ mlplab@kaist.ac.kr ํ•œํ…Œ ์ •๋ฆฌํ•ด์„œ ๋ณด๋‚ด์ค˜"

The agent opens Chromium, prints each action as it goes, and finishes with the model's answer.

Inference example (one step)

Send one screenshot with the request and get back one action:

import base64, json
from openai import OpenAI

SYSTEM = open("KORMo-Web-4B-mlx-4bit/system_prompt.md").read().strip()
TOOLS = json.load(open("KORMo-Web-4B-mlx-4bit/tools.json"))
shot = base64.b64encode(open("screen.png", "rb").read()).decode()

obs = ("<observation>\nscreen size: 1280 x 720\nCurrent tab: 0\nAvailable tabs:\n"
       "- Tab 0 (active): https://www.google.com/\n\nurl: https://www.google.com/\nscreenshot:\n")
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": [
        {"type": "text", "text": "<user_request>\n๊ตฌ๊ธ€์—์„œ KAIST ๋ฅผ ๊ฒ€์ƒ‰ํ•ด์„œ ๊ณต์‹ ํ™ˆํŽ˜์ด์ง€๋ฅผ ์—ด์–ด์ค˜.\n</user_request>\n" + obs},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{shot}"}},
        {"type": "text", "text": "\n</observation>"},
    ]},
]
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")  # vllm-mlx server
out = client.chat.completions.create(
    model="KORMo-Web-4B", messages=messages, tools=TOOLS,
    temperature=0.6, top_p=0.95, max_tokens=16384,
    extra_body={"top_k": 20, "chat_template_kwargs": {"enable_thinking": True}},
)
print(out.choices[0].message.content)

The output looks like this (illustrative):

<think>
I can see the Google home page with an empty search box in the center. I have not searched yet.
My plan: 1. Click the search box. 2. Type "KAIST". 3. Press Enter and open the official site.
</think>

<tool_call>
<function=click>
<parameter=x>
500
</parameter>
<parameter=y>
427
</parameter>
</function>
</tool_call>

Coordinates in the tool call are normalized to 0โ€“1000. Convert them to pixels with x / 1000 ร— width and y / 1000 ร— height.

How It Works

user       <user_request> task </user_request>  + observation (tabs, url, screenshot)
assistant  <think> what I see ยท what is done ยท plan </think>  <tool_call> one action </tool_call>
tool       action result  + new observation (newest screenshot only)
โ€ฆ          repeat until finished(answer)
  • Action space: 22 browser tools: click, type, hotkey, scroll, drag, goto, tab control, finished, and others (tools.json).
  • Memory: the full text history of thoughts, actions and observations is kept. Only the current screenshot is sent.
  • Recommended settings: temperature 0.6, top_p 0.95, top_k 20, thinking on, up to 16k tokens per step and 50 steps. These are already set in generation_config.json.

The exact message format, parsing rules and feedback strings are in HARNESS.md. Follow it if you build your own harness.

Evaluation Details

Tasks Online-Mind2Web: 300 English, 312 Korean
Judge GPT-6

Files

File Description
chat_template.jinja Chat template (Qwen3.5 XML tool calls, thinking on)
system_prompt.md ยท tools.json System prompt and 22-tool schema, verbatim from training
HARNESS.md Full harness specification
examples/agent_playwright.py Reference browser agent (Playwright + OpenAI-compatible API). It does not execute upload.
config.json ยท generation_config.json ยท processor_config.json ยท tokenizer*.json Model, sampling and processor configs

ํ•œ๊ตญ์–ด ์š”์•ฝ

KORMo-Web-4B๋Š” ์‚ฌ๋žŒ์ฒ˜๋Ÿผ ์›น ํ™”๋ฉด์„ ๋ณด๊ณ  ํด๋ฆญยท์ž…๋ ฅยทํƒ์ƒ‰์„ ์ˆ˜ํ–‰ํ•˜๋Š” 4B ๊ทœ๋ชจ์˜ ํ•œ์˜ ์‹œ๊ฐ ๊ธฐ๋ฐ˜ ์›น ์—์ด์ „ํŠธ์ž…๋‹ˆ๋‹ค.

  • ๋Œ€๊ทœ๋ชจ ์›น ์‚ฌ์ „ํ•™์Šต: 1,000๋งŒ ๊ฑด ์ด์ƒ์˜ ํ•œ์˜ ์›น ํ•™์Šต ์ƒ˜ํ”Œ๋กœ UI ์š”์†Œ์˜ ์œ„์น˜ยท์˜๋ฏธยท๊ตฌ์กฐยท๊ธฐ๋Šฅ ๋“ฑ์„ ์„ธ๋ฐ€ํ•˜๊ฒŒ ํ•™์Šต(Atomic-Level Pretraining)
  • ์‹ค์ œ ์›น ํ–‰๋™ ํ•™์Šต: 10๋งŒ ๊ฑด ์ด์ƒ์˜ ํ•œ๊ตญ์ธ ํ•œ์˜ ์›น ์‚ฌ์šฉ ๊ถค์ ์„ ํ•™์Šตํ•ด ์—ฌ๋Ÿฌ ์›น์‚ฌ์ดํŠธ์™€ ํƒญ์„ ๋„˜๋‚˜๋“œ๋Š” ๋ณต์žกํ•œ ์ž‘์—… ์ˆ˜ํ–‰
  • ๋›ฐ์–ด๋‚œ ์„ฑ๋Šฅ: Online-Mind2Web์—์„œ ์˜์–ดยทํ•œ๊ตญ์–ด ๋ชจ๋‘ 42.3% ์„ฑ๊ณต๋ฅ , ๋น„๊ตํ•œ ๊ณต๊ฐœ ๋ชจ๋ธ ์ค‘ ์ตœ๊ณ  ์„ฑ๋Šฅ(๊ธฐ์กด ์ตœ๊ณ  ๋Œ€๋น„ +9.6%p / +22.1%p)
  • ์‹œ๊ฐ ์ค‘์‹ฌ ์›น ์กฐ์ž‘: ๋ณ„๋„์˜ DOM ์ •๋ณด ์—†์ด ์Šคํฌ๋ฆฐ์ƒท๋งŒ์œผ๋กœ ์›น์„ ์ดํ•ดํ•˜๊ณ , ํ–‰๋™ ๊ฒฐ๊ณผ๋ฅผ ํ™•์ธํ•˜๋ฉฐ ๋‹ค์Œ ์ž‘์—… ์ˆ˜ํ–‰
  • ๋ชจ๋ธยท๋ฐ์ดํ„ฐ ๊ณต๊ฐœ: Apache-2.0 ๋ชจ๋ธ ๊ฐ€์ค‘์น˜, ๋Œ€๊ทœ๋ชจ ํ•™์Šต ๋ฐ์ดํ„ฐ, ์›น ์‚ฌ์šฉ ๊ถค์ , ํ‰๊ฐ€ ๋ฒค์น˜๋งˆํฌ ๋ฐ ์‹คํ–‰ ์ฝ”๋“œ ๊ณต๊ฐœ

Acknowledgements

Built on Qwen3.5-4B. Evaluated on Online-Mind2Web and its Korean counterpart.

Downloads last month
-
Safetensors
Model size
5B params
Tensor type
U32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for KORMo-VL/KORMo-Web-4B-mlx-4bit

Finetuned
Qwen/Qwen3.5-4B
Quantized
(1)
this model