KORMo-Web-10B

KORMo-Web-10B is a Korean-first web agent built entirely on the KORMo model family, which KAIST MLP Lab trains from scratch. It reads a screenshot, thinks, and takes one browser action at a time (click, type, scroll, move between tabs and sites) until the task is done.

Among the open web agents we compared, it scores the highest in Korean (30.1%) on Online-Mind2Web, and 29.0% in English.

From a Korean LLM to a Korean web agent

KORMo-Web-10B is the third step of one Korean lineage, not a fine-tune of a foreign base model:

Step	Model	What it adds
1	KORMo-10B	Korean–English LLM trained from scratch, with its own Korean tokenizer
2	KORMo-VL	Vision: SigLIP encoder + KORMo-10B, trained on Korean document and UI images
3	KORMo-Web-10B	Web agent: large-scale web understanding data and browser trajectories, in Korean and English

Because every stage was trained with Korean as a first language, the model handles Korean pages, Korean UI terms and natural Korean requests directly.

Performance

All models below ran the same 612 live-website tasks (300 English, 312 Korean), each with its own prompt and action format, and were scored by the same LLM judge.

Model	Size	Online-Mind2Web EN	Online-Mind2Web KO
Qwen3.5-9B (base, no fine-tuning)	9B	24.7	16.7
Fara-1.5-4B	4B	20.7	7.7
UI-TARS-1.5-7B	7B	22.3	20.2
OpenWebRL-4B	4B	32.7	15.4
KORMo-Web-10B (ours)	11B	29.0	30.1
GPT-6-luna (proprietary)	–	63.0	65.7
Gemini-3.8-flash (proprietary)	–	74.0	80.1

Success rate (%). The denominator is always the full task count, so crashed or empty runs count as failures.

KORMo-Web-10B has the best Korean score among the open web agents above, by about 10 points.

Why KORMo-Web-10B

• Korean from the ground up. Built on a Korean LLM trained from scratch, not adapted from an English-first model.
• Korean and English, natural requests. Handles short commands as well as long, conversational instructions in either language.
• Screenshot-only. It needs no DOM or accessibility tree; it acts on what is on screen and checks each action's result on the next screenshot.
• Thinks, then acts. A short reasoning step precedes every action, so its behaviour is easy to inspect.
• Open and ready to run. Weights, model code for transformers 5, a vLLM path, a reference browser harness and the exact inference settings are all included.

Quickstart

Tested with transformers 5.12.1 and vLLM 0.23.0. The model runs on a single GPU.

transformers does not ship the KORMo text model, so this repo includes it (_modeling_kormo.py) together with a small loader, kormo_load.py:

• kormo_load.register(path) registers the KORMo text model. Call it before loading the processor or the model.
• kormo_load.patch_vision(model) feeds the SigLIP features after post-layernorm to the projector, as in training. Without it the model still runs but misreads screenshots.

Inference example (one step, transformers)

pip install -U "transformers>=5.12" torch pillow huggingface_hub
huggingface-cli download KORMo-VL/KORMo-Web-10B --local-dir KORMo-Web-10B

import json, sys, torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor

M = "KORMo-Web-10B"
sys.path.insert(0, M)
import kormo_load
kormo_load.register(M)
processor = AutoProcessor.from_pretrained(M)
model = AutoModelForImageTextToText.from_pretrained(M, dtype=torch.bfloat16, device_map="cuda").eval()
kormo_load.patch_vision(model)

SYSTEM = open(f"{M}/system_prompt.md").read().strip()
TOOLS = json.load(open(f"{M}/tools.json"))
obs = ("<observation>\nscreen size: 1280 x 720\nCurrent tab: 0\nAvailable tabs:\n"
       "- Tab 0 (active): https://www.google.com/\n\nurl: https://www.google.com/\nscreenshot:\n")
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": [
        {"type": "text", "text": "<user_request>\n구글에서 KAIST 를 검색해서 공식 홈페이지를 열어줘.\n</user_request>\n" + obs},
        {"type": "image", "image": Image.open("screen.png").convert("RGB")},
        {"type": "text", "text": "\n</observation>"},
    ]},
]
inputs = processor.apply_chat_template(messages, tools=TOOLS, add_generation_prompt=True,
                                       tokenize=True, return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=16384, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False))

The output is a short <think> block followed by one tool call:

<think>
I can see the Google home page with an empty search box. I have not searched yet.
My plan: 1. Click the search box. 2. Type "KAIST". 3. Press Enter and open the official site.
</think>

<tool_call>
<function=click>
<parameter=x>
500
</parameter>
<parameter=y>
427
</parameter>
</function>
</tool_call>

Coordinates are normalized to 0–1000. Convert them to pixels with x / 1000 × width and y / 1000 × height.

Run the agent (vLLM)

vLLM does not include the KORMo architecture either. The KORMo text layers have the Llama structure, so vllm/convert_to_vllm.py renames two parameter names and writes a folder vLLM loads natively. vllm/sitecustomize.py turns SigLIP post-layernorm back on inside vLLM.

1. Convert (once)

pip install -U "vllm>=0.23.0" openai playwright safetensors
playwright install chromium
python KORMo-Web-10B/vllm/convert_to_vllm.py KORMo-Web-10B KORMo-Web-10B-vllm

2. Start the model server

PYTHONPATH=$PWD/KORMo-Web-10B/vllm vllm serve ./KORMo-Web-10B-vllm --served-model-name KORMo-Web-10B \
  --max-model-len 65536 --limit-mm-per-prompt '{"image":1,"video":0}'

Check that the server log prints [kormo-vllm] SigLIP post_layernorm enabled.

3. Run a browser task

python KORMo-Web-10B/examples/agent_playwright.py --model KORMo-Web-10B \
  --task "다나와에서 50만 원 이하, 10인치 이상 태블릿 3개를 찾아서 가격을 알려줘"

The agent opens Chromium, prints each action as it goes, and finishes with the model's answer.

How It Works

user       <user_request> task </user_request>  + observation (tabs, url, screenshot)
assistant  <think> what I see · what is done · plan </think>  <tool_call> one action </tool_call>
tool       action result  + new observation (newest screenshot only)
…          repeat until finished(answer)

• Action space: 22 browser tools: click, type, hotkey, scroll, drag, goto, tab control, finished, and others (tools.json).
• Memory: the full text history of thoughts, actions and observations is kept. Only the current screenshot is sent.
• Recommended settings: temperature 0.6, top_p 0.95, top_k 20, thinking on, up to 16k tokens per step and 50 steps. These are already set in generation_config.json.

The exact message format, parsing rules and feedback strings are in HARNESS.md. Follow it if you build your own harness.

Evaluation Details

Tasks	Online-Mind2Web: 300 English, 312 Korean
Judge	GPT-6

Files

File	Description
model.safetensors · config.json	Weights and config (LLaVA-OneVision: SigLIP + KORMo-10B text model)
_configuration_kormo.py · _modeling_kormo.py	KORMo text model code for transformers 5
kormo_load.py	register() and patch_vision()
chat_template.jinja	Chat template (XML tool calls, thinking on)
system_prompt.md · tools.json	System prompt and 22-tool schema
HARNESS.md	Full harness specification
examples/agent_playwright.py	Reference browser agent (Playwright + OpenAI-compatible API). It does not execute upload.
vllm/convert_to_vllm.py · vllm/sitecustomize.py	Convert for vLLM and the post-layernorm fix
generation_config.json · *processor_config.json · tokenizer*.json · special_tokens_map.json	Sampling, processor and tokenizer configs

한국어 요약

KORMo-Web-10B는 KAIST MLP 연구실이 처음부터 학습한 한국어 모델 KORMo-10B → KORMo-VL을 이어 만든 한국어 우선 웹 에이전트입니다. 스크린샷만 보고 한국어·영어 웹 작업을 수행합니다.

• Online-Mind2Web: EN 29.0, KO 30.1
• 비교한 공개 웹 에이전트 중 한국어 점수가 가장 높습니다.

Acknowledgements

Built on KORMo-VL and KORMo-10B by KAIST MLP Lab. Evaluated on Online-Mind2Web and its Korean counterpart.