Instructions to use KORMo-VL/KORMo-Web-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use KORMo-VL/KORMo-Web-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="KORMo-VL/KORMo-Web-4B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("KORMo-VL/KORMo-Web-4B")
model = AutoModelForMultimodalLM.from_pretrained("KORMo-VL/KORMo-Web-4B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use KORMo-VL/KORMo-Web-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "KORMo-VL/KORMo-Web-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KORMo-VL/KORMo-Web-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/KORMo-VL/KORMo-Web-4B

SGLang

How to use KORMo-VL/KORMo-Web-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "KORMo-VL/KORMo-Web-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KORMo-VL/KORMo-Web-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "KORMo-VL/KORMo-Web-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KORMo-VL/KORMo-Web-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use KORMo-VL/KORMo-Web-4B with Docker Model Runner:
```
docker model run hf.co/KORMo-VL/KORMo-Web-4B
```

JLee0 commited on about 3 hours ago

Commit

14ec22a

verified ·

1 Parent(s): 1c233c5

Add model card

Browse files

Files changed (1) hide show

README.md +36 -0

README.md ADDED Viewed

	@@ -0,0 +1,36 @@

+---
+license: other
+base_model: Qwen/Qwen3.5-4B
+tags:
+- gui-agent
+- web-agent
+---
+# KORMo-Web-4B
+Trained checkpoint: `qwen35-4b-final-rw3b-s2-len90k-3ep-4n`.
+A GUI web agent: a Qwen3.5-4B model, pretrained on 3M GUI samples (final checkpoint), then fully fine-tuned for 3 epochs on Luna trajectory sets. For trajectories whose task was rewritten, half use the rewritten task and half keep the original task (one copy per trajectory). In this data, the first line of each tool-result turn is the real execution result (for example `Succeed: click on <div> role=textbox "..." at (x, y) executed.`) instead of a short `Succeed: click executed.` line.
+- **Training data**: 29,770 trajectory steps.
+  - Luna EN rollouts: 6,224 steps.
+  - Luna EN login-site rollouts: 3,463 steps.
+  - Luna EN AI-task rollouts: 2,838 steps.
+  - Luna KO rollouts with one screenshot per step: 8,018 steps.
+  - Luna KO login-site rollouts: 4,520 steps.
+  - Luna KO login-site rollouts (model-generated): 1,426 steps.
+  - Luna KO AI-task rollouts: 2,683 steps.
+  - Luna KO mail-site rollouts: 354 steps.
+  - Luna KO KAIST-site rollouts: 244 steps.
+- **Training setup**: 3 epochs (1,398 steps, batch 64), LR 1e-5 for the LLM and projector, 2e-6 for vision, cosine schedule, warmup 42, max sequence length 90,000. Samples are grouped by length into rounds of 32, and the rounds are shuffled.
+- **Tools**: 22 tools (click, double_click, right_click, type, hotkey, scroll, wait, finished, fail, unknown, go_back, go_forward, goto, hover, drag, upload, focus, clear, EXIT, new_tab, switch_tab, close_tab). The system prompt says "Use only the 22 tools".
+- **Output format**: `<think>` reasoning followed by an XML `<tool_call>`.
+- **Inference note**: use the same 22-tool list and system prompt, and feed the real execution result as the first line of each tool-result turn, matching the training data. Coordinates in that line use the model's 0-1000 scale.
+Evaluated with the same 22-tool list and system prompt as training, starting from each task's benchmark start URL. OM2W counts missing tasks as failures (EN /300, KO /312).
+| ScreenSpot-v2 EN | ScreenSpot-v2 KO | OM2W EN | OM2W KO |
+|---|---|---|---|
+| 78.5% | 77.4% | 42.3% | 42.3% |
+OM2W EN is from a second run (play2) in which missing tasks were retried up to three times and blocked tasks were retried on another network exit; 12 tasks stayed blocked and 2 had no result, and all 14 count as failures. The first run scored 37.7%.