README.md

metadata

title: GroundGUI GUI Grounding
emoji: 🖱️
colorFrom: pink
colorTo: blue
sdk: gradio
sdk_version: 6.30.0
app_file: app.py
short_description: Ground click coordinates on any GUI screenshot.
python_version: '3.12'
startup_duration_timeout: 1h
license: apache-2.0
models:
  - PrentisAI/GroundGUI-8B

GroundGUI · GUI Grounding

This demo runs PrentisAI/GroundGUI-8B, an 8B GUI-grounding vision-language model fine-tuned from Qwen3-VL-8B-Instruct via high-quality data curation and distillation-to-reinforcement learning.

Give it a GUI screenshot plus a short instruction, and it emits the click action as a tool call with coordinates in a normalized 0–1000 space — {"name": "computer_use", "arguments": {"action": "left_click", "coordinate": [x, y]}}. The demo maps the coordinates back onto your original image and marks the predicted click point with a crosshair.

The system prompt is the exact frozen per-platform grounding prompt the model was trained with (scalecua_toolcall): pick desktop, web, or mobile to serve the matching tool schema (computer_use / browser_use / mobile_use). Inference is greedy (do_sample=False), matching the paper's evaluation.

• Paper: GroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learning
• Code & data: github.com/PrentisAI/GroundGUI
• Model: PrentisAI/GroundGUI-8B

Benchmark results (from the paper): 95.2 ScreenSpot-v2 · 63.9 ScreenSpot-Pro · 86.8 MMBench-GUI L2 · 38.3 UI-Vision · 65.4 OSWorld-G.

Example screenshots

The bundled example screenshots come from likaixin/ScreenSpot-Pro (MIT license), a professional high-resolution GUI grounding benchmark.