Slot reading · GUI-Decisions

Computer-use steps, read in one forward pass

Give it a screenshot and an instruction. Instead of writing {"action": "click", "point": [412, 288]} token by token, the model fills a fixed template of slots, and every answer is read from one forward pass: the action, and the point as a 256-bin distribution over x and over y. Nothing is decoded; the heatmap is that distribution.

blog post · code · GUI-Decisions-31B-LoRA · GUI-Decisions-31B · GUI-Decisions-2B-LoRA