# GUI-Shift Implementation

Implementation of **GUI-Shift: Enhancing VLM-Based GUI Agents through Self-supervised Reinforcement Learning** (arXiv:2505.12493).

## Overview

This repo provides a complete training pipeline for GUI-Shift using native TRL GRPOTrainer with Qwen2.5-VL.

## Key Components

- `build_data.py`: Constructs K-step GUI Transition dataset from AndroidControl
- `train_gui_shift.py`: GRPO training with rule-based rewards (format + action correctness)
- `run_train.sh`: Launch script with hyperparameters matching the paper

## Workflow

1. **Build dataset**: `python build_data.py`
   - Downloads AndroidControl (ckg/AndroidControlParsedWithImages-20k)
   - Extracts K-step state pairs (default k=1)
   - Saves 2000 transition samples as JSONL

2. **Train**: `bash run_train.sh`
   - Loads Qwen2.5-VL-7B-Instruct
   - Freezes vision encoder + projector
   - Trains with GRPO (format_reward + action_reward)
   - Pushes to Hugging Face Hub

## Hyperparameters (from paper Appendix A)

| Parameter | Value |
|---|---|
| learning_rate | 1e-6 |
| num_generations | 8 |
| num_train_epochs | 4 |
| max_prompt_length | 1024 |
| max_completion_length | 256 |
| per_device_train_batch_size | 2 |
| gradient_accumulation_steps | 8 |
| beta (KL) | 0.04 |
| epsilon (clip) | 0.2 |
| temperature | 0.9 |
| bf16 | true |

## Action Space

The unified action space covers 8 types:
- `click`, `long_press` (with x,y coordinates)
- `scroll` (with direction)
- `open_app` (with app_name)
- `input_text` (with text content)
- `navigate_back`, `navigate_home`, `wait`

## Reward Design

- **Format reward**: +1 if output contains `<answer>...</answer>` tags
- **Action reward**: +1 if parsed action matches ground truth
  - Click/long_press: point falls within ground-truth bounding box
  - Scroll/open_app/input_text: exact parameter match
  - navigate_back/navigate_home/wait: action type match only

## References

- Paper: https://arxiv.org/abs/2505.12493
- Dataset: ckg/AndroidControlParsedWithImages-20k
- Base Model: Qwen/Qwen2.5-VL-7B-Instruct