File size: 2,054 Bytes
8f96ece | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | # GUI-Shift Implementation
Implementation of **GUI-Shift: Enhancing VLM-Based GUI Agents through Self-supervised Reinforcement Learning** (arXiv:2505.12493).
## Overview
This repo provides a complete training pipeline for GUI-Shift using native TRL GRPOTrainer with Qwen2.5-VL.
## Key Components
- `build_data.py`: Constructs K-step GUI Transition dataset from AndroidControl
- `train_gui_shift.py`: GRPO training with rule-based rewards (format + action correctness)
- `run_train.sh`: Launch script with hyperparameters matching the paper
## Workflow
1. **Build dataset**: `python build_data.py`
- Downloads AndroidControl (ckg/AndroidControlParsedWithImages-20k)
- Extracts K-step state pairs (default k=1)
- Saves 2000 transition samples as JSONL
2. **Train**: `bash run_train.sh`
- Loads Qwen2.5-VL-7B-Instruct
- Freezes vision encoder + projector
- Trains with GRPO (format_reward + action_reward)
- Pushes to Hugging Face Hub
## Hyperparameters (from paper Appendix A)
| Parameter | Value |
|---|---|
| learning_rate | 1e-6 |
| num_generations | 8 |
| num_train_epochs | 4 |
| max_prompt_length | 1024 |
| max_completion_length | 256 |
| per_device_train_batch_size | 2 |
| gradient_accumulation_steps | 8 |
| beta (KL) | 0.04 |
| epsilon (clip) | 0.2 |
| temperature | 0.9 |
| bf16 | true |
## Action Space
The unified action space covers 8 types:
- `click`, `long_press` (with x,y coordinates)
- `scroll` (with direction)
- `open_app` (with app_name)
- `input_text` (with text content)
- `navigate_back`, `navigate_home`, `wait`
## Reward Design
- **Format reward**: +1 if output contains `<answer>...</answer>` tags
- **Action reward**: +1 if parsed action matches ground truth
- Click/long_press: point falls within ground-truth bounding box
- Scroll/open_app/input_text: exact parameter match
- navigate_back/navigate_home/wait: action type match only
## References
- Paper: https://arxiv.org/abs/2505.12493
- Dataset: ckg/AndroidControlParsedWithImages-20k
- Base Model: Qwen/Qwen2.5-VL-7B-Instruct
|