| --- |
| base_model: Qwen/Qwen3.5-4B |
| library_name: peft |
| license: apache-2.0 |
| pipeline_tag: image-text-to-text |
| datasets: |
| - sparklabuofu/MiniWeb |
| tags: |
| - lora |
| - web-agents |
| - gui-agents |
| - browser-use |
| - miniweb |
| --- |
| |
| # MiniWebAgent executor (Qwen3.5-4B LoRA) |
|
|
| The executor of **MiniWebAgent**: a LoRA adapter on [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) trained on |
| MiniWebGen's practice data, so the model performs single web interaction skills (primitives) such as filling a form, |
| setting a slider, drawing a signature or uploading a file. In MiniWebAgent the untrained Qwen3.5-4B is the planner: it |
| splits a task into primitive-level instructions and hands each one to this executor, which acts in the same browser; |
| reasoning steps that only read the page stay with the base model. |
|
|
| Code, benchmark and setup: [github.com/sparklabutah/MiniWeb](https://github.com/sparklabutah/MiniWeb). |
| Data: [sparklabuofu/MiniWeb](https://huggingface.co/datasets/sparklabuofu/MiniWeb). |
|
|
| ## Results |
|
|
| Task success (%) of Qwen3.5-4B (the paper's Table 3). MiniWeb: the 62 human tasks of the 13 held-out sites, from which |
| no training data comes; the other three benchmarks are not trained on either. |
|
|
| | | MiniWeb held-out | WebArena-Lite | WebVoyager | Online-Mind2Web | |
| |---|---|---|---|---| |
| | Untrained single agent | 38.7 | 27.1 | 45.5 | 17.9 | |
| | Untrained planner + untrained executor | 45.2 | 30.7 | 59.9 | 21.8 | |
| | **MiniWebAgent** (untrained planner + this executor) | **58.1** | **36.4** | **65.5** | **33.9** | |
|
|
| On the 103 generated single-primitive tasks of the held-out sites, the trained model as a single agent goes from 55.3% |
| to 80.3%, with gains in every skill family. On whole tasks it needs the planner: trained mostly on one-primitive |
| episodes, the single agent often stops before reaching the skills it learned (39.2% on the held-out tasks, 38.7% untrained). |
|
|
| ## Use |
|
|
| Serve the base model with the adapter (vLLM), then run MiniWebAgent with the repository's evaluator: |
|
|
| ```bash |
| python scripts/fetch_adapters.py # -> data/webmix/adapters/pooled_v4 |
| MODEL=Qwen/Qwen3.5-4B SERVED=qwen35-4b PORT=8400 MAX_LORA_RANK=64 \ |
| LORA_MODULES=pooled_v4=data/webmix/adapters/pooled_v4 bash src/miniwebagent/scripts/serve.sh |
| python -m miniwebagent.evaluate --human --arm agent_planner --agent-version v6 --planner-name qwen35-4b \ |
| --adapters none --fallback pooled_v4 --vllm http://127.0.0.1:8400/v1 --ports 8300,8301 --out runs/miniwebagent |
| ``` |
|
|
| With PEFT directly: `PeftModel.from_pretrained(base_model, "sparklabuofu/MiniWebAgent")`. |
|
|
| The executor expects the [browser-use](https://github.com/browser-use/browser-use) agent prompt it was trained on (the |
| system prompt, the task, the agent's memory, the indexed interactive elements and the screenshot) and answers with |
| its memory and one action. Thinking is off: the chat template renders an empty `<think></think>` block. |
|
|
| ## Training |
|
|
| - **Data**: 27,646 steps of MiniWebGen (the dataset's `training_steps` table, rows with `is_target` and not |
| `validation`): 3,128 verified single-primitive trajectories on the 52 training sites, 857 chains of two trajectories |
| and 629 trajectories with an injected and corrected mistake, replayed through the agent harness with the base model |
| writing each step's memory. |
| - **Recipe**: rank-64 LoRA (alpha 128, dropout 0.05) on the language model's attention projections (full-attention |
| layers) and MLP projections (every layer); no vision tower. One epoch, AdamW, learning rate 1e-4 with 3% warmup and a |
| cosine decay, 8 steps per optimizer update, cross-entropy on the whole target (memory and action); 4 GPUs, about 5.5 |
| hours. `training_meta.json` and `train_log.jsonl` hold the settings and the loss curve. |
|
|
| ## Other checkpoints |
|
|
| The paper's other trained executors, in `ablations/` (same base model and LoRA shape; each folder has its own |
| `training_meta.json`). `python scripts/fetch_adapters.py --all` downloads them under the names the paper's run commands |
| use (`scripts/reproduce.py show <arm>`). |
|
|
| | Folder | What it is | Held-out task success | |
| |---|---|---| |
| | (root) | MiniWebAgent's executor | 58.1 | |
| | `ablations/task_centric` | trained at matched size on task-centric trajectories: InSTA's pipeline reimplemented on the 52 training sites (the dataset's `task_centric_steps`) | 47.8 | |
| | `ablations/specialists/{drag,io,select,rest,reasoning}` | one specialist per skill family group (supplementary material): `drag`, `io`, `select` and `rest` start from the main executor and train on their families' steps with a KL penalty toward it; `reasoning` starts from the base model | 52.7 | |
| | `ablations/no_done` | the main recipe without each episode's final `done` step, a control for stopping early (single agent) | 34.4 | |
|
|
| All numbers are with the untrained planner except `no_done` (single agent; 39.2 with the `done` steps). The specialists |
| are served under their family names and picked by the planner's family for each subtask: |
| `LORA_MODULES='drag=.../drag io=.../io select=.../select rest=.../rest reasoning=.../reasoning pooled_v4=...'` and |
| `--adapters drag,io,select,rest,reasoning --fallback pooled_v4`. |
|
|
| ## Limitations |
|
|
| Trained only on MiniWeb's synthetic sites and English instructions, mostly on single primitives: on whole tasks it |
| relies on the planner for their structure. Outside MiniWeb it has been evaluated only on the three benchmarks above. |
|
|
| ## License |
|
|
| Apache 2.0, as the base model. |
|
|
