File size: 2,083 Bytes
aa4bed2 321971c aa4bed2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | ---
language: en
library_name: pytorch
license: apache-2.0
tags:
- browser-automation
- web-agents
- decision-model
- mind2web
- laya
base_model: convaiinnovations/laya
---
# Laya Fine-Tuned for Browser Automation (Mind2Web)
This is a fine-tuned version of the [Laya](https://huggingface.co/convaiinnovations/laya) decision model, specialized for browser automation tasks. It predicts the next browser action (operation and target element) from a web page's DOM structure and a user goal.
## Model Details
- **Developed by:** Shaun Andrade ([@ShaunSpark](https://huggingface.co/ShaunSpark))
- **Base model:** [convaiinnovations/laya](https://huggingface.co/convaiinnovations/laya) (421M parameters)
- **Training data:** [Mind2Web](https://huggingface.co/datasets/osunlp/Mind2Web) (671 usable training examples)
- **Validation accuracy:** 74.3% on a held-out set of 68 examples
## Training Procedure
- **Hardware:** Google Colab, single NVIDIA T4 GPU (16 GB VRAM)
- **Optimizer:** 8-bit AdamW (bitsandbytes), learning rate 1e-5
- **Effective batch size:** 8 (micro-batch 1, gradient accumulation 8)
- **Precision:** FP16 mixed precision (torch.cuda.amp)
- **Memory optimizations:** Gradient checkpointing, expandable_segments
- **Epochs:** 2 (best checkpoint; further training overfits)
## Intended Use
Decision engine for a browser automation agent. Given a goal and a list of interactive DOM elements, the model answers two questions in one forward pass:
- **Operation:** CLICK, TYPE, or SELECT
- **Target:** the correct element index from the candidate list
## Limitations
- Trained on a single dataset (Mind2Web); may not generalize to all websites.
- Overfits beyond 2 epochs on 671 examples.
- Requires the base Laya tokenizer and architecture.
## How to Use
```python
import torch
import laya
agent = laya.load("convaiinnovations/laya")
model = agent.model
model.load_state_dict(torch.load("pytorch_model.bin", map_location="cpu"))
model.eval()
```
## Citation
- Laya: https://huggingface.co/convaiinnovations/laya
- Mind2Web: https://arxiv.org/abs/2306.06070
|