Upload README.md with huggingface_hub

161d54d verified 30 days ago

8.06 kB

---
license: apache-2.0
base_model: cklxx/laya-browser
tags:
- laya
- system-1
- browser-agent
- web-navigation
- decision-model
- modernbert
- finetuned
- done-judgment
- noul
library_name: safetensors
language:
- en
pipeline_tag: text-classification
---
# laya-browser-v32b (v32b-b15)

A 322M-parameter browser-agent decision model, fine-tuned from
[`cklxx/laya-browser`](https://huggingface.co/cklxx/laya-browser) (which fine-tuned
[`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) for browser
decisions). Fully Apache-2.0 upstream; this checkpoint keeps the same license.

**What it is:** a System-1 decision head for browser agents. It never writes text —
it answers *typed questions* about a page state (which operation? which element?
does the goal-holding statement hold?) and returns **calibrated probabilities**.
That makes it safe to put in front of a browser: it can only choose from what it
was shown.

**Headline numbers** (all measured 2026-09-25, same fixtures for every model):

| Benchmark | **v32b-b15** | official laya-browser (td) | hosted Jev API |
|---|---:|---:|---:|
| recovery2-holdout (240) | **0.7125** | 0.425 | — |
| browser-suite v5 (110) | 0.5636 | **0.5818** | — |
| browser-suite v4 (70) | **0.5143** | 0.500 | — |
| MiniWoB-116 | **0.9138** | 0.6638 | — |
| JevBench hard (111) | **0.4144** | 0.243 | 0.7207 |
| JevBench easy (48) | 0.8542 | 0.979 | 1.0000 |
| decision latency (p50) | **27 ms** (RTX 3080) | — | 854 ms (network) |

vs the official browser-tuned checkpoint: **wins holdout (+28.8pp), MiniWoB (+25pp),
suite v4 (+1.4pp), JevBench hard (+17.1pp), injection_safety (0.75 vs 0.50)**; loses
suite v5 by 2 items. vs the hosted Jev API: lower absolute accuracy but **$0 cost,
fully private, offline, 31x faster** — and it *beats* Jev on `score` questions
(0.667 vs 0.333) and `temporal_numeric` (0.33 vs 0.20) on JevBench hard.
## What was trained
Starting from the official `v10s`-lineage browser checkpoint, this model adds
(v17→v32 recipe, frozen encoder + head-only training throughout):
1. **SCROLL_UP + recovery data** — the official checkpoint had a 0/203 hit rate on
   scroll-up; this one is 193/203 with no loss of general accuracy.
2. **Counterfactual ranking pairs** — Sharper target choice among similar candidates.
3. **Noul (statement-holds) training** — the upstream training pipeline never
   produced `noul` items (yes/no judgment questions), leaving done_judgment at
   0.615. A purpose-built 8k-item noul corpus (8 patterns × 320, balanced
   yes/no) raised probe accuracy from **37% → 100%** and done_judgment to 0.769
   (probability-averaged with the choice specialist).
4. **Head blending** — capability heads are blended into the champion at low
   weight (6–15%) so new skills arrive with zero regression elsewhere.
The encoder (134 tensors) is byte-identical to upstream; only the 36-tensor
decision head was trained.
## Usage
Requires [`laya`](https://pypi.org/project/laya/) (`pip install laya`) or
[`laya-mlx`](https://pypi.org/project/laya-mlx/) on Apple Silicon.
```python
import laya
agent = laya.load("ichenney/laya-browser-v32b", subfolder="v32b", device="cuda")
state = {...}  # page state: url, title, text, elements table
questions = {
    "operation": {"type": "choice",
                  "instructions": "Pick the operation that achieves the goal.",
                  "criteria": {"0": "CLICK", "1": "TYPE_TEXT", "2": "SCROLL_DOWN",
                               "3": "SCROLL_UP", "4": "SELECT_OPTION", "5": "WAIT",
                               "6": "DONE", "7": "BLOCKED"}},
}
answers = agent.predict(state, questions)["answers"]
print(answers["operation"]["choice"], answers["operation"]["probabilities"])
```
Or through the **`localdecide`** harness (recommended — it adds element-table
building, coarse-to-fine chunking, and guards). v32b is the harness's
**default** model — `model="browser"` loads this repo automatically:
```python
from localdecide import BrowserDecider
from localdecide.backends import LayaTorchBackend
backend = LayaTorchBackend()          # model="browser" → this checkpoint
# to stay on the upstream cklxx v10s instead:
# backend = LayaTorchBackend(model="browser-legacy")
decider = BrowserDecider(backend=backend)
```
## Checkpoint contents
```
v32b/
├── model.safetensors      # full model: frozen encoder (134 tensors) + trained head (36)
├── rl_agent_config.json   # laya agent config (max_len 1024, head_max_len 768, v1 format)
├── encoder/config.json    # ModernBERT-style encoder architecture config
└── tokenizer/             # tokenizer.json + config (same vocab as upstream)
```
## Training details
- **Base**: [`cklxx/laya-browser`](https://huggingface.co/cklxx/laya-browser)
  (frozen encoder — 134 tensors untouched, byte-identical), head-only fine-tune (36 tensors)
- **Data** (each family generated synthetically from real page fixtures, no user data):
  | corpus | items | purpose |
  |---|---:|---|
  | general browser decisions (v10s-lineage mix) | ~11,800 | retain upstream competence |
  | targeted 6-family patch (op-choice / state-reason / multistep / surface-trap / injection / filter-first, 8 languages) | 9,600 | v20: fix known failure axes |
  | counterfactual ranking pairs | 11,200 | v21: sharper target choice (blend-only) |
  | SCROLL_UP + recovery | 6,972 | v23: upstream had 0/203 scroll-up probe |
  | multistep replanning | 5,600 | v25: proved harmful, archived |
  | 22-failure-mode patch | 6,300 | v29: evidence for the noul investigation |
  | **noul statement-holds** | **8,000** (3,200 yes / 4,800 no) | **v31: the root-cause fix** — 8 patterns × 320, no-heavy to counter the model's yes-bias (probe NOUL_NO was 37%) |
- **Optimization**: head LR 1e-4, bf16 autocast, batch 8, GRPO-style grouped
  advantages (group 4, w_sph 0.75 / w_rps 1.0), 1–3 epochs per stage
- **Cost**: ~3 minutes per training run on one RTX 3080; the entire 15-version
  campaign totals under 2 GPU-hours
- **Method**: single-variable iterations — change one thing, run the full eval
  battery (browser-suite v4/v5 + recovery2-holdout, ~90 min), keep or discard on
  data. New capabilities are trained as separate heads and blended into the
  champion at 6–15% weight, which transfers skills with zero measured regression.
- **Champion selection**: highest held-out mean across the 3 suites; the final
  ensemble (v36) probability-averages v32b-b15 with the v31a noul specialist at
  w=0.4
- **Evaluation**: 3 suites × 3 official baselines + hosted Jev API (fresh
  231-item run), all JSONs published in the
  [training log](https://github.com/ChenneyZhuang/laya-training-log)
## Limitations
- 322M parameters: complex multi-hop reasoning (JevBench `multi_hop` 0.22 vs
  hosted Jev 0.89) is beyond current capacity — this is a capacity, not a
  recipe, gap
- English-dominant training data (the synthetic fixtures cover 8 languages, but
  natural-page diversity is English-heavy)
- Safety-adjacent judgments (phishing, abuse) should be guarded by harness
  rules, not trusted to the head alone — the v10s phishing analysis in the
  [harness README](https://github.com/ChenneyZhuang/laya-browser-agent#head-to-head-vs-hosted-jev-measured-not-claimed-v10s-era-kept-for-provenance)
  applies to v32b too
- FP32 weights: 1.3 GB on disk. An FP16/BF16 export would halve that; the
  MLX runtime loads it fine on 8 GB Apple Silicon but the torch path wants ~2 GB resident
## License
Apache-2.0 (inherited from both upstream checkpoints). The companion harness is
[github.com/ChenneyZhuang/laya-browser-agent](https://github.com/ChenneyZhuang/laya-browser-agent).
## Citation
```bibtex
@misc{laya-browser-v32b,
  title  = {laya-browser-v32b: a noul-capable browser decision model},
  author = {Chenney Zhuang},
  year   = {2026},
  url    = {https://huggingface.co/ichenney/laya-browser-v32b},
  note   = {Fine-tuned from cklxx/laya-browser (Apache-2.0), frozen-encoder head training with noul corpus injection}
}
```

	---
	license: apache-2.0
	base_model: cklxx/laya-browser
	tags:
	- laya
	- system-1
	- browser-agent
	- web-navigation
	- decision-model
	- modernbert
	- finetuned
	- done-judgment
	- noul
	library_name: safetensors
	language:
	- en
	pipeline_tag: text-classification
	---

	# laya-browser-v32b (v32b-b15)

	A 322M-parameter browser-agent decision model, fine-tuned from
	[`cklxx/laya-browser`](https://huggingface.co/cklxx/laya-browser) (which fine-tuned
	[`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) for browser
	decisions). Fully Apache-2.0 upstream; this checkpoint keeps the same license.

	What it is: a System-1 decision head for browser agents. It never writes text —
	it answers typed questions about a page state (which operation? which element?
	does the goal-holding statement hold?) and returns calibrated probabilities.
	That makes it safe to put in front of a browser: it can only choose from what it
	was shown.

	Headline numbers (all measured 2026-09-25, same fixtures for every model):

	\| Benchmark \| v32b-b15 \| official laya-browser (td) \| hosted Jev API \|
	\|---\|---:\|---:\|---:\|
	\| recovery2-holdout (240) \| 0.7125 \| 0.425 \| — \|
	\| browser-suite v5 (110) \| 0.5636 \| 0.5818 \| — \|
	\| browser-suite v4 (70) \| 0.5143 \| 0.500 \| — \|
	\| MiniWoB-116 \| 0.9138 \| 0.6638 \| — \|
	\| JevBench hard (111) \| 0.4144 \| 0.243 \| 0.7207 \|
	\| JevBench easy (48) \| 0.8542 \| 0.979 \| 1.0000 \|
	\| decision latency (p50) \| 27 ms (RTX 3080) \| — \| 854 ms (network) \|

	vs the official browser-tuned checkpoint: **wins holdout (+28.8pp), MiniWoB (+25pp),
	suite v4 (+1.4pp), JevBench hard (+17.1pp), injection_safety (0.75 vs 0.50)**; loses
	suite v5 by 2 items. vs the hosted Jev API: lower absolute accuracy but **$0 cost,
	fully private, offline, 31x faster** — and it beats Jev on `score` questions
	(0.667 vs 0.333) and `temporal_numeric` (0.33 vs 0.20) on JevBench hard.

	## What was trained

	Starting from the official `v10s`-lineage browser checkpoint, this model adds
	(v17→v32 recipe, frozen encoder + head-only training throughout):

	1. SCROLL_UP + recovery data — the official checkpoint had a 0/203 hit rate on
	scroll-up; this one is 193/203 with no loss of general accuracy.
	2. Counterfactual ranking pairs — Sharper target choice among similar candidates.
	3. Noul (statement-holds) training — the upstream training pipeline never
	produced `noul` items (yes/no judgment questions), leaving done_judgment at
	0.615. A purpose-built 8k-item noul corpus (8 patterns × 320, balanced
	yes/no) raised probe accuracy from 37% → 100% and done_judgment to 0.769
	(probability-averaged with the choice specialist).
	4. Head blending — capability heads are blended into the champion at low
	weight (6–15%) so new skills arrive with zero regression elsewhere.

	The encoder (134 tensors) is byte-identical to upstream; only the 36-tensor
	decision head was trained.

	## Usage

	Requires [`laya`](https://pypi.org/project/laya/) (`pip install laya`) or
	[`laya-mlx`](https://pypi.org/project/laya-mlx/) on Apple Silicon.

	```python
	import laya

	agent = laya.load("ichenney/laya-browser-v32b", subfolder="v32b", device="cuda")
	state = {...} # page state: url, title, text, elements table
	questions = {
	"operation": {"type": "choice",
	"instructions": "Pick the operation that achieves the goal.",
	"criteria": {"0": "CLICK", "1": "TYPE_TEXT", "2": "SCROLL_DOWN",
	"3": "SCROLL_UP", "4": "SELECT_OPTION", "5": "WAIT",
	"6": "DONE", "7": "BLOCKED"}},
	}
	answers = agent.predict(state, questions)["answers"]
	print(answers["operation"]["choice"], answers["operation"]["probabilities"])
	```

	Or through the `localdecide` harness (recommended — it adds element-table
	building, coarse-to-fine chunking, and guards). v32b is the harness's
	default model — `model="browser"` loads this repo automatically:

	```python
	from localdecide import BrowserDecider
	from localdecide.backends import LayaTorchBackend

	backend = LayaTorchBackend() # model="browser" → this checkpoint
	# to stay on the upstream cklxx v10s instead:
	# backend = LayaTorchBackend(model="browser-legacy")
	decider = BrowserDecider(backend=backend)
	```

	## Checkpoint contents

	```
	v32b/
	├── model.safetensors # full model: frozen encoder (134 tensors) + trained head (36)
	├── rl_agent_config.json # laya agent config (max_len 1024, head_max_len 768, v1 format)
	├── encoder/config.json # ModernBERT-style encoder architecture config
	└── tokenizer/ # tokenizer.json + config (same vocab as upstream)
	```

	## Training details

	- Base: [`cklxx/laya-browser`](https://huggingface.co/cklxx/laya-browser)
	(frozen encoder — 134 tensors untouched, byte-identical), head-only fine-tune (36 tensors)
	- Data (each family generated synthetically from real page fixtures, no user data):
	\| corpus \| items \| purpose \|
	\|---\|---:\|---\|
	\| general browser decisions (v10s-lineage mix) \| ~11,800 \| retain upstream competence \|
	\| targeted 6-family patch (op-choice / state-reason / multistep / surface-trap / injection / filter-first, 8 languages) \| 9,600 \| v20: fix known failure axes \|
	\| counterfactual ranking pairs \| 11,200 \| v21: sharper target choice (blend-only) \|
	\| SCROLL_UP + recovery \| 6,972 \| v23: upstream had 0/203 scroll-up probe \|
	\| multistep replanning \| 5,600 \| v25: proved harmful, archived \|
	\| 22-failure-mode patch \| 6,300 \| v29: evidence for the noul investigation \|
	\| noul statement-holds \| 8,000 (3,200 yes / 4,800 no) \| v31: the root-cause fix — 8 patterns × 320, no-heavy to counter the model's yes-bias (probe NOUL_NO was 37%) \|
	- Optimization: head LR 1e-4, bf16 autocast, batch 8, GRPO-style grouped
	advantages (group 4, w_sph 0.75 / w_rps 1.0), 1–3 epochs per stage
	- Cost: ~3 minutes per training run on one RTX 3080; the entire 15-version
	campaign totals under 2 GPU-hours
	- Method: single-variable iterations — change one thing, run the full eval
	battery (browser-suite v4/v5 + recovery2-holdout, ~90 min), keep or discard on
	data. New capabilities are trained as separate heads and blended into the
	champion at 6–15% weight, which transfers skills with zero measured regression.
	- Champion selection: highest held-out mean across the 3 suites; the final
	ensemble (v36) probability-averages v32b-b15 with the v31a noul specialist at
	w=0.4
	- Evaluation: 3 suites × 3 official baselines + hosted Jev API (fresh
	231-item run), all JSONs published in the
	[training log](https://github.com/ChenneyZhuang/laya-training-log)

	## Limitations

	- 322M parameters: complex multi-hop reasoning (JevBench `multi_hop` 0.22 vs
	hosted Jev 0.89) is beyond current capacity — this is a capacity, not a
	recipe, gap
	- English-dominant training data (the synthetic fixtures cover 8 languages, but
	natural-page diversity is English-heavy)
	- Safety-adjacent judgments (phishing, abuse) should be guarded by harness
	rules, not trusted to the head alone — the v10s phishing analysis in the
	[harness README](https://github.com/ChenneyZhuang/laya-browser-agent#head-to-head-vs-hosted-jev-measured-not-claimed-v10s-era-kept-for-provenance)
	applies to v32b too
	- FP32 weights: 1.3 GB on disk. An FP16/BF16 export would halve that; the
	MLX runtime loads it fine on 8 GB Apple Silicon but the torch path wants ~2 GB resident

	## License

	Apache-2.0 (inherited from both upstream checkpoints). The companion harness is
	[github.com/ChenneyZhuang/laya-browser-agent](https://github.com/ChenneyZhuang/laya-browser-agent).

	## Citation

	```bibtex
	@misc{laya-browser-v32b,
	title = {laya-browser-v32b: a noul-capable browser decision model},
	author = {Chenney Zhuang},
	year = {2026},
	url = {https://huggingface.co/ichenney/laya-browser-v32b},
	note = {Fine-tuned from cklxx/laya-browser (Apache-2.0), frozen-encoder head training with noul corpus injection}
	}
	```