GoMini / README.md
Mhmadhassnz's picture
GoMini v6: continues v5 on AgentGo 0.1.6 practice pages and agent-labelled steps
3dae519 verified
|
Raw History Blame Contribute Delete
16.9 kB
---
license: apache-2.0
language:
- en
base_model: doofz/THX-01
base_model_relation: finetune
datasets:
- osunlp/Mind2Web
library_name: custom
pipeline_tag: text-classification
inference: false
tags:
- modernbert
- browser-agent
- decision-making
- agentgo
- full-fine-tuning
- safetensors
- gguf
- development-checkpoint
---
![GoMini v6: picks the next browser step, on your computer](assets/banner.svg)
# GoMini v6
**A small, local decision model for a bounded browser loop.**
GoMini is a full fine-tune of [THX-01](https://huggingface.co/doofz/THX-01)
for selecting the next action from an explicit set of valid candidates. It is
designed for **AgentGo browser** workflows: a larger model supplies a bounded
subgoal; GoMini chooses among prepared actions; AgentGo executes permitted
actions and checks their effects. Ordinary verified steps can stay local, with
the larger model called when the subgoal finishes or needs help.
Created by [Mhmadhassnz](https://huggingface.co/Mhmadhassnz).
> **Development release, not a passed production gate.** V6 continues v5 on
> 29,000 decisions: v5's set plus 12,000 new ones recorded in AgentGo 0.1.6 on
> new practice families (rows of identical buttons, scroll until found, retry
> after an error, covering dialogs, wait until enabled, "the 3rd option",
> focus a box, collapsed sections) and 577 steps of a large model on MiniWoB++
> pages it solved. On held-out practice pages it picks the right step
> **96.6%** of the time (v5 on the same pages: 76.6%). Through AgentGo 0.1.6's
> delegate loop, on BrowserGym MiniWoB++ pages it never saw, it finishes
> **53 of 65** subgoals (v5 on 0.1.6: 42; v5 on 0.1.5: 39). Calibrated
> fallback reliability and an independent final audit remain pending.
| Model | GoMini v6 (continues v5) |
| --- | --- |
| Foundation | `doofz/THX-01` · ModernBERT decision architecture |
| Parameters | 321.9M total; 321.7M optimized |
| Fine-tuning | Full encoder, embeddings and decision head; no LoRA |
| Validated task | One bounded `choice` question named `next` |
| Input contract | 1,024 packed tokens; 2–24 explicit choices |
| Formats | Complete native Safetensors checkpoint + Q8_0 GGUF |
| Training hardware | AMD Radeon RX 7800 XT · 16 GB |
| License | Apache 2.0; dataset attribution retained separately |
## What GoMini does
GoMini scores supplied candidates in a non-autoregressive forward pass. Choices
can represent a click, a supplied field/value pair, a prepared read method, or
explicit `wait`, `done`, `escalate` and `stop` controls. It does not generate an
arbitrary plan, new selector, tool argument, value or chat response.
```text
Large model: bounded goal + completion condition + permissions
|
AgentGo: observe -> prepare valid choices -> GoMini selects
^ |
| independent permission checks
| |
+----- verify actual effects <- execute allowed choice
|
verified done / needs_help / stopped
|
return to planner
```
The release contains reference selection clients, not a complete production
AgentGo integration or permission system. Every client response carries
`execution_authorized: false`; the supervisor remains responsible for execution.
## Results at a glance
### v6
![GoMini v5 to v6: right step on held-out practice pages 76.6% to 96.6%, MiniWoB++ subgoals 42 to 53 of 65, false done 3 to 4, the eight new step kinds 59.2% to 95.8%](assets/results.svg)
| Evaluation | v5 | v6 | Scope |
| --- | ---: | ---: | --- |
| Held-out practice pages (769 decisions, v5 and v6 families) | 76.6% | **96.6%** | Whole pages never trained on |
| ↳ retry after an error | 33.3% | **100%** | an optimistic toggle the server rejects once |
| ↳ rows of identical buttons | 58.8% | **100%** | "Dismiss" for the named notification |
| ↳ wait until enabled | 58.6% | **100%** | a button enabled after a moment |
| ↳ focus a box | 59.1% | **100%** | put the cursor in a named field |
| ↳ covering dialog | 66.7% | **100%** | close "What's new" first |
| ↳ collapsed sections | 78.6% | **100%** | open the right section, then click |
| ↳ scroll until found | 60.0% | 91.7% | a link further down a long page |
| ↳ "the Nth option" | 58.3% | 75.0% | unlabelled radio buttons |
| ↳ tabs | 80.6% | 75.0% | |
| ↳ tree | 73.5% | 85.7% | |
| MiniWoB++ subgoals through AgentGo 0.1.6 (13 task types x 5 seeds) | 42 / 65 | **53 / 65** | Held-out benchmark pages, Q8_0 on Vulkan, one delegation per task |
| False "done" in those 65 | 3 | 4 | GoMini said done but the page did not count the task |
Per task (v5 -> v6): focus-text 0 -> 4/5, click-collapsible 0 -> 4/5,
social-media 0 -> 3/5; click-tab-2 2/5 and form-sequence-2 0/5 for both (the
latter's instruction never says to submit).
### Inside AgentGo, driven by a large model
One model (gpt-6-luna) drives AgentGo on 38 tasks (24 BrowserGym MiniWoB++
tasks and 14 single-page-app tasks), a fresh session per task, both setups
running at the same time, twice (76 runs each). AgentGo 0.1.6 with GoMini v6
solves 94.7% of the runs with no wrong action, against 86.8% and 6 wrong
actions for AgentGo 0.1.5 with GoMini v5. The planner did not delegate to
GoMini in these runs, so they measure the browser with GoMini installed; what
GoMini does on its own is the sweep above. Details:
[AgentGo 0.1.6 benchmarks](https://github.com/Moha2005269/agentgo/blob/main/docs/benchmarks/agentgo-0.1.6.md).
![Agent benchmark: AgentGo 0.1.5 with GoMini v5 against 0.1.6 with GoMini v6, 38 tasks, 76 runs each](assets/bench-agents.svg)
### v5 (kept for reference)
| Evaluation | v4 | v5 | Scope |
| --- | ---: | ---: | --- |
| Held-out practice pages (794 decisions) | 48.7% | **96.0%** | Whole pages never trained on; forms, trees, check lists, tabs, icons, menus, sign-ins, payment buttons |
| MiniWoB++ subgoals through AgentGo (13 task types x 5 seeds) | 15 / 65 (AgentGo 0.1.4) | **39 / 65** (AgentGo 0.1.5) | Held-out benchmark pages, Q8_0 on Vulkan, one delegation per task |
| False "done" in those 65 | 26 | **3** | GoMini said done but the page did not count the task |
| AgentGo 24-form benchmark | 24 / 24 | **24 / 24** | One delegation per form |
| AgentGo install self-test | pass | **pass** | `example_request.json` |
Per-family and per-task numbers are in [EVALUATION.md](EVALUATION.md).
### v4 (kept for reference)
These are **native v4 development results**. Local suites used actual model
selections and AgentGo execution/readback on owned procedural pages. They are
not public-web accuracy, independent test scores or Q8 browser scores.
| Development evaluation | Result | Scope |
| --- | ---: | --- |
| Single-action episodes | 400 / 400 · 100% | Local click/fill/select/check and control scenarios |
| Multi-step form episodes | 133 / 133 · 100% | 471 decisions; local draft and completion checks |
| Reveal-only viewport episodes | 200 / 200 · 100% | 352 decisions; reveal without activation |
| Strict-navigation episodes | 300 / 300 · 100% | 650 decisions; prescribed scroll/back/wait policy |
| Prepared read-method episodes | 377 / 380 · 99.21% | 19 canonical read methods, prepared arguments |
| ↳ Element geometry subgroup | **17 / 20 · 85%** | Confusion with viewport/HTML reads |
| Mixed offline accepted decisions | 17,502 / 17,590 · 99.50% | Mostly procedural/synthetic examples |
| ↳ Adapted human snapshots, accepted | **365 / 441 · 82.77%** | Mind2Web-derived, target included in shortlist |
| ↳ Adapted human snapshots, all raw inputs | **365 / 504 · 72.42%** | 63 packing rejections counted as failures |
Zero false-done predictions were observed in the listed native action, form,
viewport and navigation suites. These include scenarios where a correct stop or
escalation is success; they do not prove universal task completion or reliable
failure detection. Navigation's oracle requires explicit scrolling even when a
browser click could auto-reveal its target.
**Q8_0 delivery smoke:** 64 attempted synthetic/TRAIN-derived inputs; **56
supported inputs matched both exact token IDs and native top choices**. Eight
inputs were rejected before Q8 decision inference by the packing guard. Maximum
absolute probability drift on compared inputs was 0.0008011311. This checks
conversion and interfaces, not browser success.
Read [EVALUATION.md](EVALUATION.md) for every native episode subgroup, all 19 read
methods, offline source breakdowns, rejection counts, methodology and remaining
evaluation. Machine-readable summaries and the original packing failure are in
[reports/](reports/).
## Download
| Artifact | Size | Use |
| --- | ---: | --- |
| [native/model.safetensors](native/model.safetensors) | 643,835,524 bytes | Full native checkpoint; download its configs/tokenizer/runtime too |
| [gguf/GoMini-v6-Q8_0.gguf](gguf/GoMini-v6-Q8_0.gguf) | 356,866,560 bytes | Decision-capable GGUF runtime only (SHA-256 `b90ce841…5a49bb6`) |
| [adapter/](adapter/) | Python helpers | Shared packing and resident selection clients; no learned adapters |
| [runtime/thx01/](runtime/thx01/) | Python implementation | Bundled upstream native inference code |
Use revision **`v6`** for the tagged release (`v5` and `v4` stay available). Checksums are in
[SHA256SUMS.txt](SHA256SUMS.txt), with conversion identities and status in
[PROVENANCE.json](PROVENANCE.json). No Q4 or intermediate F16 GGUF is distributed.
## Quickstart
Use a compatible **GPU PyTorch** environment with Transformers, Safetensors and
NumPy. The recorded environment and Q8 server requirements are in
[ENVIRONMENT.md](ENVIRONMENT.md). CPU portability and other GPU stacks have not
been validated by this release. It is a custom decision checkpoint; loading it
as `AutoModelForCausalLM` or sending chat-completion messages is not supported.
Download and run the included single-request example with that environment:
```python
import subprocess
import sys
from pathlib import Path
from huggingface_hub import snapshot_download
root = Path(snapshot_download("Mhmadhassnz/GoMini", revision="v6"))
subprocess.run([sys.executable, str(root / "examples" / "choose.py")], check=True)
```
The example reads [example_request.json](example_request.json), serializes it to
one line and asks the native chooser to select a candidate. It does not execute
browser actions. For an integration, keep `adapter/choose_native.py` resident:
write one compact JSON object per line to stdin and read one JSON result per line
from stdout. Diagnostic messages use stderr.
For Q8_0, first start a compatible **ModernBERT decision-capable llama.cpp
server** with `/tokenize` and `/v1/systemone`, following
[ENVIRONMENT.md](ENVIRONMENT.md). Its binary is not included. Then run:
```python
subprocess.run([
sys.executable, str(root / "examples" / "choose.py"),
"--q8", "--url", "http://127.0.0.1:8765",
], check=True)
```
Use `adapter/q8_client.py` for a resident guarded Q8 client. Keep its packing and
runtime token checks: some literal-special-token/trailing-whitespace inputs do
not match native per-piece packing and must escalate. Native/Q8 agreement on the
limited smoke suite does not establish equivalence for every possible input.
## Integrating with AgentGo
The recommended loop and implementation details are in
[LIMITATIONS_AND_AGENTGO_INTEGRATION.md](LIMITATIONS_AND_AGENTGO_INTEGRATION.md).
1. Have the large model specify the subgoal, completion predicate, permitted tab
and origin, exact supplied values, effect permissions, deadline and step budget.
2. Build a fresh compact observation and bind valid candidate keys to prepared
browser operations. Include explicit wait, stop, done and escalation controls.
3. Use the shared `gomini.agentgo/1` packer. Keep the 512-token instruction/options
head budget, 1,024-token total, maximum 24 choices and 48 tokens per rendered
option. Reject oversize input rather than silently truncating it.
4. Independently check permissions, tab/origin/revision, candidate membership,
exact field/value binding and attempt limits before executing any selection.
5. Verify effects from browser evidence. Stop on cancellation; return compact
evidence to the larger model on verified completion, ambiguity, missing
information, repeated failure or exhausted budget.
For a known fixed request such as an element rectangle, deterministic dispatch
to the correct prepared method can be better than asking a model to rediscover
the mapping. A successful command or HTTP 200 does not prove goal completion.
## Limitations
- **General browsing is unproven.** Local fixtures share families and some markup;
split-separated local labels are not independent real websites or accounts.
- **Human transfer is weaker than the procedural average.** The adapted dataset
includes the annotated target in each shortlist. Its scores are not official
Mind2Web benchmark results or full browser trajectories.
- **1,088 registry names are not 1,088 verified tools.** Training includes synthetic
description routing, including aliases; actual read-method checks cover 19
canonical methods, with prepared arguments and no write-operation coverage.
- **Confidence is uncalibrated.** V4 uses temperature 1.0; confident mistakes
occurred. Probability, margin and entropy must not grant permission. The unused
auxiliary action head was not supervised and must not gate safety.
- **Fallback is an unmeasured property.** Do not assume every failed action will
request help. Validate independent checks and larger-model handoffs separately.
- **Q8 browser evaluation and final audit remain pending.** Packing guards reject
some inputs; reference clients are not a hardened supervisor. No general >90%
success, latency, multilingual retention or production-readiness claim is made.
## Training and provenance
V6 is a full fine-tune of the v5 checkpoint on 28,317 decisions (v5's
14,902-record set plus 12,049 new teacher records on 0.1.6's practice pages
and 577 agent-labelled steps; the best of 2 epochs by held-out pages, batch
8 x 4 accumulation, AdamW 3e-6, same objective). The agent-labelled steps are
the actions of gpt-6-luna on MiniWoB++ pages outside every evaluation set,
taken only from runs the page scored as solved; held-out records are whole
pages (seeds 39001-39300). No evaluation page (the 24 agent-benchmark tasks,
the 13 GoMini sweep task types, the 14 SPA tasks) was used.
V5 is a full fine-tune of the v4 checkpoint on 14,902 new decisions (2 epochs,
AdamW 3e-6, same objective), recorded by a teacher on 6,000 procedurally
generated practice pages served to AgentGo 0.1.5 and packed with AgentGo's
own packer; held-out records are whole pages. No MiniWoB++ or other benchmark
page was used. The recipe and per-family counts are in [TRAINING.md](TRAINING.md).
V4 continues the prior native checkpoint with **53,146 accepted unique training
records**, combining procedural examples, synthetic registry routing, adapted
Mind2Web snapshots and verified decisions on owned local AgentGo fixtures. It
reached epoch 2 / step 5,660 using AdamW, BF16 autocast, FP32 optimization and a
stable log-score plus spherical-score objective. Embeddings and all encoder
blocks were trainable; this was full fine-tuning.
See [TRAINING.md](TRAINING.md) for the recipe and source counts, and
[native/training-settings.json](native/training-settings.json) for exact data
file hashes and settings. No raw website corpus or complete collection harness
is hosted here. Development influenced selection; frozen calibration/final-test
records were not consumed for this handoff.
## Credits and license
GoMini builds on **THX-01 by HAL-X AI**, with its mmBERT encoder attribution
preserved in [NOTICE](NOTICE). Upstream weights and bundled runtime retain their
[Apache 2.0 license](LICENSE). The pinned foundation revision is
`b323afcef149f9288a9d0daef4badd67b9b87c61`.
**Mind2Web** is credited to Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel
Stevens, Boshi Wang, Huan Sun and Yu Su (2023). Its dataset is CC BY 4.0; the local
adaptation changes representation, packing and split assignment. See
[THIRD_PARTY_DATA.md](THIRD_PARTY_DATA.md) and the
[official Mind2Web project](https://github.com/OSU-NLP-Group/Mind2Web).
AgentGo is the browser integration target and execution environment for the
reported local fixture checks. AgentGo browser software is not distributed in
this model repository.