| # GoMini evaluation record (v5 first, then v4) | |
| ## v6 | |
| ### Held-out practice pages (769 decisions, seeds 39001-39300) | |
| | Family | v5 | v6 | | |
| | --- | ---: | ---: | | |
| | all | 76.6% | **96.6%** | | |
| | checks | 97.4% | 100% | | |
| | click | 100% | 100% | | |
| | complete | 100% | 100% | | |
| | dialog | 66.7% | 100% | | |
| | expand | 78.6% | 100% | | |
| | focus | 59.1% | 100% | | |
| | form | 97.7% | 98.4% | | |
| | icons | 75.3% | 100% | | |
| | login | 100% | 100% | | |
| | ordinal | 58.3% | 75.0% | | |
| | radio | 100% | 100% | | |
| | retry | 33.3% | 100% | | |
| | rows | 58.8% | 100% | | |
| | scrollfind | 60.0% | 91.7% | | |
| | select | 100% | 100% | | |
| | sensitive | 93.3% | 100% | | |
| | tabs | 80.6% | 75.0% | | |
| | tree | 73.5% | 85.7% | | |
| | wait | 58.6% | 100% | | |
| ### Held-out MiniWoB++ subgoals through AgentGo 0.1.6 | |
| 13 task types x 5 seeds (200-204), one delegation each, Q8_0 on Vulkan, | |
| scored by the page. A final Submit is the planner's; the script presses the | |
| step the reply names. | |
| | Task | v5 | v6 | | |
| | --- | ---: | ---: | | |
| | click-button | 5/5 | 5/5 | | |
| | click-link | 5/5 | 5/5 | | |
| | click-option | 5/5 | 5/5 | | |
| | click-checkboxes | 5/5 | 5/5 | | |
| | click-tab-2 | 2/5 | 2/5 | | |
| | enter-text | 5/5 | 5/5 | | |
| | focus-text | 0/5 | 4/5 | | |
| | choose-list | 5/5 | 5/5 | | |
| | enter-date | 5/5 | 5/5 | | |
| | navigate-tree | 5/5 | 5/5 | | |
| | click-collapsible | 0/5 | 4/5 | | |
| | social-media | 0/5 | 3/5 | | |
| | form-sequence-2 | 0/5 | 0/5 | | |
| | **Solved** | **42/65** | **53/65** | | |
| | False "done" | 3 | 4 | | |
| The same 65 with AgentGo 0.1.5 and v5: 39/65, 3 false "done". | |
| ## v5 | |
| V5 was measured through AgentGo 0.1.5's delegate loop with the Q8_0 file on | |
| the GPU (llama.cpp b11559, Vulkan, Radeon RX 7800 XT), and natively on the | |
| held-out practice records. | |
| ### Held-out practice pages (native, 794 decisions on whole pages never trained on) | |
| | Family | v4 | v5 | | |
| | --- | ---: | ---: | | |
| | all | 48.7% | **96.0%** | | |
| | checks | 26.8% | 95.3% | | |
| | click | 54.8% | 100% | | |
| | complete | 93.3% | 100% | | |
| | form | 60.3% | 100% | | |
| | icons | 47.3% | 85.5% | | |
| | login | 96.7% | 100% | | |
| | radio | 73.3% | 100% | | |
| | select | 100% | 100% | | |
| | sensitive | 0% | 100% | | |
| | tabs | 37.7% | 91.3% | | |
| | tree | 32.0% | 91.4% | | |
| ### MiniWoB++ subgoals through AgentGo (held-out benchmark pages) | |
| BrowserGym's MiniWoB++ pages (browsergym-miniwob 0.14.3), served locally with | |
| BrowserGym's own start-up, 13 task types x 5 seeds (200-204). Each task's | |
| instruction became one `gomini.delegate` subgoal; a final Submit is the | |
| planner's and was pressed by the script after a delegation that said done. | |
| The page's own reward decides. No MiniWoB++ page was used in training. | |
| | Task | v4 + AgentGo 0.1.4 | v5 + AgentGo 0.1.5 | | |
| | --- | ---: | ---: | | |
| | click-button | 4/5 | 4/5 | | |
| | click-link | 5/5 | 4/5 | | |
| | click-option | 0/5 | 4/5 | | |
| | click-checkboxes | 0/5 | 5/5 | | |
| | click-tab-2 | 2/5 | 2/5 | | |
| | enter-text | 0/5 | 5/5 | | |
| | focus-text | 0/5 | 0/5 | | |
| | choose-list | 0/5 | 5/5 | | |
| | enter-date | 0/5 | 5/5 | | |
| | navigate-tree | 4/5 | 5/5 | | |
| | click-collapsible | 0/5 | 0/5 | | |
| | social-media | 0/5 | 0/5 | | |
| | form-sequence-2 | 0/5 | 0/5 | | |
| | **Solved** | **15/65** | **39/65** | | |
| | False "done" | 26 | **3** | | |
| Part of the gain is AgentGo 0.1.5's supervisor (it waits for loaded pages, | |
| lists script-driven controls, binds values to named fields and checks "done" | |
| against them), so the two columns compare releases, not weights alone. V5 | |
| has no step for focusing a text box or opening a collapsed section (never in | |
| its training), does not map "the 3rd radio button" to `radio "3 of 3"`, and | |
| on click-tab-2 clicked a wrong link in 3 of 5 seeds. | |
| ### Kept from v4 | |
| - AgentGo's 24-form benchmark: 24/24, one delegation each. | |
| - AgentGo's install self-test (`example_request.json`, choice `r748|entry`): | |
| passes. | |
| - Q8_0 made with the same converter and quantizer as v4; re-converting v4 | |
| reproduces its published SHA-256 byte for byte. | |
| The v4 record follows unchanged. | |
| All scores below are **development results**, not a final independent test. | |
| Development data influenced checkpoint selection and iterative improvements. | |
| Frozen calibration and final-test records were not accessed for this handoff. | |
| Metrics were measured before publication; publication does not rerun training. | |
| ## Native model-driven AgentGo episodes | |
| | Suite | Successful / total episodes | Correct / total decisions | Rate | | |
| | --- | ---: | ---: | ---: | | |
| | Single action | 400 / 400 | 400 / 400 | 100% | | |
| | Multi-step local forms | 133 / 133 | 471 / 471 | 100% | | |
| | Physical reveal-only viewport | 200 / 200 | 352 / 352 | 100% | | |
| | Strict navigation policy | 300 / 300 | 650 / 650 | 100% | | |
| | Prepared read-capability selection | 377 / 380 | 377 / 380 | 99.21% | | |
| The first four suites recorded zero false-done predictions. The protocol suite | |
| does not report a false-done metric. An episode can succeed by correctly waiting, | |
| stopping or escalating when its control scenario requires that behavior; success | |
| does not mean every episode performed a mutation. These are owned loopback pages, | |
| not arbitrary public-site browsing. The browser executed model selections under | |
| a harness with prepared candidates, independent outcome checks and strict scope. | |
| Wrong choices can terminate evaluation before execution. | |
| Navigation requires a separate reveal/scroll step before offscreen activation. | |
| AgentGo can auto-reveal a target when clicking, so this score measures adherence | |
| to the prescribed policy, not all physically successful navigation behavior. | |
| Local domain/layout assignments and episodes were separated across splits, but | |
| procedural families and some markup were shared. This is weaker evidence than | |
| independent websites, real accounts or human-authored unseen workflows. | |
| ### All native episode subgroups | |
| #### Single-action categories | |
| | Group | Correct / count | Rate | | |
| | --- | ---: | ---: | | |
| | `select` | 58 / 58 | 100.00% | | |
| | `fill` | 73 / 73 | 100.00% | | |
| | `escalate` | 46 / 46 | 100.00% | | |
| | `wait` | 44 / 44 | 100.00% | | |
| | `click` | 96 / 96 | 100.00% | | |
| | `check` | 45 / 45 | 100.00% | | |
| | `done` | 38 / 38 | 100.00% | | |
| #### Multi-step form episode modes | |
| | Group | Correct / count | Rate | | |
| | --- | ---: | ---: | | |
| | `exhausted` | 16 / 16 | 100.00% | | |
| | `complete_form` | 90 / 90 | 100.00% | | |
| | `already_saved` | 5 / 5 | 100.00% | | |
| | `cancelled` | 7 / 7 | 100.00% | | |
| | `missing_value` | 15 / 15 | 100.00% | | |
| #### Reveal-only viewport episode modes | |
| | Group | Correct / count | Rate | | |
| | --- | ---: | ---: | | |
| | `exhausted` | 17 / 17 | 100.00% | | |
| | `cancelled` | 17 / 17 | 100.00% | | |
| | `completed` | 17 / 17 | 100.00% | | |
| | `missing` | 17 / 17 | 100.00% | | |
| | `below` | 17 / 17 | 100.00% | | |
| | `above` | 17 / 17 | 100.00% | | |
| | `visible` | 17 / 17 | 100.00% | | |
| | `previous_page` | 17 / 17 | 100.00% | | |
| | `pending` | 16 / 16 | 100.00% | | |
| | `disabled` | 16 / 16 | 100.00% | | |
| | `ambiguous` | 16 / 16 | 100.00% | | |
| | `stale` | 16 / 16 | 100.00% | | |
| #### Strict-navigation episode modes | |
| | Group | Correct / count | Rate | | |
| | --- | ---: | ---: | | |
| | `pending` | 25 / 25 | 100.00% | | |
| | `disabled` | 25 / 25 | 100.00% | | |
| | `ambiguous` | 25 / 25 | 100.00% | | |
| | `stale` | 25 / 25 | 100.00% | | |
| | `exhausted` | 25 / 25 | 100.00% | | |
| | `cancelled` | 25 / 25 | 100.00% | | |
| | `completed` | 25 / 25 | 100.00% | | |
| | `missing` | 25 / 25 | 100.00% | | |
| | `below` | 25 / 25 | 100.00% | | |
| | `above` | 25 / 25 | 100.00% | | |
| | `visible` | 25 / 25 | 100.00% | | |
| | `previous_page` | 25 / 25 | 100.00% | | |
| #### Read-only canonical methods | |
| | Group | Correct / count | Rate | | |
| | --- | ---: | ---: | | |
| | `observe.text` | 20 / 20 | 100.00% | | |
| | `observe.links` | 20 / 20 | 100.00% | | |
| | `observe.tables` | 20 / 20 | 100.00% | | |
| | `observe.forms` | 20 / 20 | 100.00% | | |
| | `observe.page_metadata` | 20 / 20 | 100.00% | | |
| | `observe.lists` | 20 / 20 | 100.00% | | |
| | `observe.images` | 20 / 20 | 100.00% | | |
| | `runtime.get_value` | 20 / 20 | 100.00% | | |
| | `runtime.get_text` | 20 / 20 | 100.00% | | |
| | `runtime.get_attribute` | 20 / 20 | 100.00% | | |
| | `runtime.get_html` | 20 / 20 | 100.00% | | |
| | `runtime.get_rect` | 17 / 20 | 85.00% | | |
| | `observe.viewport` | 20 / 20 | 100.00% | | |
| | `observe.computed_style` | 20 / 20 | 100.00% | | |
| | `observe.loading_indicators` | 20 / 20 | 100.00% | | |
| | `observe.shadow_roots` | 20 / 20 | 100.00% | | |
| | `observe.canvas` | 20 / 20 | 100.00% | | |
| | `observe.layout` | 20 / 20 | 100.00% | | |
| | `observe.dom_subtree` | 20 / 20 | 100.00% | | |
| The three `runtime.get_rect` errors selected `observe.viewport` twice and | |
| `runtime.get_html` once. A valid read response can still be wrong for the goal. | |
| Use exact capability/schema checks or deterministic dispatch for known geometry | |
| requests. No subgroup-level >90% production gate was achieved. | |
| ## Offline development: source breakdown | |
| This is a separate offline decision evaluation; it must not be added to the | |
| browser episode counts. Some offline/browser suites represent related procedural | |
| families. The combined score is dominated by procedural data. | |
| | Source | Correct / accepted | Accuracy | | |
| | --- | ---: | ---: | | |
| | `registry_routing_synthetic` | 1,988 / 2,000 | 99.40% | | |
| | `procedural_oracle` | 8,000 / 8,000 | 100.00% | | |
| | `agentgo_browser_lab` | 250 / 250 | 100.00% | | |
| | `agentgo_multistep_lab` | 369 / 369 | 100.00% | | |
| | `multistep_procedural_oracle` | 4,000 / 4,000 | 100.00% | | |
| | `agentgo_navigation_lab` | 130 / 130 | 100.00% | | |
| | `navigation_recovery_procedural_oracle` | 2,000 / 2,000 | 100.00% | | |
| | `agentgo_protocol_read_lab` | 190 / 190 | 100.00% | | |
| | `agentgo_viewport_lab` | 210 / 210 | 100.00% | | |
| | `mind2web_cc_by_4` | 365 / 441 | 82.77% | | |
| | `overall` | 17,502 / 17,590 | 99.50% | | |
| Mind2Web adaptation: 504 raw development records, 63 packing rejections, | |
| 441 accepted records, 365 correct. Accepted-input accuracy is **82.77%**; counting | |
| packing rejections as failures gives **72.42% (365/504)**. The annotated target was | |
| included in each shortlist, with websites hash-partitioned across local splits. | |
| This is not official Mind2Web end-to-end success, retrieval performance or live | |
| browser accuracy. The corpus's upstream license is CC BY 4.0; see attribution. | |
| Registry-routing descriptions cover 1,088 implemented names, including aliases. | |
| They are synthetic matching examples, not verification of 1,088 executed tools. | |
| Only 19 canonical read methods were exercised in the read-capability suite. | |
| The complete offline source/category/intersection records are retained in | |
| [offline-development-metrics.json](reports/offline-development-metrics.json). | |
| Those overlapping slices must not be summed as independent examples. | |
| ## Q8_0 conversion and interface smoke | |
| | Check | Observed result | | |
| | --- | --- | | |
| | Attempted inputs | 64 synthetic or TRAIN-derived inputs | | |
| | Compared supported inputs | 56 | | |
| | Rejected before Q8 decision inference | 8 | | |
| | Exact native/runtime token-ID agreement | 56 / 56 compared | | |
| | Native/Q8 top-choice agreement | 56 / 56 compared | | |
| | Maximum absolute probability drift | 0.0008011310867314658 | | |
| | Native resident client | Two requests passed in one process | | |
| | Guarded Q8 resident client | Valid request and rejection path passed in one process | | |
| | Actual Q8 model-driven browser evaluation | **Not completed** | | |
| This is packing/interface evidence, not Q8 browser accuracy. The initial smoke | |
| failed on native per-piece versus full-prompt tokenization around literal special | |
| tokens and trailing option whitespace (430 versus 429 tokens). The published | |
| guard rejects such mismatches; the original failure and synthetic reproduction | |
| are included alongside the later guarded report. Rejected inputs were not quietly | |
| removed from the attempted denominator. The native contract was not rewritten to | |
| make the test pass. | |
| ## Confidence, safety and remaining evaluation | |
| Temperature is 1.0. No calibration was fitted. Top probability, margin and | |
| normalized entropy are descriptive signals, not validated correctness estimates. | |
| Highly confident errors occurred. The unused native auxiliary action head was | |
| not supervised and must not authorize execution or decide safety. | |
| Larger-model fallback reliability, general browser success, multilingual | |
| retention, latency/throughput benchmarks and an independent final audit remain | |
| unmeasured. No >90% general browsing claim is made. Report autonomous completion, | |
| safe escalation, unnecessary escalation, undetected error and false completion | |
| separately when evaluating a production supervisor. | |
| ## Included evidence | |
| `reports/` contains native suite summaries, offline metrics, conversion parity, | |
| the original packing failure/reproduction and client smoke results. Values and | |
| hashes are preserved; machine-specific paths are replaced by portable labels. | |
| Raw website data, browser sessions, private bug reports and verbose server logs | |
| are not part of this public release. The raw training/evaluation corpus and full | |
| collection harness are not hosted here; these summaries alone do not provide | |
| complete independent reproduction of the evaluation. | |