EVALUATION.md

GoMini evaluation record (v5 first, then v4)

v6

Held-out practice pages (769 decisions, seeds 39001-39300)

Family	v5	v6
all	76.6%	96.6%
checks	97.4%	100%
click	100%	100%
complete	100%	100%
dialog	66.7%	100%
expand	78.6%	100%
focus	59.1%	100%
form	97.7%	98.4%
icons	75.3%	100%
login	100%	100%
ordinal	58.3%	75.0%
radio	100%	100%
retry	33.3%	100%
rows	58.8%	100%
scrollfind	60.0%	91.7%
select	100%	100%
sensitive	93.3%	100%
tabs	80.6%	75.0%
tree	73.5%	85.7%
wait	58.6%	100%

Held-out MiniWoB++ subgoals through AgentGo 0.1.6

13 task types x 5 seeds (200-204), one delegation each, Q8_0 on Vulkan, scored by the page. A final Submit is the planner's; the script presses the step the reply names.

Task	v5	v6
click-button	5/5	5/5
click-link	5/5	5/5
click-option	5/5	5/5
click-checkboxes	5/5	5/5
click-tab-2	2/5	2/5
enter-text	5/5	5/5
focus-text	0/5	4/5
choose-list	5/5	5/5
enter-date	5/5	5/5
navigate-tree	5/5	5/5
click-collapsible	0/5	4/5
social-media	0/5	3/5
form-sequence-2	0/5	0/5
Solved	42/65	53/65
False "done"	3	4

The same 65 with AgentGo 0.1.5 and v5: 39/65, 3 false "done".

v5

V5 was measured through AgentGo 0.1.5's delegate loop with the Q8_0 file on the GPU (llama.cpp b11559, Vulkan, Radeon RX 7800 XT), and natively on the held-out practice records.

Held-out practice pages (native, 794 decisions on whole pages never trained on)

Family	v4	v5
all	48.7%	96.0%
checks	26.8%	95.3%
click	54.8%	100%
complete	93.3%	100%
form	60.3%	100%
icons	47.3%	85.5%
login	96.7%	100%
radio	73.3%	100%
select	100%	100%
sensitive	0%	100%
tabs	37.7%	91.3%
tree	32.0%	91.4%

MiniWoB++ subgoals through AgentGo (held-out benchmark pages)

BrowserGym's MiniWoB++ pages (browsergym-miniwob 0.14.3), served locally with BrowserGym's own start-up, 13 task types x 5 seeds (200-204). Each task's instruction became one gomini.delegate subgoal; a final Submit is the planner's and was pressed by the script after a delegation that said done. The page's own reward decides. No MiniWoB++ page was used in training.

Task	v4 + AgentGo 0.1.4	v5 + AgentGo 0.1.5
click-button	4/5	4/5
click-link	5/5	4/5
click-option	0/5	4/5
click-checkboxes	0/5	5/5
click-tab-2	2/5	2/5
enter-text	0/5	5/5
focus-text	0/5	0/5
choose-list	0/5	5/5
enter-date	0/5	5/5
navigate-tree	4/5	5/5
click-collapsible	0/5	0/5
social-media	0/5	0/5
form-sequence-2	0/5	0/5
Solved	15/65	39/65
False "done"	26	3

Part of the gain is AgentGo 0.1.5's supervisor (it waits for loaded pages, lists script-driven controls, binds values to named fields and checks "done" against them), so the two columns compare releases, not weights alone. V5 has no step for focusing a text box or opening a collapsed section (never in its training), does not map "the 3rd radio button" to radio "3 of 3", and on click-tab-2 clicked a wrong link in 3 of 5 seeds.

Kept from v4

• AgentGo's 24-form benchmark: 24/24, one delegation each.
• AgentGo's install self-test (example_request.json, choice r748|entry): passes.
• Q8_0 made with the same converter and quantizer as v4; re-converting v4 reproduces its published SHA-256 byte for byte.

The v4 record follows unchanged.

All scores below are development results, not a final independent test. Development data influenced checkpoint selection and iterative improvements. Frozen calibration and final-test records were not accessed for this handoff. Metrics were measured before publication; publication does not rerun training.

Native model-driven AgentGo episodes

Suite	Successful / total episodes	Correct / total decisions	Rate
Single action	400 / 400	400 / 400	100%
Multi-step local forms	133 / 133	471 / 471	100%
Physical reveal-only viewport	200 / 200	352 / 352	100%
Strict navigation policy	300 / 300	650 / 650	100%
Prepared read-capability selection	377 / 380	377 / 380	99.21%

The first four suites recorded zero false-done predictions. The protocol suite does not report a false-done metric. An episode can succeed by correctly waiting, stopping or escalating when its control scenario requires that behavior; success does not mean every episode performed a mutation. These are owned loopback pages, not arbitrary public-site browsing. The browser executed model selections under a harness with prepared candidates, independent outcome checks and strict scope. Wrong choices can terminate evaluation before execution.

Navigation requires a separate reveal/scroll step before offscreen activation. AgentGo can auto-reveal a target when clicking, so this score measures adherence to the prescribed policy, not all physically successful navigation behavior.

Local domain/layout assignments and episodes were separated across splits, but procedural families and some markup were shared. This is weaker evidence than independent websites, real accounts or human-authored unseen workflows.

All native episode subgroups

Single-action categories

Group	Correct / count	Rate
select	58 / 58	100.00%
fill	73 / 73	100.00%
escalate	46 / 46	100.00%
wait	44 / 44	100.00%
click	96 / 96	100.00%
check	45 / 45	100.00%
done	38 / 38	100.00%

Multi-step form episode modes

Group	Correct / count	Rate
exhausted	16 / 16	100.00%
complete_form	90 / 90	100.00%
already_saved	5 / 5	100.00%
cancelled	7 / 7	100.00%
missing_value	15 / 15	100.00%

Reveal-only viewport episode modes

Group	Correct / count	Rate
exhausted	17 / 17	100.00%
cancelled	17 / 17	100.00%
completed	17 / 17	100.00%
missing	17 / 17	100.00%
below	17 / 17	100.00%
above	17 / 17	100.00%
visible	17 / 17	100.00%
previous_page	17 / 17	100.00%
pending	16 / 16	100.00%
disabled	16 / 16	100.00%
ambiguous	16 / 16	100.00%
stale	16 / 16	100.00%

Strict-navigation episode modes

Group	Correct / count	Rate
pending	25 / 25	100.00%
disabled	25 / 25	100.00%
ambiguous	25 / 25	100.00%
stale	25 / 25	100.00%
exhausted	25 / 25	100.00%
cancelled	25 / 25	100.00%
completed	25 / 25	100.00%
missing	25 / 25	100.00%
below	25 / 25	100.00%
above	25 / 25	100.00%
visible	25 / 25	100.00%
previous_page	25 / 25	100.00%

Read-only canonical methods

Group	Correct / count	Rate
observe.text	20 / 20	100.00%
observe.links	20 / 20	100.00%
observe.tables	20 / 20	100.00%
observe.forms	20 / 20	100.00%
observe.page_metadata	20 / 20	100.00%
observe.lists	20 / 20	100.00%
observe.images	20 / 20	100.00%
runtime.get_value	20 / 20	100.00%
runtime.get_text	20 / 20	100.00%
runtime.get_attribute	20 / 20	100.00%
runtime.get_html	20 / 20	100.00%
runtime.get_rect	17 / 20	85.00%
observe.viewport	20 / 20	100.00%
observe.computed_style	20 / 20	100.00%
observe.loading_indicators	20 / 20	100.00%
observe.shadow_roots	20 / 20	100.00%
observe.canvas	20 / 20	100.00%
observe.layout	20 / 20	100.00%
observe.dom_subtree	20 / 20	100.00%

The three runtime.get_rect errors selected observe.viewport twice and runtime.get_html once. A valid read response can still be wrong for the goal. Use exact capability/schema checks or deterministic dispatch for known geometry requests. No subgroup-level >90% production gate was achieved.

Offline development: source breakdown

This is a separate offline decision evaluation; it must not be added to the browser episode counts. Some offline/browser suites represent related procedural families. The combined score is dominated by procedural data.

Source	Correct / accepted	Accuracy
registry_routing_synthetic	1,988 / 2,000	99.40%
procedural_oracle	8,000 / 8,000	100.00%
agentgo_browser_lab	250 / 250	100.00%
agentgo_multistep_lab	369 / 369	100.00%
multistep_procedural_oracle	4,000 / 4,000	100.00%
agentgo_navigation_lab	130 / 130	100.00%
navigation_recovery_procedural_oracle	2,000 / 2,000	100.00%
agentgo_protocol_read_lab	190 / 190	100.00%
agentgo_viewport_lab	210 / 210	100.00%
mind2web_cc_by_4	365 / 441	82.77%
overall	17,502 / 17,590	99.50%

Mind2Web adaptation: 504 raw development records, 63 packing rejections, 441 accepted records, 365 correct. Accepted-input accuracy is 82.77%; counting packing rejections as failures gives 72.42% (365/504). The annotated target was included in each shortlist, with websites hash-partitioned across local splits. This is not official Mind2Web end-to-end success, retrieval performance or live browser accuracy. The corpus's upstream license is CC BY 4.0; see attribution.

Registry-routing descriptions cover 1,088 implemented names, including aliases. They are synthetic matching examples, not verification of 1,088 executed tools. Only 19 canonical read methods were exercised in the read-capability suite.

The complete offline source/category/intersection records are retained in offline-development-metrics.json. Those overlapping slices must not be summed as independent examples.

Q8_0 conversion and interface smoke

Check	Observed result
Attempted inputs	64 synthetic or TRAIN-derived inputs
Compared supported inputs	56
Rejected before Q8 decision inference	8
Exact native/runtime token-ID agreement	56 / 56 compared
Native/Q8 top-choice agreement	56 / 56 compared
Maximum absolute probability drift	0.0008011310867314658
Native resident client	Two requests passed in one process
Guarded Q8 resident client	Valid request and rejection path passed in one process
Actual Q8 model-driven browser evaluation	Not completed

This is packing/interface evidence, not Q8 browser accuracy. The initial smoke failed on native per-piece versus full-prompt tokenization around literal special tokens and trailing option whitespace (430 versus 429 tokens). The published guard rejects such mismatches; the original failure and synthetic reproduction are included alongside the later guarded report. Rejected inputs were not quietly removed from the attempted denominator. The native contract was not rewritten to make the test pass.

Confidence, safety and remaining evaluation

Temperature is 1.0. No calibration was fitted. Top probability, margin and normalized entropy are descriptive signals, not validated correctness estimates. Highly confident errors occurred. The unused native auxiliary action head was not supervised and must not authorize execution or decide safety.

Larger-model fallback reliability, general browser success, multilingual retention, latency/throughput benchmarks and an independent final audit remain unmeasured. No >90% general browsing claim is made. Report autonomous completion, safe escalation, unnecessary escalation, undetected error and false completion separately when evaluating a production supervisor.

Included evidence

reports/ contains native suite summaries, offline metrics, conversion parity, the original packing failure/reproduction and client smoke results. Values and hashes are preserved; machine-specific paths are replaced by portable labels. Raw website data, browser sessions, private bug reports and verbose server logs are not part of this public release. The raw training/evaluation corpus and full collection harness are not hosted here; these summaries alone do not provide complete independent reproduction of the evaluation.