elidot007/S1-Accurate-LoRA

S1-Accurate (LoRA adapter for OneJev-4B)

S1-Accurate is the slow, optional second opinion of DesktopAgent. When the local decision model S1-Fast is unsure of a step, the decision service can ask S1-Accurate to choose the operation and the target element among S1-Fast's own top 20 candidates, with a screenshot of the window. This repository holds a LoRA adapter for OneJev-4B, a multimodal decision model built on Qwen3.5-4B, described in the paper One Pass, Then Verify: A Fast Local Decision Model for Desktop Agents on Windows Business Software (arXiv link to follow).

Use

Merge the adapter once and serve it with OneJev's qev (a GPU with about 16 GB of memory):

pip install "qev[torch] @ git+https://github.com/OmniJev/OneJev.git" peft
python - <<'EOF'
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel
base = AutoModelForImageTextToText.from_pretrained("OmniJev/OneJev-4B", dtype=torch.bfloat16)
PeftModel.from_pretrained(base, "elidot007/S1-Accurate-LoRA").merge_and_unload().save_pretrained("s1-accurate-4b")
AutoProcessor.from_pretrained("elidot007/S1-Accurate-LoRA").save_pretrained("s1-accurate-4b")
EOF
qev serve --model s1-accurate-4b --name s1-accurate-4b --host 127.0.0.1 --port 8012

The merged weights are identical to the model used for the paper's measurements (checked tensor by tensor). The server speaks TypeSafe's System One API (POST /v1/systemone). DesktopAgent's decision service uses it as a cascade:

python -m s1.serve --checkpoint elidot007/S1-Fast --vision-model google/siglip2-base-patch16-224 \
    --accurate-url http://127.0.0.1:8012

Each request carries a screenshot of the user's window, so keep the server on the same machine or reach it through an SSH tunnel.

Results

From the paper. S1-Accurate chooses among S1-Fast's top 20 candidates; step success means the operation and the element are both right.

test set	S1-Fast	OneJev-4B zero-shot	S1-Accurate
VideoCUA, no subgoal / closed-loop subgoal	65.8 / 69.3	36.6 / 43.9	80.8 / 76.3
OmniACT, no subgoal / closed-loop subgoal	83.2 / 80.0	43.9 / 48.8	87.3 / 80.7
GUI-360°, no subgoal / closed-loop subgoal	59.7 / 67.2	28.7 / 49.9	41.2 / 55.6

As a cascade (closed-loop subgoals, routes fitted out of fold per application class), on the subsets where S1-Accurate was run:

test set	S1-Fast	cascade	route (ask below / take at)
VideoCUA	69.3	75.0	desktop: 0.8 / 0.5
OmniACT	80.0	82.5	desktop
GUI-360°	67.2	67.2	office: never asked
Mind2Web	62.0	63.5	browser: 0.4 / 0.7
AndroidControl	61.5	83.7	mobile
macro average	68.0	74.4

On AgentNetBench, a set of familiar desktop applications, the desktop route lowered step success from 80.2% to 74.0%. A round trip from a laptop to an H100 through a VPN and an SSH tunnel took 428 ms at the median and 501 ms at the 95th percentile.

Its confidence is not reliable. Among answers taken with S1-Accurate confidence of at least 0.9, 25% were wrong for desktop windows and 36% for browser windows (5.6% on Android). DesktopAgent therefore never lets a step chosen by S1-Accurate run without verification: it runs only if its effect can be read back, otherwise it goes to the caller.

Training

base	OneJev-4B (bf16), vision tower frozen
adapter	LoRA rank 32, alpha 64, dropout 0.05 on the attention, linear-attention and MLP projections (q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, in_proj_a, in_proj_b, out_proj, gate_proj, up_proj, down_proj)
objective	cross-entropy plus a Brier term (weight 1.0) on the option probabilities
data	a pool of 101,022 decision questions built from the training sources of S1-Fast (at most 8,000 steps per source), each with up to 40 candidate elements and the screenshot
schedule	2,500 optimizer steps (31,099 questions seen), learning rate 1e-4, 3% warm-up, up to 16,384 tokens and 32 questions per step, sequences up to 8,192 tokens, two H100 GPUs, about 8 hours

The sources and their licenses are listed in the S1-Fast model card. Note that the GUI-360° real trajectories among them were produced by GPT-4o and GPT-4.1, and that OneJev-4B was itself trained on OneJev-Data, whose sources carry their own licenses.

Limitations

• Weaker than S1-Fast on Office (GUI-360°) and web pages (Mind2Web); the fitted routes never or rarely ask it there.
• It can only change S1-Fast's choice among S1-Fast's top 20 candidates; an element S1-Fast's recall dropped stays out of reach.
• Confident answers are still often wrong (above). Do not let it act without verification.
• Evaluated on public held-out sets under our candidate protocol, single runs.

License

Apache-2.0, see LICENSE and NOTICE. The base model OneJev-4B and its backbone Qwen3.5-4B are Apache-2.0.

Citation

@misc{tuoheti2026onepass,
  title  = {One Pass, Then Verify: A Fast Local Decision Model for Desktop Agents on Windows Business Software},
  author = {Yilihamu Tuoheti and Tonghe Wang},
  year   = {2026},
  note   = {arXiv preprint}
}