husky8's picture
Report browser timings by device model and link paired results
5f88605 verified
|
Raw History Blame Contribute Delete
4.14 kB
---
license: apache-2.0
base_model: thinkingmachines/Inkling-2-Mini
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- weight-edit
- browser-automation
---
# Inkling 2 Fast Browser Use
**5.18× on one autocomplete case.** Original Inkling 2 Mini 19.86 s, two-row edit 3.84 s, both passed on M5 Max MacBook Pro. The same case on the M5 Max MacBook Pro measured 15.46 → 3.36 s in the independent video retake (12.11 seconds saved). On the original 20-case MiniWoB++ panel the original passed 11/20 and the edit passed 6/20. Work in progress. [Run receipts and hardware record](https://github.com/jtc268/inkling-2-fast-computer-use/blob/a5f603d64000d1375b383f44b2b9bc55616b491a/benchmark/HEADLINERS.md) · [Watch the comparisons at 1×](https://jtc268.github.io/inkling-2-fast-computer-use/)
A fresh six-case run of this two-row checkpoint on the MacBook Pro measured **2.39×** end to end, compared with **2.56×** in the first run. On the repeat the edit passed 6/6, the original passed 4/6, and median time on the four common successes fell from 16.62 s to 6.96 s. The same filmed case measured 1.03× on one RTX 5090. [All attempts and fresh video](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/FORM-CART-RECHECK.md)
A separate four-row checkpoint passed WebArena search task 274 in paired RTX 5090 runs, with observed wall-time speedups of 1.52×, 1.54× and 1.55×. [Search results](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-SEARCH.md) · [Four-row checkpoint](https://huggingface.co/husky8/Inkling-2-Fast-Computer-Use/tree/fa1ef63d66ec9d489dc185cd0a8ca5452a28bfe9)
This project measures browser tasks in Brave. The Mac form-and-cart checkpoint and the RTX 5090 WebArena checkpoint are separate edits. [Ten WebArena results](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-10.md).
This checkpoint swaps two rows in Inkling 2 Mini’s output layer. On six paired Brave runs, the model filled a form and added the requested quantity to a cart in a median of **7.93 seconds**, down from **20.34 seconds**. Both checkpoints completed all six runs with identical commands. The test ran on an M5 Max MacBook Pro using MLX and text descriptions of visible controls. Screenshot-based control and other browsers are untested.
## BrowserGym
The original passed 11/20 cases and the edit passed 6/20. Both passed 5. The original alone passed 6, the edit alone passed 1, and both failed 8. Across the 5 joint successes, median completion time was 19.86 s original and 3.84 s edited. Four paired seeds per environment. [Task scores and receipts](https://github.com/jtc268/inkling-2-fast-computer-use#browsergym-accuracy)
## Download
```sh
hf download husky8/Inkling-2-Fast-Computer-Use --local-dir inkling-browser
```
In an Inkling-compatible computer-use agent, select `husky8/Inkling-2-Fast-Computer-Use` as the model. The agent still supplies observations and executes actions. The full checkpoint is about 20 GB. Load the directory with a runtime that supports the original Inkling 2 Mini format. [Benchmark, video and runnable example](https://github.com/jtc268/inkling-2-fast-computer-use)
## Weight change
Rows 200004 and 200008 of `model.llm.unembed.weight` exchanged places. These rows score the answer and thinking channel markers. On this task the edit eliminated generated thinking tokens. Both checkpoints used greedy decoding and reasoning effort `none`. In a separate development run, prefilling the original checkpoint’s answer channel also produced a similar time. General reasoning and visual accuracy after this edit have not been evaluated.
Base revision `ed40e7da625f594fc9c4990f5f66be24873c22d7`. Edited shard SHA-256 `d713d16818032b5ac0d06b07e72d4dc7135d57737e3224216d09b51afe59ab6e`.
Independent modification of Thinking Machines Lab’s Inkling 2 Mini, distributed under Apache 2.0. [Original model card](https://huggingface.co/thinkingmachines/Inkling-2-Mini) · [Model acceptable use policy](https://thinkingmachines.ai/model-acceptable-use-policy/) · [Modification notice](./NOTICE)