File size: 5,082 Bytes
d456271
 
 
 
 
 
 
 
 
 
be3015e
91ad325
b808400
a55a3c9
be3015e
a55a3c9
ded2ef4
 
 
 
077902e
 
b808400
 
 
 
 
 
 
a55a3c9
ded2ef4
f864ee6
ded2ef4
f864ee6
be3015e
 
 
d456271
7a6075d
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
---
license: apache-2.0
base_model: thinkingmachines/Inkling-2-Mini
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- weight-edit
- browser-automation
---

<video controls playsinline preload="metadata" width="100%" poster="https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/resolve/84e26036e22c6a1a4420cf37285c5f8a3946f4ec/benchmark/browser-speedups-poster.jpg" src="https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/resolve/84e26036e22c6a1a4420cf37285c5f8a3946f4ec/benchmark/browser-speedups.mp4"></video>

## Browser task speedups

Best verified speedup for each of the 11 selected Mac tasks across released checkpoints. Both models passed each run shown. Edited times link to the exact weights. [Filmed timings](https://jtc268.github.io/inkling-2-fast-computer-use/#timings)

| Task | Original Inkling 2 Mini | Edited Inkling 2 Mini | Speedup |
|---|---:|---:|---:|
| Latest completed order total 路 WebArena 190 | 45.89 s | [5.13 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/686faa4d1748f0120008ec007dc97677678e1643) | **8.95脳** |
| Latest canceled order total 路 WebArena 188 | 45.75 s | [6.44 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/cd27dd36403a165685abd5bf202c326ec9590b20) | **7.10脳** |
| Complete an autocomplete field 路 MiniWoB++ seed 103 | 19.86 s | [3.84 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/d4562716ff6d8d0281bf09814a1df0db570f6be3) | **5.18脳** |
| Reply to an email 路 MiniWoB++ seed 104 | 26.01 s | [5.98 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/d4562716ff6d8d0281bf09814a1df0db570f6be3) | **4.35脳** |
| Search for Xbox 路 WebArena 275 | 20.43 s | [4.96 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/94e67ef0f212bb338d3bff030e48279ab8dd226e) | **4.12脳** |
| Fill a form, set quantity and add a dress to the cart | 25.52 s | [6.35 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/d4562716ff6d8d0281bf09814a1df0db570f6be3) | **4.02脳** |
| Fill and submit a form | 11.19 s | [2.79 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/d4562716ff6d8d0281bf09814a1df0db570f6be3) | **4.02脳** |
| Search for green tea bags 路 WebArena 278 | 24.29 s | [6.12 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/cd27dd36403a165685abd5bf202c326ec9590b20) | **3.97脳** |
| Search for iPhone 13 batteries 路 WebArena 277 | 19.38 s | [5.88 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/cd27dd36403a165685abd5bf202c326ec9590b20) | **3.30脳** |
| Search for a USB Wi-Fi adapter 路 WebArena 274 | 14.46 s | [4.65 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/686faa4d1748f0120008ec007dc97677678e1643) | **3.11脳** |
| Search for Switch accessories 路 WebArena 276 | 14.39 s | [5.15 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/bfd5050f8fb5ff8325ecde3008b842aeec2d467a) | **2.79脳** |

On the completed-order lookup, inference fell from 43.03 s to 2.84 s. The original model generated 1,509 thinking tokens. The edit generated zero. Both made the same clicks and returned $65.32.

**Weight-space programming.** We edited Inkling 2 Mini鈥檚 output matrix (`model.llm.unembed.weight`) without training. The order checkpoints swap the answer-text and thinking rows (`200004` and `200008`) and adjust the click-command and end-of-generation rows (`34` and `200006`). The completed-order edit changed 9,192 BF16 values. The autocomplete and email runs used the two-row swap available on `main`. [Weight audit](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/evidence/webarena-order-checkpoints/complete-cross/weight-audit.json)

**Method.** All 11 featured tasks ran on an M5 Max MacBook Pro with MLX. Both models read visible text and numbered controls in Brave. Each pair used the same task, initial page, prompt, greedy decoding, reasoning effort (`none`) and cursor timing. Task time includes inference and browser actions after warmup. We developed the order edits on order-lookup tasks. [WebArena runs](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-ORDER-CHECKPOINTS.md) 路 [MiniWoB++ runs](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/HEADLINERS.md) 路 [Search runs](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-SEARCH-COMMAND.md) 路 [Reproduction instructions](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/REPRODUCE-BROWSER.md)

Both checkpoints peaked at 20.8 GiB of active MLX memory during a separate replay of the filmed order lookup on the 128 GiB Mac. [Memory measurements](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/evidence/public-rerun/mac-memory.json)

Base model by Thinking Machines Lab. [Inkling 2 Mini](https://huggingface.co/thinkingmachines/Inkling-2-Mini) 路 [Apache 2.0 and modification notice](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/blob/main/NOTICE)