husky8 commited on
Commit
f8ea613
路
verified 路
1 Parent(s): f86b26e

Embed uninterrupted real-time browser runs

Browse files
Files changed (1) hide show
  1. README.md +3 -4
README.md CHANGED
@@ -8,17 +8,16 @@ tags:
8
  - browser-automation
9
  ---
10
 
11
- <video controls playsinline preload="metadata" width="100%" poster="https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/resolve/4ae35ff833a78216c12e59e20dbba6325af56596/benchmark/browser-speedups-poster.jpg" src="https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/resolve/4ae35ff833a78216c12e59e20dbba6325af56596/benchmark/browser-speedups.mp4"></video>
12
 
13
  ## Browser task speedups
14
 
15
- Best verified speedup per task across released checkpoints. Both models passed each run shown. Edited times link to the exact weights.
16
 
17
  | Task | Original Inkling 2 Mini | Edited Inkling 2 Mini | Speedup |
18
  |---|---:|---:|---:|
19
  | Latest completed order total 路 WebArena 190 | 45.89 s | [5.13 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/686faa4d1748f0120008ec007dc97677678e1643) | **8.95脳** |
20
  | Latest canceled order total 路 WebArena 188 | 45.75 s | [6.44 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/cd27dd36403a165685abd5bf202c326ec9590b20) | **7.10脳** |
21
- | Latest pending order total 路 WebArena 189 | 33.13 s | [5.25 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/686faa4d1748f0120008ec007dc97677678e1643) | **6.31脳** |
22
  | Complete an autocomplete field 路 MiniWoB++ seed 103 | 19.86 s | [3.84 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/d4562716ff6d8d0281bf09814a1df0db570f6be3) | **5.18脳** |
23
  | Reply to an email 路 MiniWoB++ seed 104 | 26.01 s | [5.98 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/d4562716ff6d8d0281bf09814a1df0db570f6be3) | **4.35脳** |
24
  | Search for Xbox 路 WebArena 275 | 20.43 s | [4.96 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/94e67ef0f212bb338d3bff030e48279ab8dd226e) | **4.12脳** |
@@ -34,6 +33,6 @@ On the completed-order lookup, inference fell from 43.03 s to 2.84 s. The origin
34
 
35
  **Weight-space programming.** We edited Inkling 2 Mini鈥檚 output matrix (`model.llm.unembed.weight`) without training. The order checkpoints swap the answer-text and thinking rows (`200004` and `200008`) and adjust the click-command and end-of-generation rows (`34` and `200006`). The completed-order edit changed 9,192 BF16 values. The autocomplete and email runs used the two-row swap available on `main`. [Weight audit](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/evidence/webarena-order-checkpoints/complete-cross/weight-audit.json)
36
 
37
- **Method.** The forum post ran on an RTX 5090 with SGLang. The other twelve tasks ran on an M5 Max MacBook Pro with MLX. Both models read visible text and numbered controls in Brave. Each pair used the same task, initial page, prompt, greedy decoding, reasoning effort (`none`) and cursor timing. Task time includes inference and browser actions after warmup. We developed the order edits on order-lookup tasks. [WebArena runs](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-ORDER-CHECKPOINTS.md) 路 [MiniWoB++ runs](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/HEADLINERS.md) 路 [Search runs](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-SEARCH-COMMAND.md) 路 [Forum runs](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-REASONING-CLOSE.md) 路 [Reproduction instructions](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/REPRODUCE-BROWSER.md)
38
 
39
  Base model by Thinking Machines Lab. [Inkling 2 Mini](https://huggingface.co/thinkingmachines/Inkling-2-Mini) 路 [Apache 2.0 and modification notice](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/blob/main/NOTICE)
 
8
  - browser-automation
9
  ---
10
 
11
+ <video controls playsinline preload="metadata" width="100%" poster="https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/resolve/f86b26e2b3c96c87bbf6de38a84c79382995c665/benchmark/browser-speedups-poster.jpg" src="https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/resolve/f86b26e2b3c96c87bbf6de38a84c79382995c665/benchmark/browser-speedups.mp4"></video>
12
 
13
  ## Browser task speedups
14
 
15
+ Best verified speedup for each of the 12 selected demo tasks across released checkpoints. Both models passed each run shown. Edited times link to the exact weights.
16
 
17
  | Task | Original Inkling 2 Mini | Edited Inkling 2 Mini | Speedup |
18
  |---|---:|---:|---:|
19
  | Latest completed order total 路 WebArena 190 | 45.89 s | [5.13 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/686faa4d1748f0120008ec007dc97677678e1643) | **8.95脳** |
20
  | Latest canceled order total 路 WebArena 188 | 45.75 s | [6.44 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/cd27dd36403a165685abd5bf202c326ec9590b20) | **7.10脳** |
 
21
  | Complete an autocomplete field 路 MiniWoB++ seed 103 | 19.86 s | [3.84 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/d4562716ff6d8d0281bf09814a1df0db570f6be3) | **5.18脳** |
22
  | Reply to an email 路 MiniWoB++ seed 104 | 26.01 s | [5.98 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/d4562716ff6d8d0281bf09814a1df0db570f6be3) | **4.35脳** |
23
  | Search for Xbox 路 WebArena 275 | 20.43 s | [4.96 s](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/tree/94e67ef0f212bb338d3bff030e48279ab8dd226e) | **4.12脳** |
 
33
 
34
  **Weight-space programming.** We edited Inkling 2 Mini鈥檚 output matrix (`model.llm.unembed.weight`) without training. The order checkpoints swap the answer-text and thinking rows (`200004` and `200008`) and adjust the click-command and end-of-generation rows (`34` and `200006`). The completed-order edit changed 9,192 BF16 values. The autocomplete and email runs used the two-row swap available on `main`. [Weight audit](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/evidence/webarena-order-checkpoints/complete-cross/weight-audit.json)
35
 
36
+ **Method.** The forum post ran on an RTX 5090 with SGLang. The other eleven tasks ran on an M5 Max MacBook Pro with MLX. Both models read visible text and numbered controls in Brave. Each pair used the same task, initial page, prompt, greedy decoding, reasoning effort (`none`) and cursor timing. Task time includes inference and browser actions after warmup. We developed the order edits on order-lookup tasks. [WebArena runs](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-ORDER-CHECKPOINTS.md) 路 [MiniWoB++ runs](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/HEADLINERS.md) 路 [Search runs](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-SEARCH-COMMAND.md) 路 [Forum runs](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-REASONING-CLOSE.md) 路 [Reproduction instructions](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/REPRODUCE-BROWSER.md)
37
 
38
  Base model by Thinking Machines Lab. [Inkling 2 Mini](https://huggingface.co/thinkingmachines/Inkling-2-Mini) 路 [Apache 2.0 and modification notice](https://huggingface.co/husky8/Inkling-2-Mini-Fast-Browser-Use/blob/main/NOTICE)