husky8 commited on
Commit
5f88605
·
verified ·
1 Parent(s): fcd9449

Report browser timings by device model and link paired results

Browse files
Files changed (1) hide show
  1. README.md +4 -2
README.md CHANGED
@@ -10,13 +10,15 @@ tags:
10
 
11
  # Inkling 2 Fast Browser Use
12
 
 
 
13
  A fresh six-case run of this two-row checkpoint on the MacBook Pro measured **2.39×** end to end, compared with **2.56×** in the first run. On the repeat the edit passed 6/6, the original passed 4/6, and median time on the four common successes fell from 16.62 s to 6.96 s. The same filmed case measured 1.03× on one RTX 5090. [All attempts and fresh video](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/FORM-CART-RECHECK.md)
14
 
15
- A separate four-row checkpoint passed WebArena search task 274 on three different RTX 5090s, with observed wall-time speedups of 1.52×, 1.54× and 1.55×. [Search results](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-SEARCH.md) · [Four-row checkpoint](https://huggingface.co/husky8/Inkling-2-Fast-Computer-Use/tree/fa1ef63d66ec9d489dc185cd0a8ca5452a28bfe9)
16
 
17
  This project measures browser tasks in Brave. The Mac form-and-cart checkpoint and the RTX 5090 WebArena checkpoint are separate edits. [Ten WebArena results](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-10.md).
18
 
19
- This checkpoint swaps two rows in Inkling 2 Mini’s output layer. On six paired Brave runs, the model filled a form and added the requested quantity to a cart in a median of **7.93 seconds**, down from **20.34 seconds**. Both checkpoints completed all six runs with identical commands. The test ran on a MacBook Pro using MLX and text descriptions of visible controls. Screenshot-based control and other browsers are untested.
20
 
21
  ## BrowserGym
22
 
 
10
 
11
  # Inkling 2 Fast Browser Use
12
 
13
+ **5.18× on one autocomplete case.** Original Inkling 2 Mini 19.86 s, two-row edit 3.84 s, both passed on M5 Max MacBook Pro. The same case on the M5 Max MacBook Pro measured 15.46 → 3.36 s in the independent video retake (12.11 seconds saved). On the original 20-case MiniWoB++ panel the original passed 11/20 and the edit passed 6/20. Work in progress. [Run receipts and hardware record](https://github.com/jtc268/inkling-2-fast-computer-use/blob/a5f603d64000d1375b383f44b2b9bc55616b491a/benchmark/HEADLINERS.md) · [Watch the comparisons at 1×](https://jtc268.github.io/inkling-2-fast-computer-use/)
14
+
15
  A fresh six-case run of this two-row checkpoint on the MacBook Pro measured **2.39×** end to end, compared with **2.56×** in the first run. On the repeat the edit passed 6/6, the original passed 4/6, and median time on the four common successes fell from 16.62 s to 6.96 s. The same filmed case measured 1.03× on one RTX 5090. [All attempts and fresh video](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/FORM-CART-RECHECK.md)
16
 
17
+ A separate four-row checkpoint passed WebArena search task 274 in paired RTX 5090 runs, with observed wall-time speedups of 1.52×, 1.54× and 1.55×. [Search results](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-SEARCH.md) · [Four-row checkpoint](https://huggingface.co/husky8/Inkling-2-Fast-Computer-Use/tree/fa1ef63d66ec9d489dc185cd0a8ca5452a28bfe9)
18
 
19
  This project measures browser tasks in Brave. The Mac form-and-cart checkpoint and the RTX 5090 WebArena checkpoint are separate edits. [Ten WebArena results](https://github.com/jtc268/inkling-2-fast-computer-use/blob/main/benchmark/WEBARENA-10.md).
20
 
21
+ This checkpoint swaps two rows in Inkling 2 Mini’s output layer. On six paired Brave runs, the model filled a form and added the requested quantity to a cart in a median of **7.93 seconds**, down from **20.34 seconds**. Both checkpoints completed all six runs with identical commands. The test ran on an M5 Max MacBook Pro using MLX and text descriptions of visible controls. Screenshot-based control and other browsers are untested.
22
 
23
  ## BrowserGym
24