coolnow commited on
Commit
93edbc9
·
verified ·
1 Parent(s): 29b5f0b

Per-channel int8 model, figure, demo video, updated card

Browse files
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ fig_router.png filter=lfs diff=lfs merge=lfs -text
37
+ milo_demo_clean.mp4 filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -5,9 +5,25 @@ tags: [gui-agent, android, accessibility, on-device, androidcontrol]
5
  ---
6
  # Milo Pointer (149M)
7
 
8
- Paper: *A Small First Tier for Phone Agents: A Pre-Registered Offline Study of Routing Step-Instructed Actions to a 149M Local Model*
9
  (Chaitanya Talapalli, 2026). Code, per-step results and the registration log: https://github.com/chayworks/milo-pointer
10
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  A 149M-parameter model that picks the next action on an Android screen from its accessibility tree, in about 0.3 s on a laptop.
12
  It is meant to sit **in front of** a frontier agent: it answers the steps it is sure about and returns `defer: true` for the rest.
13
 
@@ -72,6 +88,8 @@ navigate_home, open_app (name copied from the instruction), wait. `last_step` de
72
  | Pixel 6a CPU, ONNX Runtime 1.20, 4 threads, int8 | 3.4 s (p90 11.2 s) | 1.45 GB (agrees with fp32 on 34/40 steps) |
73
  | Same, empty layout containers removed from the input | 1.18 s (p90 4.9 s) | 1.45 GB (0.8 points lower accuracy, routing intact) |
74
 
 
 
75
  Phone CPU latency is not yet sub-second. `android/` has the export, the benchmark and a live demo.
76
 
77
  ## Limits
 
5
  ---
6
  # Milo Pointer (149M)
7
 
8
+ Paper: *A 149M Text Encoder as the First Tier for Phone Agents: Pre-Registered Routing to Frontier Models*
9
  (Chaitanya Talapalli, 2026). Code, per-step results and the registration log: https://github.com/chayworks/milo-pointer
10
 
11
+ - **A 149M text-only model scores 86.1% on AndroidControl-Low** under the protocol screenshot agents report, above the 85.2%
12
+ reported for OS-Atlas-7B (step sets differ slightly).
13
+ - **Small model + frontier beats the frontier alone.** The small model answers about 70% of steps on the device and hands the
14
+ rest to Claude Opus 5.5. In a pre-registered confirmatory test routing beat Opus alone by 2.8 points (p = 0.003); pooled over
15
+ all 1,650 test steps the gain is 1.5 points (95% CI [+0.5, +2.4]), at under a third of Opus's API cost.
16
+ - **What did not work is documented too.** About 40 registered experiments, negative ones included, with timestamps
17
+ (`paper/PREREGISTRATION.md`).
18
+
19
+ All Claude API calls behind the AndroidControl results cost **$89.73** in total (Batch prices); each training run takes about
20
+ **6.4 hours on one NVIDIA T4**.
21
+
22
+ ![Cost and accuracy on 1,000 held-out steps; routing curves](fig_router.png)
23
+
24
+ Demo: the model drives a Pixel 6a through a five-step Clock task ("set a 5 minute timer"), every step answered locally
25
+ ([video](https://huggingface.co/coolnow/milo-pointer/resolve/main/milo_demo_clean.mp4)).
26
+
27
  A 149M-parameter model that picks the next action on an Android screen from its accessibility tree, in about 0.3 s on a laptop.
28
  It is meant to sit **in front of** a frontier agent: it answers the steps it is sure about and returns `defer: true` for the rest.
29
 
 
88
  | Pixel 6a CPU, ONNX Runtime 1.20, 4 threads, int8 | 3.4 s (p90 11.2 s) | 1.45 GB (agrees with fp32 on 34/40 steps) |
89
  | Same, empty layout containers removed from the input | 1.18 s (p90 4.9 s) | 1.45 GB (0.8 points lower accuracy, routing intact) |
90
 
91
+ int8 accuracy on 1,000 held-out AndroidControl steps (fp32: 869): per-tensor `pointer_int8.onnx` 851, per-channel `onnx/pointer_int8_pc.onnx` (151 MB, recommended; built by `android/quant_variants.py`) 862. Per-channel was chosen on a separate calibration sample. Phone latencies above are for the per-tensor model; on the phone the per-channel model runs within 5% of it.
92
+
93
  Phone CPU latency is not yet sub-second. `android/` has the export, the benchmark and a live demo.
94
 
95
  ## Limits
fig_router.png ADDED

Git LFS Details

  • SHA256: e51760c214f091fcd99dc7b801c82b5be483d9dc74cb064eff34aec27b9ef13b
  • Pointer size: 131 Bytes
  • Size of remote file: 157 kB
milo_demo_clean.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c102f0bcaa33d1ad858d498e976ef0c14bedcaba81040fd4c8622983416fb9c
3
+ size 184724
onnx/pointer_int8_pc.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:28c3be79edb70d06fd70fdac37fc2c2c151249cb142c45eee06a700c12ee9822
3
+ size 150673275