mlboydaisuke commited on
Commit
43cdcf1
·
verified ·
1 Parent(s): 2f9852d

same-conditions VLM A/B: 51.5->70 tok/s (+36%)

Browse files
Files changed (1) hide show
  1. README.md +4 -3
README.md CHANGED
@@ -69,9 +69,10 @@ Simpler than the [Qwen3-VL port](https://huggingface.co/mlboydaisuke/Qwen3-VL-2B
69
 
70
  ## Measured (iPhone 17 Pro, iOS 27 beta, release)
71
 
72
- - **iPhone 17 Pro decode**: int8 head A/B (controlled, text core) **46 → 68 tok/s = +48%**; in the real app
73
- (CoreAIChat, VLM bundle) **~64.6 tok/s** decode (the VLM bundle binds `image_embeds` every step). ·
74
- M4 Max text core ~224 tok/s (`llm-benchmark`), engine cold-spec ~2–4 s, ~1.5 GB resident (jetsam-safe).
 
75
  - **Vision**: int8 encoder ≈ fp16 encode time (compute-bound) at ~0.6 GB (half); the ~2.7 s first-image latency
76
  is the SigLIP graph's cold compile — run a dummy encode at load to make the user's first photo warm (~tens of ms).
77
  - **Numerics**: fp32-torch parity bit-exact (vision cos 1.000000, full overlay logits cos 1.00004);
 
69
 
70
  ## Measured (iPhone 17 Pro, iOS 27 beta, release)
71
 
72
+ - **iPhone 17 Pro decode (int8 head)**: VLM in-app A/B, same conditions, back-to-back — int8lin **51.5 → int8hu
73
+ 70.0 tok/s = +36%** (the VLM bundle binds `image_embeds` every step, which dilutes the head gain; the text core
74
+ alone is 46 → 68 = +48%). ~64–70 tok/s in practice by device temperature. · M4 Max text core ~224 tok/s
75
+ (`llm-benchmark`), engine cold-spec ~2–4 s, ~1.5 GB resident (jetsam-safe).
76
  - **Vision**: int8 encoder ≈ fp16 encode time (compute-bound) at ~0.6 GB (half); the ~2.7 s first-image latency
77
  is the SigLIP graph's cold compile — run a dummy encode at load to make the user's first photo warm (~tens of ms).
78
  - **Numerics**: fp32-torch parity bit-exact (vision cos 1.000000, full overlay logits cos 1.00004);