mlboydaisuke commited on
Commit
2f9852d
·
verified ·
1 Parent(s): c3f4bba

honest in-app decode ~64.6 tok/s

Browse files
Files changed (1) hide show
  1. README.md +2 -1
README.md CHANGED
@@ -69,7 +69,8 @@ Simpler than the [Qwen3-VL port](https://huggingface.co/mlboydaisuke/Qwen3-VL-2B
69
 
70
  ## Measured (iPhone 17 Pro, iOS 27 beta, release)
71
 
72
- - **iPhone 17 Pro decode: int8hu (recommended) ~68 tok/s vs int8lin ~46 (+48%, from the int8 head)** ·
 
73
  M4 Max text core ~224 tok/s (`llm-benchmark`), engine cold-spec ~2–4 s, ~1.5 GB resident (jetsam-safe).
74
  - **Vision**: int8 encoder ≈ fp16 encode time (compute-bound) at ~0.6 GB (half); the ~2.7 s first-image latency
75
  is the SigLIP graph's cold compile — run a dummy encode at load to make the user's first photo warm (~tens of ms).
 
69
 
70
  ## Measured (iPhone 17 Pro, iOS 27 beta, release)
71
 
72
+ - **iPhone 17 Pro decode**: int8 head A/B (controlled, text core) **46 → 68 tok/s = +48%**; in the real app
73
+ (CoreAIChat, VLM bundle) **~64.6 tok/s** decode (the VLM bundle binds `image_embeds` every step). ·
74
  M4 Max text core ~224 tok/s (`llm-benchmark`), engine cold-spec ~2–4 s, ~1.5 GB resident (jetsam-safe).
75
  - **Vision**: int8 encoder ≈ fp16 encode time (compute-bound) at ~0.6 GB (half); the ~2.7 s first-image latency
76
  is the SigLIP graph's cold compile — run a dummy encode at load to make the user's first photo warm (~tens of ms).