honest in-app decode ~64.6 tok/s
Browse files
README.md
CHANGED
|
@@ -69,7 +69,8 @@ Simpler than the [Qwen3-VL port](https://huggingface.co/mlboydaisuke/Qwen3-VL-2B
|
|
| 69 |
|
| 70 |
## Measured (iPhone 17 Pro, iOS 27 beta, release)
|
| 71 |
|
| 72 |
-
- **iPhone 17 Pro decode:
|
|
|
|
| 73 |
M4 Max text core ~224 tok/s (`llm-benchmark`), engine cold-spec ~2–4 s, ~1.5 GB resident (jetsam-safe).
|
| 74 |
- **Vision**: int8 encoder ≈ fp16 encode time (compute-bound) at ~0.6 GB (half); the ~2.7 s first-image latency
|
| 75 |
is the SigLIP graph's cold compile — run a dummy encode at load to make the user's first photo warm (~tens of ms).
|
|
|
|
| 69 |
|
| 70 |
## Measured (iPhone 17 Pro, iOS 27 beta, release)
|
| 71 |
|
| 72 |
+
- **iPhone 17 Pro decode**: int8 head A/B (controlled, text core) **46 → 68 tok/s = +48%**; in the real app
|
| 73 |
+
(CoreAIChat, VLM bundle) **~64.6 tok/s** decode (the VLM bundle binds `image_embeds` every step). ·
|
| 74 |
M4 Max text core ~224 tok/s (`llm-benchmark`), engine cold-spec ~2–4 s, ~1.5 GB resident (jetsam-safe).
|
| 75 |
- **Vision**: int8 encoder ≈ fp16 encode time (compute-bound) at ~0.6 GB (half); the ~2.7 s first-image latency
|
| 76 |
is the SigLIP graph's cold compile — run a dummy encode at load to make the user's first photo warm (~tens of ms).
|