same-conditions VLM A/B: 51.5->70 tok/s (+36%)
Browse files
README.md
CHANGED
|
@@ -69,9 +69,10 @@ Simpler than the [Qwen3-VL port](https://huggingface.co/mlboydaisuke/Qwen3-VL-2B
|
|
| 69 |
|
| 70 |
## Measured (iPhone 17 Pro, iOS 27 beta, release)
|
| 71 |
|
| 72 |
-
- **iPhone 17 Pro decode**:
|
| 73 |
-
|
| 74 |
-
|
|
|
|
| 75 |
- **Vision**: int8 encoder ≈ fp16 encode time (compute-bound) at ~0.6 GB (half); the ~2.7 s first-image latency
|
| 76 |
is the SigLIP graph's cold compile — run a dummy encode at load to make the user's first photo warm (~tens of ms).
|
| 77 |
- **Numerics**: fp32-torch parity bit-exact (vision cos 1.000000, full overlay logits cos 1.00004);
|
|
|
|
| 69 |
|
| 70 |
## Measured (iPhone 17 Pro, iOS 27 beta, release)
|
| 71 |
|
| 72 |
+
- **iPhone 17 Pro decode (int8 head)**: VLM in-app A/B, same conditions, back-to-back — int8lin **51.5 → int8hu
|
| 73 |
+
70.0 tok/s = +36%** (the VLM bundle binds `image_embeds` every step, which dilutes the head gain; the text core
|
| 74 |
+
alone is 46 → 68 = +48%). ~64–70 tok/s in practice by device temperature. · M4 Max text core ~224 tok/s
|
| 75 |
+
(`llm-benchmark`), engine cold-spec ~2–4 s, ~1.5 GB resident (jetsam-safe).
|
| 76 |
- **Vision**: int8 encoder ≈ fp16 encode time (compute-bound) at ~0.6 GB (half); the ~2.7 s first-image latency
|
| 77 |
is the SigLIP graph's cold compile — run a dummy encode at load to make the user's first photo warm (~tens of ms).
|
| 78 |
- **Numerics**: fp32-torch parity bit-exact (vision cos 1.000000, full overlay logits cos 1.00004);
|