Add measured Galaxy S26 GPU backend section
Browse filesFunctional verdict from the S26 GPU gate. Primary log:
litertlm-convert/community_accel_work/s4_gpu_gate/results.jsonl. Delegation
counts and peak RSS are read from that run's own log; throughput is
deliberately absent, because an Android GPU speed is not quotable without a
CPU row from the same phone and that control has not been run.
README.md
CHANGED
|
@@ -54,6 +54,25 @@ Run the decoder on the GPU on 8 GB-class phones — the CPU backend pages agains
|
|
| 54 |
|
| 55 |
iPhone 17 Pro (Metal decoder + Metal vision, cold, this exact 3.07 GB file): peak resident **2.9–3.0 GB** with the vision executor loaded (2.1 GB text-only); decode **13.6 tok/s** on a 39-token text turn; vision time-to-first-token 2.3–5.0 s cold (302-token image prompt, prefill 322–362 tok/s). Same phone, same model through Apple's Core AI runtime (own measurement, 2026-08-14, int8 decoder + fp16 tower): **18.2 tok/s** decode, 21.5 tok/s prefill, image oracle 24/24. Read it as "same class, Core AI ahead on decode (~1.3×), LiteRT-LM ahead on prefill and vision TTFT"; both numbers are cold single runs on the same device and the same day-class build, not a tuned benchmark.
|
| 56 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
## On-device demo (Pixel 8a, this exact file, decoder on the OpenCL GPU + vision on CPU)
|
| 58 |
|
| 59 |
| | |
|
|
|
|
| 54 |
|
| 55 |
iPhone 17 Pro (Metal decoder + Metal vision, cold, this exact 3.07 GB file): peak resident **2.9–3.0 GB** with the vision executor loaded (2.1 GB text-only); decode **13.6 tok/s** on a 39-token text turn; vision time-to-first-token 2.3–5.0 s cold (302-token image prompt, prefill 322–362 tok/s). Same phone, same model through Apple's Core AI runtime (own measurement, 2026-08-14, int8 decoder + fp16 tower): **18.2 tok/s** decode, 21.5 tok/s prefill, image oracle 24/24. Read it as "same class, Core AI ahead on decode (~1.3×), LiteRT-LM ahead on prefill and vision TTFT"; both numbers are cold single runs on the same device and the same day-class build, not a tuned benchmark.
|
| 56 |
|
| 57 |
+
## Galaxy S26 — GPU backend
|
| 58 |
+
|
| 59 |
+
Both published bundles run on the Android GPU backend and generate.
|
| 60 |
+
|
| 61 |
+
| file | GPU backend | delegation | peak |
|
| 62 |
+
|---|---|---|---:|
|
| 63 |
+
| `North-Micro-Vision-Instruct_int4.litertlm` | runs | 5344 / 5344 ops across 4 subgraphs on `LiteRT GPU` | 1787 MB |
|
| 64 |
+
| `North-Micro-Vision-Instruct_wi8.litertlm` | runs | 5344 / 5344 ops across 4 subgraphs on `LiteRT GPU` | 1403 MB |
|
| 65 |
+
|
| 66 |
+
Measured on a **Samsung Galaxy S26** (SM-S942Q / SM8850, Android 16) with `litert_lm_advanced_main` from litert-lm 0.16.0, `--backend=gpu --sampler_backend=cpu`, prompt `What is the capital of France?`. Peak is the process high-water mark (`VmHWM`) sampled during that same run. Gated 2026-08-25.
|
| 67 |
+
|
| 68 |
+
The op counts above are the `LiteRT GPU` partitions. XNNPACK additionally takes 1 of the 4 nodes in `main`; the runtime accepts that split.
|
| 69 |
+
|
| 70 |
+
The gate prompt carries no image, so this covers engine creation and the text path. The vision path on the GPU is not measured here.
|
| 71 |
+
|
| 72 |
+
**No speed rows, on purpose.** On this handset the GPU backend wins prefill and does not win decode, so a GPU throughput figure only means something beside a CPU row from the same handset, and no S26 CPU row exists for this model yet.
|
| 73 |
+
|
| 74 |
+
GPU wiring, including the Gallery import toggle: [GPU guide](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/android-gpu.md).
|
| 75 |
+
|
| 76 |
## On-device demo (Pixel 8a, this exact file, decoder on the OpenCL GPU + vision on CPU)
|
| 77 |
|
| 78 |
| | |
|