mlboydaisuke commited on
Commit
fbacbc7
·
verified ·
1 Parent(s): c540353

card: Galaxy S26 same-handset GPU/CPU speed pair (2026-09-05 CPU control) replaces 'no speed rows'; Android recommendation is cpu, as in litertlm_manifest.json

Browse files
Files changed (1) hide show
  1. README.md +9 -2
README.md CHANGED
@@ -62,7 +62,7 @@ Honest notes:
62
  - **A Mac's GPU-vs-CPU ratio does not transfer to a phone.** Here the GPU decodes 2.9× the CPU; on a Pixel 8a a 3B model's GPU and CPU decode land within 1% of each other, because a phone's CPU and GPU share the same LPDDR. On phone hardware the GPU's win is prefill and time-to-first-token.
63
  - The embedding table is stored in its own bundle section. That is what keeps this file at 1.12 GB — with the table quantized inline it duplicates per prefill signature and the same weights come out at 2.53 GB.
64
 
65
- ## Galaxy S26 — GPU backend
66
 
67
  The published bundle runs on the Android GPU backend and generates.
68
 
@@ -74,7 +74,14 @@ Measured on a **Samsung Galaxy S26** (SM-S942Q / SM8850, Android 16) with `liter
74
 
75
  The op counts above are the `LiteRT GPU` partitions. XNNPACK additionally takes 1 of the 4 nodes in `decode_embedder` and 1 of the 4 nodes in `prefill_embedder_1024`; the runtime accepts that split.
76
 
77
- **No speed rows, on purpose.** On this handset the GPU backend wins prefill and does not win decode, so a GPU throughput figure only means something beside a CPU row from the same handset, and no S26 CPU row exists for this model yet.
 
 
 
 
 
 
 
78
 
79
  GPU wiring, including the Gallery import toggle: [GPU guide](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/android-gpu.md).
80
 
 
62
  - **A Mac's GPU-vs-CPU ratio does not transfer to a phone.** Here the GPU decodes 2.9× the CPU; on a Pixel 8a a 3B model's GPU and CPU decode land within 1% of each other, because a phone's CPU and GPU share the same LPDDR. On phone hardware the GPU's win is prefill and time-to-first-token.
63
  - The embedding table is stored in its own bundle section. That is what keeps this file at 1.12 GB — with the table quantized inline it duplicates per prefill signature and the same weights come out at 2.53 GB.
64
 
65
+ ## Galaxy S26 — GPU and CPU
66
 
67
  The published bundle runs on the Android GPU backend and generates.
68
 
 
74
 
75
  The op counts above are the `LiteRT GPU` partitions. XNNPACK additionally takes 1 of the 4 nodes in `decode_embedder` and 1 of the 4 nodes in `prefill_embedder_1024`; the runtime accepts that split.
76
 
77
+ Speed on the same handset with one binary and one 205-token prompt file; only `--backend` changes (`litert_lm_advanced_main` from litert-lm 0.16.0, `--benchmark`, `--sampler_backend=cpu`). The GPU row is the gate run of 2026-08-24 (one run); the CPU rows were taken 2026-09-05, two cold runs (compile and weight caches deleted between runs) with the phone cooled below 42 °C before each run.
78
+
79
+ | Backend | Prefill | Decode | TTFT | Engine init | Peak (`VmHWM`) |
80
+ |---|---|---|---|---|---:|
81
+ | GPU (`LiteRT GPU`, full delegation) | 234.9 tok/s | 8.6 tok/s | 1.07 s | 8.3 s | 885 MB |
82
+ | CPU (XNNPACK) | 57–106 tok/s | **27.0–29.7 tok/s** | 2.14–3.94 s | 2.7 s | 1566 MB |
83
+
84
+ The CPU decodes 3.3× faster; the GPU wins prefill and peaks lower. A code answer is mostly decode (the bench replies ran 360–409 tokens), so this repo's `litertlm_manifest.json` recommends `cpu` for flagship-class Android on that pair. Pick `gpu` for long prompts with short answers.
85
 
86
  GPU wiring, including the Gallery import toggle: [GPU guide](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/android-gpu.md).
87