mlboydaisuke commited on
Commit
3183689
·
verified ·
1 Parent(s): e5bd551

Add measured Galaxy S26 NPU/GPU section

Browse files

Numbers from the S26 classic sweep. Primary logs:
litertlm-convert/community_accel_work/s2_npu_sweep/results.jsonl (on-device
JIT) and portal_work/bench_all.txt + bench_redo.txt (the 2026-08-23 AOT
sweep); every row states which of the two produced it. Rows carry device,
SoC, runtime, N and thermal state, and join to this repo by exact .tflite
file name. Rows taken outside the sweep's thermal window are not reported.

Files changed (1) hide show
  1. README.md +22 -0
README.md CHANGED
@@ -131,3 +131,25 @@ On this delegate the CPU is the faster choice for `ram_taghead_fp16.tflite` (155
131
 
132
  Note that the GPU does not take the whole graph here (30 / 139 in `ram_taghead_fp16.tflite`, 9 / 18 in `ram_reweight_fp16.tflite`, 105 / 2129 in `ram_swin_s012_fp16.tflite`, 41 / 178 in `ram_stage3_tail_fp16.tflite`); the remainder runs on the CPU and the split costs a per-partition round trip.
133
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
131
 
132
  Note that the GPU does not take the whole graph here (30 / 139 in `ram_taghead_fp16.tflite`, 9 / 18 in `ram_reweight_fp16.tflite`, 105 / 2129 in `ram_swin_s012_fp16.tflite`, 41 / 178 in `ram_stage3_tail_fp16.tflite`); the remainder runs on the CPU and the split costs a per-partition round trip.
133
 
134
+ ## Snapdragon NPU (Hexagon)
135
+
136
+ - `ram_reweight_fp16.tflite` — the NPU runs it at 28.30 ms. The GPU does not — `LiteRtException: Failed to compile model`.
137
+ - `ram_stage3_tail_fp16.tflite` — the **GPU is faster**: 8.01 ms against 14.57 ms on the NPU, a factor of 1.82. The NPU still loads 23x faster (148 ms against 3335 ms).
138
+ - `ram_swin_s012_fp16.tflite` — the **GPU is faster**: 103.0 ms against 396.9 ms on the NPU, a factor of 3.85. The NPU still loads 6.51x faster (920 ms against 5986 ms).
139
+ - `ram_taghead_fp16.tflite` — the **GPU is faster**: 44.03 ms against 170.8 ms on the NPU, a factor of 3.88. The NPU still loads 3.57x faster (346 ms against 1235 ms).
140
+
141
+ | file | backend | compiled | inference (median / min) | load |
142
+ |---|---|---|---:|---:|
143
+ | `ram_reweight_fp16.tflite` | NPU (Hexagon v81) | on-device JIT | 28.30 ms / 22.41 ms | 257 ms |
144
+ | `ram_stage3_tail_fp16.tflite` | NPU (Hexagon v81) | on-device JIT | 14.57 ms / 14.40 ms | 148 ms |
145
+ | `ram_stage3_tail_fp16.tflite` | GPU (Adreno) | — | 8.01 ms / 7.59 ms | 3335 ms |
146
+ | `ram_swin_s012_fp16.tflite` | NPU (Hexagon v81) | on-device JIT | 396.9 ms / 392.9 ms | 920 ms |
147
+ | `ram_swin_s012_fp16.tflite` | GPU (Adreno) | — | 103.0 ms / 102.0 ms | 5986 ms |
148
+ | `ram_taghead_fp16.tflite` | NPU (Hexagon v81) | on-device JIT | 170.8 ms / 161.3 ms | 346 ms |
149
+ | `ram_taghead_fp16.tflite` | GPU (Adreno) | — | 44.03 ms / 42.86 ms | 1235 ms |
150
+
151
+ Measured on a **Samsung Galaxy S26** (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT `CompiledModel` 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status `NONE` throughout. Headroom 0.72–0.82, where 1.0 is the throttling threshold.
152
+
153
+ **The NPU rows ran the published file unchanged.** LiteRT compiled it for the Hexagon on the device at first load. Those first compiles took 7.3 s to 13.7 min here. The `load` column above is the cached load every later run pays. Recipe and the runtime libraries it needs: [NPU guide](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/android-npu.md).
154
+
155
+ GPU wiring: [GPU guide](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/android-gpu.md).