mlboydaisuke commited on
Commit
5f60d83
Β·
verified Β·
1 Parent(s): 2574906

Add measured Galaxy S26 NPU/GPU section

Browse files

Numbers from the S26 classic sweep. Primary logs:
litertlm-convert/community_accel_work/s2_npu_sweep/results.jsonl (on-device
JIT) and portal_work/bench_all.txt + bench_redo.txt (the 2026-08-23 AOT
sweep); every row states which of the two produced it. Rows carry device,
SoC, runtime, N and thermal state, and join to this repo by exact .tflite
file name. Rows taken outside the sweep's thermal window are not reported.

Files changed (1) hide show
  1. README.md +16 -1
README.md CHANGED
@@ -178,7 +178,22 @@ Android figures use the standard TFLite [`benchmark_model`](https://ai.google.de
178
  | `encode_512` | 1287 ms | **581 ms** |
179
  | `mlm_128` | 431 ms | **135 ms** |
180
 
181
- **GPU works as of the 2026-08-13 re-export β€” via the single-signature file.** This re-export respells the one idiom mobile GPU delegates refuse β€” transformers' rank-5 `repeat_kv` expand β€” into an equivalent rank-4 matmul (outputs bitwise-identical on CPU), which makes the graph fully GPU-delegable. The 5-signature file still cannot compile on phone GPUs: each signature subgraph materializes its own fp32 weight copy (~7 GB total for this model), which exceeds phone memory on both iOS and Android. The new **`LFM2.5-Encoder-350M_wi8fc_single-sig.tflite`** (`encode_512` only, 359 MB) is the phone-GPU artifact: Pixel 8a OpenCL full delegation, `encode_512` **12.2 ms**, cosine **0.9994** vs the fp32 desktop reference (LiteRT CompiledModel API, fp32 GPU precision, best of 3 warm runs); iPhone 17 Pro Metal 171 ms, cosine 0.9994. The 5-signature file remains the CPU artifact (bit-exact on device; iPhone `encode_512` 144–158 ms). Set the GPU precision to **fp32** β€” at fp16 GPU precision this family's norm reductions overflow and every output is NaN. These CompiledModel timings are not comparable to the classic-delegate `benchmark_model` timings above (different GPU runtime).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
182
 
183
  ## License
184
 
 
178
  | `encode_512` | 1287 ms | **581 ms** |
179
  | `mlm_128` | 431 ms | **135 ms** |
180
 
181
+ **GPU works as of the 2026-08-13 re-export β€” via the single-signature file.** This re-export respells the one idiom mobile GPU delegates refuse β€” transformers' rank-5 `repeat_kv` expand β€” into an equivalent rank-4 matmul (outputs bitwise-identical on CPU), which makes the graph fully GPU-delegable. The 5-signature file does not compile on the Pixel 8a OpenCL or iPhone 17 Pro Metal delegates: each signature subgraph materializes its own fp32 weight copy (~7 GB total for this model), which exceeds those phones' memory. It does compile and run on the Adreno GPU of a Galaxy S26 under LiteRT `CompiledModel` 2.2.0 β€” 82.04 ms β€” but it pays a 17.3 s load there against 4.07 s for the single-signature file (see Snapdragon NPU (Hexagon) below). The new **`LFM2.5-Encoder-350M_wi8fc_single-sig.tflite`** (`encode_512` only, 359 MB) is the phone-GPU artifact: Pixel 8a OpenCL full delegation, `encode_512` **12.2 ms**, cosine **0.9994** vs the fp32 desktop reference (LiteRT CompiledModel API, fp32 GPU precision, best of 3 warm runs); iPhone 17 Pro Metal 171 ms, cosine 0.9994. The 5-signature file remains the CPU artifact (bit-exact on device; iPhone `encode_512` 144–158 ms). Set the GPU precision to **fp32** β€” at fp16 GPU precision this family's norm reductions overflow and every output is NaN. These CompiledModel timings are not comparable to the classic-delegate `benchmark_model` timings above (different GPU runtime).
182
+
183
+ ## Snapdragon NPU (Hexagon)
184
+
185
+ - `LFM2.5-Encoder-350M_fp16.tflite` β€” neither accelerator produced a usable row on the S26. Both ended the same way: `LiteRtException: Failed to compile model`.
186
+ - `LFM2.5-Encoder-350M_wi8fc.tflite` β€” the GPU runs it at 82.04 ms. The NPU does not β€” `LiteRtException: Failed to compile model`.
187
+ - `LFM2.5-Encoder-350M_wi8fc_single-sig.tflite` β€” the GPU runs it at 82.24 ms. The NPU does not β€” the graph compiles and then fails to run (`LiteRtException: Failed to invoke the compiled model`).
188
+
189
+ | file | backend | compiled | inference (median / min) | load |
190
+ |---|---|---|---:|---:|
191
+ | `LFM2.5-Encoder-350M_wi8fc.tflite` | GPU (Adreno) | β€” | 82.04 ms / 81.14 ms | 17349 ms |
192
+ | `LFM2.5-Encoder-350M_wi8fc_single-sig.tflite` | GPU (Adreno) | β€” | 82.24 ms / 80.41 ms | 4074 ms |
193
+
194
+ Measured on a **Samsung Galaxy S26** (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16) with LiteRT `CompiledModel` 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status `NONE` throughout. Headroom 0.72–0.78, where 1.0 is the throttling threshold.
195
+
196
+ GPU wiring: [GPU guide](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/android-gpu.md). NPU recipe: [NPU guide](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/android-npu.md).
197
 
198
  ## License
199