mlboydaisuke commited on
Commit
6d0c68a
·
verified ·
1 Parent(s): 9b6a09f

Card update: rank-4 re-export GPU results (full mobile delegation; measured on-device)

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -174,12 +174,12 @@ On the iPhone 17 Pro the int8 file reproduces the desktop outputs **bit-exactly*
174
 
175
  Android figures use the standard TFLite [`benchmark_model`](https://ai.google.dev/edge/litert/models/measurement) on a **Pixel 8a** (Tensor G3, Android 16) — 5 warm-up runs then 20 timed runs, the signature selected explicitly with `--signature_to_run_for`, CPU at 4 threads.
176
 
177
- | Signature | GPU (OpenCL) | CPU (XNNPACK, 4 threads) |
178
  |---|---|---|
179
  | `route_128` | 348 ms | **133 ms** |
180
  | `route_512` | 1558 ms | **611 ms** |
181
 
182
- **Use the CPU on Android.** The OpenCL delegate claims almost none of the graph — 3 of the 24 nodes in the selected signature's subgraph — and every signature lands 2.6–2.8× slower on the GPU than on the CPU. This is the opposite of the vision models in this collection, which are GPU-resident by design.
183
 
184
  ## License
185
 
 
174
 
175
  Android figures use the standard TFLite [`benchmark_model`](https://ai.google.dev/edge/litert/models/measurement) on a **Pixel 8a** (Tensor G3, Android 16) — 5 warm-up runs then 20 timed runs, the signature selected explicitly with `--signature_to_run_for`, CPU at 4 threads.
176
 
177
+ | Signature | GPU (OpenCL, previous export) | CPU (XNNPACK, 4 threads) |
178
  |---|---|---|
179
  | `route_128` | 348 ms | **133 ms** |
180
  | `route_512` | 1558 ms | **611 ms** |
181
 
182
+ **GPU works as of the 2026-08-13 re-export.** The re-export respells the one idiom mobile GPU delegates refuse — transformers' rank-5 `repeat_kv` expand — into an equivalent rank-4 matmul (outputs bitwise-identical on CPU); the OpenCL delegate now takes the whole graph. Measured with the LiteRT CompiledModel API (fp32 GPU precision, real inputs incl. pooling matrices, best of 3 warm runs): `route_512` **18.7 ms** on the Pixel 8a, cosine **0.9949** vs the fp32 desktop reference; iPhone 17 Pro Metal `route_512` 172 ms, cosine **1.000000**. Set the GPU precision to **fp32** — at fp16 GPU precision this family's norm reductions overflow and every output is NaN. These CompiledModel timings are not comparable to the classic-delegate `benchmark_model` timings above (different GPU runtime).
183
 
184
  ## License
185