Galaxy Z Fold 8 results beside the S23's; GPU priority depends on the GPU

README.md

@@ -59,9 +59,9 @@ LiteRT. Schema handling, scoring and decoding are the library's own code.
 3. Call the smallest signature that holds the request.
 4. For classification, read each task's `[L]` marker positions from `logits`.
 
-GPU settings: set the precision explicitly (FP32 for `wfp16`, FP16 for `fp16hw`), turn on constant tensor sharing, and use
-NORMAL priority (see the S23 results). On the CPU, set XNNPACK's weight cache path. On a Snapdragon NPU, load the chip's
-precompiled file.
+GPU settings: set the precision explicitly (FP32 for `wfp16`, FP16 for `fp16hw`) and turn on constant tensor sharing.
+The right priority depends on the GPU; see "GPU priority" below. On the CPU, set XNNPACK's weight cache path. On a
+Snapdragon NPU, load the chip's precompiled file.
 
 **Fixed sizes.** The original runs with dynamic shapes; this conversion has fixed limits:
 - a request holds at most 512 tokens (schema and text);
@@ -92,13 +92,34 @@ Every tensor is [1, rows, features], the shapes LiteRT's GPU delegate accepts.
 
 `validation-full.json` and `validation.json` hold the numbers.
 
-## Galaxy S23 Ultra (SM8550)
+## Phones
 
-Snapdragon 8 Gen 2, Android 16, LiteRT 2.3.2 with `qnn-runtime` 2.50.0. Each graph ran on a request recorded from the
-corpus: 10 calls after a 30 s rest, at thermal status 0. The CPU ran 4 threads; the GPU ran at NORMAL priority, timed as
-whole calls. CPU and NPU times are the model alone.
+LiteRT 2.3.2 with `qnn-runtime` 2.50.0. Each graph ran on a request recorded from the corpus: 10 calls after a rest, at
+thermal status 0. The CPU ran 4 threads; the GPU ran at NORMAL priority, timed as whole calls. CPU and NPU times are the
+model alone. The NPU rows use the chip's precompiled files; at 512 tokens they use the classification file, because the
+hidden-state file stops at 256.
 
-| Graph | CPU | GPU FP16 (`fp16hw`) | GPU FP32 (`wfp16`) | NPU (precompiled) |
+### Galaxy Z Fold 8 (SM8850, Snapdragon 8 Elite Gen 5, Android 17)
+
+| Graph | CPU | GPU FP16 (`fp16hw`) | GPU FP32 (`wfp16`) | NPU |
+|---|---|---|---|---|
+| encoder, 128 / 160 / 256 / 512 tokens | 115 / 145 / 262 / 722 ms | 60 / 72 / 121 / 404 ms | 105 / 137 / 214 / 610 ms | 23 / 30 / 95 / 644 ms |
+| span file `w64` / `w128` / `w256` / `w512` | 10 / 20 / 38 / 77 ms | 5.3 / 7.1 / 12 / 24 ms | 9.5 / 14 / 35 / 76 ms | 7.0 / 4.4 / 8.7 / 31 ms |
+| fused scores `f64` … `f512`, 16 queries | 6.4 / 14 / 30 / 55 ms | 4.4 / 5.4 / 7.9 / 15 ms | 7.6 / 10 / 16 / 27 ms | 3.1 / 3.9 / 8.2 / 33 ms |
+| count embedding, 1 step × 8 fields / 20 × 32 | 1.5 / 37 ms | not supported by LiteRT's GPU | not supported | 0.9 / 2.6 ms |
+| classifier, 32 / 128 rows | 0.2 / 0.8 ms | 0.5 / 0.6 ms | 1.0 / 1.0 ms | 0.5 / 0.8 ms |
+| dirty memory, encoder / span file | 2.3 / 1.4 GB | 2.8 / 0.8 GB | 4.8 / 1.4 GB | 0.14 / 0.19 GB |
+
+- **An extraction request** (up to 128 tokens and 64 words) takes about 35 ms on the NPU. On the GPU it takes about
+  70 ms at FP16 and 125 ms at FP32; on the CPU, about 135 ms.
+- **Loading:** the precompiled NPU files load in 0.3–1.0 s; the encoder's first GPU load takes 11–13 s.
+- **Answers with the phone's outputs:**
+  - the encoder: unchanged on the CPU and at GPU FP32 and FP16; on the NPU, 31 of 32 (one family-relation request changed);
+  - the span file and the heads file: unchanged on the GPU and the NPU.
+
+### Galaxy S23 Ultra (SM8550, Snapdragon 8 Gen 2, Android 16)
+
+| Graph | CPU | GPU FP16 (`fp16hw`) | GPU FP32 (`wfp16`) | NPU |
 |---|---|---|---|---|
 | encoder, 128 / 160 / 256 / 512 tokens | 628 / 866 / 1,467 / 3,486 ms | 103 / 124 / 227 / 773 ms | 202 / 268 / 471 / 1,549 ms | 34 / 55 / 168 / 1,149 ms |
 | span file `w64` / `w128` / `w256` / `w512` | 43 / 105 / 221 / 450 ms | 10 / 15 / 25 / 47 ms | 19 / 26 / 42 / 79 ms | 7.3 / 6.0 / 14 / 42 ms |
@@ -107,26 +128,31 @@ whole calls. CPU and NPU times are the model alone.
 | classifier, 32 / 128 rows | 0.9 / 3.8 ms | 1.2 / 1.5 ms | 1.6 / 2.4 ms | 0.8 / 1.3 ms |
 | dirty memory, encoder / span file | 1.9 / 1.7 GB | 2.7 / 0.8 GB | 4.6 / 1.3 GB | 0.12 / 0.21 GB |
 
-- **The NPU's 512-token time** comes from the classification file; the hidden-state file stops at 256 tokens.
 - **An extraction request** (up to 128 tokens and 64 words) takes about 43 ms on the NPU. On the GPU it takes about
   115 ms at FP16 and 220 ms at FP32; on the CPU, about 660 ms.
-- **Loading:** a precompiled NPU file loads in 0.5–1.7 s. The encoder's first GPU load takes 6–10 s.
-- **Answers with the phone's outputs.** 32 recorded requests, two per call type, ran through the rest of the pipeline:
+- **Loading:** the precompiled NPU files load in 0.5–1.7 s; the encoder's first GPU load takes 6–10 s.
+- **Answers with the phone's outputs:**
   - the encoder: unchanged on the NPU, at GPU FP32 and on the CPU; at GPU FP16, 31 of 32 (one relation request changed);
-  - the span file: unchanged at GPU FP16 and on the NPU;
-  - the heads file: unchanged on the NPU.
-- **GPU priority beside a rendering app.** An app rendered every frame (Godot 4.7, Vulkan, 120 Hz) while the model ran
-  three classification calls each round (`s160` and two `s256`, `fp16hw` at FP16):
+  - the span file and the heads file: unchanged on the GPU and the NPU.
+
+Each answer check ran 32 recorded requests (two per call type) with the phone's outputs through the rest of the pipeline.
 
-  | The model on | The app's longest frame per round | The three calls |
-  |---|---|---|
-  | GPU, NORMAL priority | 228 ms | about 0.65 s |
-  | GPU, LOW priority | 600 ms | about 1.6 s |
-  | NPU | 81 ms | about 0.5 s |
-  | CPU | 48 ms | 4–5 s |
+### GPU priority beside a rendering app
+
+An app rendered every frame (Godot 4.7, Vulkan) while the model ran three classification calls each round (`s160` and two
+`s256`, `fp16hw` at GPU FP16). Each cell gives the app's longest frame per round (median) and the three calls' total time.
+
+| The model on | Galaxy Z Fold 8 (Adreno 840, 60 Hz) | Galaxy S23 Ultra (Adreno 740, 120 Hz) |
+|---|---|---|
+| GPU, LOW priority | **37 ms**, 0.66 s | 600 ms, 1.6 s |
+| GPU, NORMAL priority | 110 ms, 0.37 s | **228 ms**, 0.65 s |
+| NPU | 39 ms, 0.24 s | 81 ms, 0.5 s |
+| CPU | 41 ms, 0.81 s | 48 ms, 4–5 s |
 
-  On this phone and LiteRT version, LOW does not let the app's rendering in; it only makes each call longer. On the GPU,
-  use NORMAL and the smallest signature that fits. The NPU keeps the frames.
+- **LOW priority depends on the GPU.** On the Fold's Adreno 840, LOW keeps the frames and costs about 0.3 s per round. On
+  the S23's Adreno 740, LOW does not let the app's rendering in, and only makes each call longer.
+- **Measure on the target GPU before choosing.** Use the smallest signature that fits.
+- **The NPU keeps the frames on both phones** and is the fastest option.
 
 ## Qualcomm NPU files (`npu/`)
 
@@ -148,7 +174,7 @@ The NPU path runs through LiteRT's Qualcomm accelerator:
 
 | Chip | Snapdragon | Hexagon | Run on a phone |
 |---|---|---|---|
-| SM8850 | 8 Elite Gen 5 | V81 | classification file (Galaxy Z Fold 8) |
+| SM8850 | 8 Elite Gen 5 | V81 | all four files (Galaxy Z Fold 8, the results above) |
 | SM8750 | 8 Elite | V79 | not yet |
 | SM8650 | 8 Gen 3 | V75 | not yet |
 | SM8550 | 8 Gen 2 | V73 | all four files (Galaxy S23 Ultra, the results above) |

SHA256SUMS

@@ -1,6 +1,6 @@
 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30  LICENSE
 c810d9c1789eef9890a1844136e77a9fb475c950df02a19ffdf3ecbbc50ec7be  NOTICE.txt
-58a06d79fa17abe067a350bfb6e2b11f198ab5463d019e61570381e22eec0753  README.md
+c0acb7d4fdc909f4f6cb35b9e42c74e0e34d5b8390059d3b8e7eb889689664f5  README.md
 98e61f6cec59c2238b2a80c4d3e829de88ba570ebd7ab22187672caa6737d9e7  config.json
 43a132d3571d4a028013990f4866e266d01e35394819d89cbb1cd24d558608bc  gliner25_decide_count_wfp16.tflite
 f90f5d837721b3d4df7194a89ea18bbea5c449e6944fb6078664f1a8d20484fd  gliner25_decide_embeddings_f16.bin
@@ -12,7 +12,7 @@ ce29834da27c7ddacc74e5ebb560c9104e1b83d04a8ba8dd82059737e9dbd419  model-card.md
 323e3f48ad74ba4110f69c58d40d8e2f55e5f1b9723cfb82c30c2a622be86f7a  npu/gliner25_decide_fp16hw_s128.tflite
 1783d73131c2395c3e39e6364536e2f9643b4c1eba3051db21b4cd91a868e0ed  npu/gliner25_decide_fp16hw_s160.tflite
 08fbd3820bace55861948c392a130ab934895d2d75a54e7e1666aabdf0ba16e1  npu/gliner25_decide_fp16hw_s256.tflite
-e726a28e3acf74b551380551a7948b993adc30f20b5edeade2841fc1ed17a34a  npu/manifest.json
+9dfc7fbb9ac0afea880d294cddb7b909caa87771ed5bd06449228331ebdb3344  npu/manifest.json
 b82f3b26b9cd9ce59ea1340486e81bec51a4aae6fcb7b5ff7769b717326dc32a  npu/qualcomm/gliner25_decide_fp16hw_SM8450.tflite
 86bfd5ca3aea9075dcb0482c04572dcfd247f2f6f3bb040ac0d989d8e60ed6bd  npu/qualcomm/gliner25_decide_fp16hw_SM8550.tflite
 48a382ae593afe724eb40b833991b5c8fb8dbb2bb6b14c74ebf9abb862177e02  npu/qualcomm/gliner25_decide_fp16hw_SM8650.tflite

npu/manifest.json

@@ -138,7 +138,7 @@
       "bytes": 792047904,
       "sha256": "d95903041604a96b4e68ff8afd17a1d71fc97a806e91ac73635dc152338f92c5",
       "compileSeconds": 406.0,
-      "checkedOn": null
+      "checkedOn": "Galaxy Z Fold 8, 2026-11-02"
     },
     {
       "path": "npu/qualcomm/gliner25_decide_heads_SM8850.tflite",
@@ -164,7 +164,7 @@
       "bytes": 115163440,
       "sha256": "f582aeed6c4ed83dd74a96e485815fa819d5cf26b7fb1fdb3de1bba26ca8d483",
       "compileSeconds": 76.3,
-      "checkedOn": null
+      "checkedOn": "Galaxy Z Fold 8, 2026-11-02"
     },
     {
       "path": "npu/qualcomm/gliner25_decide_fp16hw_se128-256_SM8750.tflite",
@@ -372,7 +372,7 @@
       "bytes": 81475504,
       "sha256": "47ff970ddafda7f643788d7bce883858667028e3ca1dc50e366a39c06e972b19",
       "compileSeconds": 16.4,
-      "checkedOn": null
+      "checkedOn": "Galaxy Z Fold 8, 2026-11-02"
     },
     {
       "path": "npu/qualcomm/gliner25_decide_span_SM8750.tflite",