Galaxy Z Fold 8 results beside the S23's; GPU priority depends on the GPU

fc56546 verified 6 days ago

11.5 kB

---
license: apache-2.0
language:
- en
base_model: fastino/GLiNER2.5-Decide
library_name: litert
tags:
- gliner2
- litert
- tflite
- android
- on-device
---
# GLiNER2.5-Decide — LiteRT (Android)

A LiteRT conversion of [fastino/GLiNER2.5-Decide](https://huggingface.co/fastino/GLiNER2.5-Decide) (revision
`bbe10ff77ebb238777c17d3a8ac9260e30929057`, Apache-2.0, by Fastino), converted by spybyscript. It carries every trained head:
classification, plus the span heads that extract entities, structures and relations. gliner2's own code decodes the
outputs. For the model itself (training, labels, benchmarks, intended use), see the
[original model card](https://huggingface.co/fastino/GLiNER2.5-Decide), copied here as `model-card.md`. Parts of the
graph code are ported from [litert-community/GLiNER2.5-Decide-LiteRT](https://huggingface.co/litert-community/GLiNER2.5-Decide-LiteRT).

## Files

| File | Size | Contents |
|---|---|---|
| `gliner25_decide_wfp16.tflite` | 1028 MB | encoder for the CPU and the GPU at FP32: `s128` / `s160` / `s256` / `s512` (classifier logits at every position) and `e128` … `e512` (hidden states), one set of float16-stored weights |
| `gliner25_decide_fp16hw.tflite` | 1028 MB | the same encoder rewritten for FP16 hardware (GPU at FP16, NPU): LayerNorm power-of-two pre-scale, first feed-forward layer split in two |
| `gliner25_decide_span_wfp16.tflite` | 76 MB | span heads in a form LiteRT's GPU compiles: `w{W}` span representations, `f{W}_p{P}` fused span scores, `count`, classifier `k32` / `k128` |
| `gliner25_decide_count_wfp16.tflite` | 38 MB | count embedding `c{C}_m{M}` (CPU) |
| `gliner25_decide_heads_wfp16.tflite` | 105 MB | all span heads in the library's layout (CPU or NPU); the span file and the count file replace it |
| `gliner25_decide_embeddings_f16.bin` | 262 MB | token table, float16 [128011, 1024], little-endian, row-major |
| `runtime/` | | `litert_span_runtime.py` (reference runtime), `verify_parity.py`, `parity_corpus.json` |
| `npu/` | | Qualcomm NPU files (below) |

`config.json` lists every graph's signatures, shapes and host steps.

## Use

```python
# pip install gliner2==2.0.0 torch numpy ai-edge-litert
import sys; sys.path.insert(0, "runtime")
from litert_span_runtime import LiteRTSpanExtractor
model = LiteRTSpanExtractor(".", span="gliner25_decide_span_wfp16.tflite", count="gliner25_decide_count_wfp16.tflite")
model.classify_text("Please cancel my order.", {"intent": ["cancel", "refund", "track"]})
model.extract_entities("Alice founded Acme in Toronto in 2020.", ["person", "organization", "city"])
model.extract_relations("Alice founded Acme in Toronto.", ["founded", "located_in"])
```

Every public gliner2 call works: `extract`, `classify_text`, `extract_entities`, `extract_json` and `extract_relations`,
plus their `batch_*` and `*_long` forms. The runtime runs gliner2's span extractor with the five trained modules on
LiteRT. Schema handling, scoring and decoding are the library's own code.

**On a phone,** run the same graphs with LiteRT's `CompiledModel`:
1. Tokenize with `tokenizer.json` and pack the prompt as gliner2 2.0.0 does.
2. Look the ids up in the table (row `id` at byte `id * 2048`) into `inputs_embeds`.
3. Call the smallest signature that holds the request.
4. For classification, read each task's `[L]` marker positions from `logits`.

GPU settings: set the precision explicitly (FP32 for `wfp16`, FP16 for `fp16hw`) and turn on constant tensor sharing.
The right priority depends on the GPU; see "GPU priority" below. On the CPU, set XNNPACK's weight cache path. On a
Snapdragon NPU, load the chip's precompiled file.

**Fixed sizes.** The original runs with dynamic shapes; this conversion has fixed limits:
- a request holds at most 512 tokens (schema and text);
- spans up to 8 words, and up to 20 instances per structure (the checkpoint's own limits);
- up to 32 fields or entity types per task.

A larger request is refused with a ValueError, never truncated. For long text, use the `*_long` calls with a chunk that
fits. The weights change only in storage and compute precision.

**The span file** computes the heads file's span representations with two exact rearrangements (only the order of float
sums differs):
- the first layer of the span head is computed once per word instead of once per span;
- `f{W}_p{P}` folds the last layer into the queries, so a host that needs only the scores skips the [W, 8, 1024] span tensor.

Every tensor is [1, rows, features], the shapes LiteRT's GPU delegate accepts.

## Validation

`runtime/verify_parity.py` runs each request through gliner2 with the original weights and through the reference runtime
(LiteRT CPU), then compares the structured results:
- **The corpus** (`runtime/parity_corpus.json`): 296 requests over 16 call types (entities, relations, structures, the
  three record modes, attributes, single- and multi-label classification, mixed schemas). All 296 are identical, with
  span F1 1.0 and confidences within 0.0013.
- **The same check with the span file and the count file:** also 296 of 296.
- **FluidInference's eleven schema-path fixtures:** 11 of 11.
- **2,338 recorded classification decisions:** `wfp16` gives the library's label on all of them, and `fp16hw` at FP16
  precision on 2,332.

`validation-full.json` and `validation.json` hold the numbers.

## Phones

LiteRT 2.3.2 with `qnn-runtime` 2.50.0. Each graph ran on a request recorded from the corpus: 10 calls after a rest, at
thermal status 0. The CPU ran 4 threads; the GPU ran at NORMAL priority, timed as whole calls. CPU and NPU times are the
model alone. The NPU rows use the chip's precompiled files; at 512 tokens they use the classification file, because the
hidden-state file stops at 256.

### Galaxy Z Fold 8 (SM8850, Snapdragon 8 Elite Gen 5, Android 17)

| Graph | CPU | GPU FP16 (`fp16hw`) | GPU FP32 (`wfp16`) | NPU |
|---|---|---|---|---|
| encoder, 128 / 160 / 256 / 512 tokens | 115 / 145 / 262 / 722 ms | 60 / 72 / 121 / 404 ms | 105 / 137 / 214 / 610 ms | 23 / 30 / 95 / 644 ms |
| span file `w64` / `w128` / `w256` / `w512` | 10 / 20 / 38 / 77 ms | 5.3 / 7.1 / 12 / 24 ms | 9.5 / 14 / 35 / 76 ms | 7.0 / 4.4 / 8.7 / 31 ms |
| fused scores `f64` … `f512`, 16 queries | 6.4 / 14 / 30 / 55 ms | 4.4 / 5.4 / 7.9 / 15 ms | 7.6 / 10 / 16 / 27 ms | 3.1 / 3.9 / 8.2 / 33 ms |
| count embedding, 1 step × 8 fields / 20 × 32 | 1.5 / 37 ms | not supported by LiteRT's GPU | not supported | 0.9 / 2.6 ms |
| classifier, 32 / 128 rows | 0.2 / 0.8 ms | 0.5 / 0.6 ms | 1.0 / 1.0 ms | 0.5 / 0.8 ms |
| dirty memory, encoder / span file | 2.3 / 1.4 GB | 2.8 / 0.8 GB | 4.8 / 1.4 GB | 0.14 / 0.19 GB |

- **An extraction request** (up to 128 tokens and 64 words) takes about 35 ms on the NPU. On the GPU it takes about
  70 ms at FP16 and 125 ms at FP32; on the CPU, about 135 ms.
- **Loading:** the precompiled NPU files load in 0.3–1.0 s; the encoder's first GPU load takes 11–13 s.
- **Answers with the phone's outputs:**
  - the encoder: unchanged on the CPU and at GPU FP32 and FP16; on the NPU, 31 of 32 (one family-relation request changed);
  - the span file and the heads file: unchanged on the GPU and the NPU.

### Galaxy S23 Ultra (SM8550, Snapdragon 8 Gen 2, Android 16)

| Graph | CPU | GPU FP16 (`fp16hw`) | GPU FP32 (`wfp16`) | NPU |
|---|---|---|---|---|
| encoder, 128 / 160 / 256 / 512 tokens | 628 / 866 / 1,467 / 3,486 ms | 103 / 124 / 227 / 773 ms | 202 / 268 / 471 / 1,549 ms | 34 / 55 / 168 / 1,149 ms |
| span file `w64` / `w128` / `w256` / `w512` | 43 / 105 / 221 / 450 ms | 10 / 15 / 25 / 47 ms | 19 / 26 / 42 / 79 ms | 7.3 / 6.0 / 14 / 42 ms |
| fused scores `f64` … `f512`, 16 queries | 26 / 55 / 111 / 220 ms | 8 / 10 / 16 / 27 ms | 15 / 18 / 28 / 46 ms | 3.8 / 5.6 / 14 / 44 ms |
| count embedding, 1 step × 8 fields / 20 × 32 | 2.1 / 169 ms | not supported by LiteRT's GPU | not supported | 1.4 / 5.4 ms |
| classifier, 32 / 128 rows | 0.9 / 3.8 ms | 1.2 / 1.5 ms | 1.6 / 2.4 ms | 0.8 / 1.3 ms |
| dirty memory, encoder / span file | 1.9 / 1.7 GB | 2.7 / 0.8 GB | 4.6 / 1.3 GB | 0.12 / 0.21 GB |

- **An extraction request** (up to 128 tokens and 64 words) takes about 43 ms on the NPU. On the GPU it takes about
  115 ms at FP16 and 220 ms at FP32; on the CPU, about 660 ms.
- **Loading:** the precompiled NPU files load in 0.5–1.7 s; the encoder's first GPU load takes 6–10 s.
- **Answers with the phone's outputs:**
  - the encoder: unchanged on the NPU, at GPU FP32 and on the CPU; at GPU FP16, 31 of 32 (one relation request changed);
  - the span file and the heads file: unchanged on the GPU and the NPU.

Each answer check ran 32 recorded requests (two per call type) with the phone's outputs through the rest of the pipeline.

### GPU priority beside a rendering app

An app rendered every frame (Godot 4.7, Vulkan) while the model ran three classification calls each round (`s160` and two
`s256`, `fp16hw` at GPU FP16). Each cell gives the app's longest frame per round (median) and the three calls' total time.

| The model on | Galaxy Z Fold 8 (Adreno 840, 60 Hz) | Galaxy S23 Ultra (Adreno 740, 120 Hz) |
|---|---|---|
| GPU, LOW priority | **37 ms**, 0.66 s | 600 ms, 1.6 s |
| GPU, NORMAL priority | 110 ms, 0.37 s | **228 ms**, 0.65 s |
| NPU | 39 ms, 0.24 s | 81 ms, 0.5 s |
| CPU | 41 ms, 0.81 s | 48 ms, 4–5 s |

- **LOW priority depends on the GPU.** On the Fold's Adreno 840, LOW keeps the frames and costs about 0.3 s per round. On
  the S23's Adreno 740, LOW does not let the app's rendering in, and only makes each call longer.
- **Measure on the target GPU before choosing.** Use the smallest signature that fits.
- **The NPU keeps the frames on both phones** and is the fastest option.

## Qualcomm NPU files (`npu/`)

The NPU path runs through LiteRT's Qualcomm accelerator:
- LiteRT 2.3.2 with `litert-npu-runtime-qualcomm` 2.3.2 and Qualcomm's `qnn-runtime` 2.50.0 (Qualcomm's AI Stack License);
- files precompiled on x86-64 Linux with QAIRT 2.50.0;
- each signature in one NPU partition;
- each file runs only on its chip (`Build.SOC_MODEL`) and with those runtime versions.

`npu/manifest.json` lists every file with its chip, signatures, size and SHA-256.

| File | Contents | Size |
|---|---|---|
| `npu/qualcomm/gliner25_decide_fp16hw_<chip>.tflite` | classification, `s128` … `s512` | 856–877 MB |
| `npu/qualcomm/gliner25_decide_fp16hw_se128-256_<chip>.tflite` | classification and hidden states, 128–256 tokens | 777–792 MB |
| `npu/qualcomm/gliner25_decide_heads_<chip>.tflite` | the heads file | 108–115 MB |
| `npu/qualcomm/gliner25_decide_span_<chip>.tflite` | the span file | 80–81 MB |
| `npu/gliner25_decide_fp16hw_s128.tflite`, `_s160`, `_s256` | single-signature cuts that the phone compiles itself, once (on the S23: 21 / 41 / 117 s) | 660–710 MB |

| Chip | Snapdragon | Hexagon | Run on a phone |
|---|---|---|---|
| SM8850 | 8 Elite Gen 5 | V81 | all four files (Galaxy Z Fold 8, the results above) |
| SM8750 | 8 Elite | V79 | not yet |
| SM8650 | 8 Gen 3 | V75 | not yet |
| SM8550 | 8 Gen 2 | V73 | all four files (Galaxy S23 Ultra, the results above) |
| SM8450 | 8 Gen 1 | V69 | not yet |

## Files

- `LICENSE` (Apache-2.0) and `NOTICE.txt` (attribution and changes)
- `model-card.md`, the original card
- `config.json`, the contract
- `validation.json` and `validation-full.json`
- `SHA256SUMS`
- the tokenizer files
- `source/` (the checkpoint's configs)
- `runtime/` and `npu/`

	---
	license: apache-2.0
	language:
	- en
	base_model: fastino/GLiNER2.5-Decide
	library_name: litert
	tags:
	- gliner2
	- litert
	- tflite
	- android
	- on-device
	---

	# GLiNER2.5-Decide — LiteRT (Android)

	A LiteRT conversion of [fastino/GLiNER2.5-Decide](https://huggingface.co/fastino/GLiNER2.5-Decide) (revision
	`bbe10ff77ebb238777c17d3a8ac9260e30929057`, Apache-2.0, by Fastino), converted by spybyscript. It carries every trained head:
	classification, plus the span heads that extract entities, structures and relations. gliner2's own code decodes the
	outputs. For the model itself (training, labels, benchmarks, intended use), see the
	[original model card](https://huggingface.co/fastino/GLiNER2.5-Decide), copied here as `model-card.md`. Parts of the
	graph code are ported from [litert-community/GLiNER2.5-Decide-LiteRT](https://huggingface.co/litert-community/GLiNER2.5-Decide-LiteRT).

	## Files

	\| File \| Size \| Contents \|
	\|---\|---\|---\|
	\| `gliner25_decide_wfp16.tflite` \| 1028 MB \| encoder for the CPU and the GPU at FP32: `s128` / `s160` / `s256` / `s512` (classifier logits at every position) and `e128` … `e512` (hidden states), one set of float16-stored weights \|
	\| `gliner25_decide_fp16hw.tflite` \| 1028 MB \| the same encoder rewritten for FP16 hardware (GPU at FP16, NPU): LayerNorm power-of-two pre-scale, first feed-forward layer split in two \|
	\| `gliner25_decide_span_wfp16.tflite` \| 76 MB \| span heads in a form LiteRT's GPU compiles: `w{W}` span representations, `f{W}_p{P}` fused span scores, `count`, classifier `k32` / `k128` \|
	\| `gliner25_decide_count_wfp16.tflite` \| 38 MB \| count embedding `c{C}_m{M}` (CPU) \|
	\| `gliner25_decide_heads_wfp16.tflite` \| 105 MB \| all span heads in the library's layout (CPU or NPU); the span file and the count file replace it \|
	\| `gliner25_decide_embeddings_f16.bin` \| 262 MB \| token table, float16 [128011, 1024], little-endian, row-major \|
	\| `runtime/` \| \| `litert_span_runtime.py` (reference runtime), `verify_parity.py`, `parity_corpus.json` \|
	\| `npu/` \| \| Qualcomm NPU files (below) \|

	`config.json` lists every graph's signatures, shapes and host steps.

	## Use

	```python
	# pip install gliner2==2.0.0 torch numpy ai-edge-litert
	import sys; sys.path.insert(0, "runtime")
	from litert_span_runtime import LiteRTSpanExtractor

	model = LiteRTSpanExtractor(".", span="gliner25_decide_span_wfp16.tflite", count="gliner25_decide_count_wfp16.tflite")
	model.classify_text("Please cancel my order.", {"intent": ["cancel", "refund", "track"]})
	model.extract_entities("Alice founded Acme in Toronto in 2020.", ["person", "organization", "city"])
	model.extract_relations("Alice founded Acme in Toronto.", ["founded", "located_in"])
	```

	Every public gliner2 call works: `extract`, `classify_text`, `extract_entities`, `extract_json` and `extract_relations`,
	plus their `batch_` and `_long` forms. The runtime runs gliner2's span extractor with the five trained modules on
	LiteRT. Schema handling, scoring and decoding are the library's own code.

	On a phone, run the same graphs with LiteRT's `CompiledModel`:
	1. Tokenize with `tokenizer.json` and pack the prompt as gliner2 2.0.0 does.
	2. Look the ids up in the table (row `id` at byte `id * 2048`) into `inputs_embeds`.
	3. Call the smallest signature that holds the request.
	4. For classification, read each task's `[L]` marker positions from `logits`.

	GPU settings: set the precision explicitly (FP32 for `wfp16`, FP16 for `fp16hw`) and turn on constant tensor sharing.
	The right priority depends on the GPU; see "GPU priority" below. On the CPU, set XNNPACK's weight cache path. On a
	Snapdragon NPU, load the chip's precompiled file.

	Fixed sizes. The original runs with dynamic shapes; this conversion has fixed limits:
	- a request holds at most 512 tokens (schema and text);
	- spans up to 8 words, and up to 20 instances per structure (the checkpoint's own limits);
	- up to 32 fields or entity types per task.

	A larger request is refused with a ValueError, never truncated. For long text, use the `*_long` calls with a chunk that
	fits. The weights change only in storage and compute precision.

	The span file computes the heads file's span representations with two exact rearrangements (only the order of float
	sums differs):
	- the first layer of the span head is computed once per word instead of once per span;
	- `f{W}_p{P}` folds the last layer into the queries, so a host that needs only the scores skips the [W, 8, 1024] span tensor.

	Every tensor is [1, rows, features], the shapes LiteRT's GPU delegate accepts.

	## Validation

	`runtime/verify_parity.py` runs each request through gliner2 with the original weights and through the reference runtime
	(LiteRT CPU), then compares the structured results:
	- The corpus (`runtime/parity_corpus.json`): 296 requests over 16 call types (entities, relations, structures, the
	three record modes, attributes, single- and multi-label classification, mixed schemas). All 296 are identical, with
	span F1 1.0 and confidences within 0.0013.
	- The same check with the span file and the count file: also 296 of 296.
	- FluidInference's eleven schema-path fixtures: 11 of 11.
	- 2,338 recorded classification decisions: `wfp16` gives the library's label on all of them, and `fp16hw` at FP16
	precision on 2,332.

	`validation-full.json` and `validation.json` hold the numbers.

	## Phones

	LiteRT 2.3.2 with `qnn-runtime` 2.50.0. Each graph ran on a request recorded from the corpus: 10 calls after a rest, at
	thermal status 0. The CPU ran 4 threads; the GPU ran at NORMAL priority, timed as whole calls. CPU and NPU times are the
	model alone. The NPU rows use the chip's precompiled files; at 512 tokens they use the classification file, because the
	hidden-state file stops at 256.

	### Galaxy Z Fold 8 (SM8850, Snapdragon 8 Elite Gen 5, Android 17)

	\| Graph \| CPU \| GPU FP16 (`fp16hw`) \| GPU FP32 (`wfp16`) \| NPU \|
	\|---\|---\|---\|---\|---\|
	\| encoder, 128 / 160 / 256 / 512 tokens \| 115 / 145 / 262 / 722 ms \| 60 / 72 / 121 / 404 ms \| 105 / 137 / 214 / 610 ms \| 23 / 30 / 95 / 644 ms \|
	\| span file `w64` / `w128` / `w256` / `w512` \| 10 / 20 / 38 / 77 ms \| 5.3 / 7.1 / 12 / 24 ms \| 9.5 / 14 / 35 / 76 ms \| 7.0 / 4.4 / 8.7 / 31 ms \|
	\| fused scores `f64` … `f512`, 16 queries \| 6.4 / 14 / 30 / 55 ms \| 4.4 / 5.4 / 7.9 / 15 ms \| 7.6 / 10 / 16 / 27 ms \| 3.1 / 3.9 / 8.2 / 33 ms \|
	\| count embedding, 1 step × 8 fields / 20 × 32 \| 1.5 / 37 ms \| not supported by LiteRT's GPU \| not supported \| 0.9 / 2.6 ms \|
	\| classifier, 32 / 128 rows \| 0.2 / 0.8 ms \| 0.5 / 0.6 ms \| 1.0 / 1.0 ms \| 0.5 / 0.8 ms \|
	\| dirty memory, encoder / span file \| 2.3 / 1.4 GB \| 2.8 / 0.8 GB \| 4.8 / 1.4 GB \| 0.14 / 0.19 GB \|

	- An extraction request (up to 128 tokens and 64 words) takes about 35 ms on the NPU. On the GPU it takes about
	70 ms at FP16 and 125 ms at FP32; on the CPU, about 135 ms.
	- Loading: the precompiled NPU files load in 0.3–1.0 s; the encoder's first GPU load takes 11–13 s.
	- Answers with the phone's outputs:
	- the encoder: unchanged on the CPU and at GPU FP32 and FP16; on the NPU, 31 of 32 (one family-relation request changed);
	- the span file and the heads file: unchanged on the GPU and the NPU.

	### Galaxy S23 Ultra (SM8550, Snapdragon 8 Gen 2, Android 16)

	\| Graph \| CPU \| GPU FP16 (`fp16hw`) \| GPU FP32 (`wfp16`) \| NPU \|
	\|---\|---\|---\|---\|---\|
	\| encoder, 128 / 160 / 256 / 512 tokens \| 628 / 866 / 1,467 / 3,486 ms \| 103 / 124 / 227 / 773 ms \| 202 / 268 / 471 / 1,549 ms \| 34 / 55 / 168 / 1,149 ms \|
	\| span file `w64` / `w128` / `w256` / `w512` \| 43 / 105 / 221 / 450 ms \| 10 / 15 / 25 / 47 ms \| 19 / 26 / 42 / 79 ms \| 7.3 / 6.0 / 14 / 42 ms \|
	\| fused scores `f64` … `f512`, 16 queries \| 26 / 55 / 111 / 220 ms \| 8 / 10 / 16 / 27 ms \| 15 / 18 / 28 / 46 ms \| 3.8 / 5.6 / 14 / 44 ms \|
	\| count embedding, 1 step × 8 fields / 20 × 32 \| 2.1 / 169 ms \| not supported by LiteRT's GPU \| not supported \| 1.4 / 5.4 ms \|
	\| classifier, 32 / 128 rows \| 0.9 / 3.8 ms \| 1.2 / 1.5 ms \| 1.6 / 2.4 ms \| 0.8 / 1.3 ms \|
	\| dirty memory, encoder / span file \| 1.9 / 1.7 GB \| 2.7 / 0.8 GB \| 4.6 / 1.3 GB \| 0.12 / 0.21 GB \|

	- An extraction request (up to 128 tokens and 64 words) takes about 43 ms on the NPU. On the GPU it takes about
	115 ms at FP16 and 220 ms at FP32; on the CPU, about 660 ms.
	- Loading: the precompiled NPU files load in 0.5–1.7 s; the encoder's first GPU load takes 6–10 s.
	- Answers with the phone's outputs:
	- the encoder: unchanged on the NPU, at GPU FP32 and on the CPU; at GPU FP16, 31 of 32 (one relation request changed);
	- the span file and the heads file: unchanged on the GPU and the NPU.

	Each answer check ran 32 recorded requests (two per call type) with the phone's outputs through the rest of the pipeline.

	### GPU priority beside a rendering app

	An app rendered every frame (Godot 4.7, Vulkan) while the model ran three classification calls each round (`s160` and two
	`s256`, `fp16hw` at GPU FP16). Each cell gives the app's longest frame per round (median) and the three calls' total time.

	\| The model on \| Galaxy Z Fold 8 (Adreno 840, 60 Hz) \| Galaxy S23 Ultra (Adreno 740, 120 Hz) \|
	\|---\|---\|---\|
	\| GPU, LOW priority \| 37 ms, 0.66 s \| 600 ms, 1.6 s \|
	\| GPU, NORMAL priority \| 110 ms, 0.37 s \| 228 ms, 0.65 s \|
	\| NPU \| 39 ms, 0.24 s \| 81 ms, 0.5 s \|
	\| CPU \| 41 ms, 0.81 s \| 48 ms, 4–5 s \|

	- LOW priority depends on the GPU. On the Fold's Adreno 840, LOW keeps the frames and costs about 0.3 s per round. On
	the S23's Adreno 740, LOW does not let the app's rendering in, and only makes each call longer.
	- Measure on the target GPU before choosing. Use the smallest signature that fits.
	- The NPU keeps the frames on both phones and is the fastest option.

	## Qualcomm NPU files (`npu/`)

	The NPU path runs through LiteRT's Qualcomm accelerator:
	- LiteRT 2.3.2 with `litert-npu-runtime-qualcomm` 2.3.2 and Qualcomm's `qnn-runtime` 2.50.0 (Qualcomm's AI Stack License);
	- files precompiled on x86-64 Linux with QAIRT 2.50.0;
	- each signature in one NPU partition;
	- each file runs only on its chip (`Build.SOC_MODEL`) and with those runtime versions.

	`npu/manifest.json` lists every file with its chip, signatures, size and SHA-256.

	\| File \| Contents \| Size \|
	\|---\|---\|---\|
	\| `npu/qualcomm/gliner25_decide_fp16hw_<chip>.tflite` \| classification, `s128` … `s512` \| 856–877 MB \|
	\| `npu/qualcomm/gliner25_decide_fp16hw_se128-256_<chip>.tflite` \| classification and hidden states, 128–256 tokens \| 777–792 MB \|
	\| `npu/qualcomm/gliner25_decide_heads_<chip>.tflite` \| the heads file \| 108–115 MB \|
	\| `npu/qualcomm/gliner25_decide_span_<chip>.tflite` \| the span file \| 80–81 MB \|
	\| `npu/gliner25_decide_fp16hw_s128.tflite`, `_s160`, `_s256` \| single-signature cuts that the phone compiles itself, once (on the S23: 21 / 41 / 117 s) \| 660–710 MB \|

	\| Chip \| Snapdragon \| Hexagon \| Run on a phone \|
	\|---\|---\|---\|---\|
	\| SM8850 \| 8 Elite Gen 5 \| V81 \| all four files (Galaxy Z Fold 8, the results above) \|
	\| SM8750 \| 8 Elite \| V79 \| not yet \|
	\| SM8650 \| 8 Gen 3 \| V75 \| not yet \|
	\| SM8550 \| 8 Gen 2 \| V73 \| all four files (Galaxy S23 Ultra, the results above) \|
	\| SM8450 \| 8 Gen 1 \| V69 \| not yet \|

	## Files

	- `LICENSE` (Apache-2.0) and `NOTICE.txt` (attribution and changes)
	- `model-card.md`, the original card
	- `config.json`, the contract
	- `validation.json` and `validation-full.json`
	- `SHA256SUMS`
	- the tokenizer files
	- `source/` (the checkpoint's configs)
	- `runtime/` and `npu/`