A concise card: the files, use, validation and the Galaxy S23 Ultra results

README.md

@@ -14,47 +14,29 @@ tags:
 
 # GLiNER2.5 base (gliner2.5-base-v1) — LiteRT (Android)
 
-An on-device conversion of gliner2.5-base-v1 for Android's LiteRT with every trained head: entities, classification,
-structures, relations and records, decoded by gliner2's own code, validated against the library and measured on phones.
-We know of no other LiteRT conversion of gliner2.5-base-v1.
+A LiteRT conversion of [fastino/gliner2.5-base-v1](https://huggingface.co/fastino/gliner2.5-base-v1) (revision
+`1a8bc24e00dc7300b9017c81d63e3dcdabb26596`, Apache-2.0, by Fastino), converted by spybyscript. It carries every trained head:
+entities, classification, structures, relations and records. gliner2's own code decodes the outputs. For the model itself
+(training, labels, benchmarks, intended use), see the [original model card](https://huggingface.co/fastino/gliner2.5-base-v1),
+copied here as `model-card.md`. The split into graphs follows FluidInference's Core ML port
+([gliner2-5-base-coreml](https://huggingface.co/FluidInference/gliner2-5-base-coreml), Apache-2.0).
 
-Converted from [fastino/gliner2.5-base-v1](https://huggingface.co/fastino/gliner2.5-base-v1) at revision `1a8bc24e00dc7300b9017c81d63e3dcdabb26596` (Apache-2.0, by Fastino).
-The split into graphs follows FluidInference's Core ML port ([gliner2-5-base-coreml](https://huggingface.co/FluidInference/gliner2-5-base-coreml), Apache-2.0),
-with the token table on the host. `config.json` is the full contract.
+## Files
 
 | File | Size | Contents |
 |---|---|---|
-| `gliner2_base_encoder_wfp16.tflite` | 189 MB | encoder (`inputs_embeds`, `attention_mask` → `hidden_states`): FULLY_CONNECTED weights stored float16 behind DEQUANTIZE; FP32 activations, position tables and norms; finite -1e4 mask |
-| `gliner2_base_encoder_fp16hw.tflite` | 189 MB | the same encoder rewritten for FP16 hardware: LayerNorm power-of-two pre-scale (x*2^-k, eps*4^-k), first feed-forward FULLY_CONNECTED split in two halves |
-| `gliner2_base_head_fp32.tflite` | 8.5 MB | boundary head: routed text and query states, boundary marginals, pool projections, null and count heads, classification logits |
-| `gliner2_base_scorer_fp32.tflite` | 2.1 MB | candidate scorer and the record candidate states |
-| `gliner2_base_explicit_fp32.tflite` | 2.1 MB | explicit-span scorer: the boundary proposer's prior and the pair scorer over forced spans (entity attributes, choice fields) |
+| `gliner2_base_encoder_wfp16.tflite` | 189 MB | encoder for the CPU and the GPU at FP32 (`inputs_embeds`, `attention_mask` → `hidden_states`), float16-stored weights |
+| `gliner2_base_encoder_fp16hw.tflite` | 189 MB | the same encoder rewritten for FP16 hardware (NPU): LayerNorm power-of-two pre-scale, first feed-forward layer split in two |
+| `gliner2_base_head_fp32.tflite` | 8.5 MB | boundary head: routed states, boundary marginals, pool projections, null and count heads, classification logits |
+| `gliner2_base_scorer_fp32.tflite` | 2.1 MB | candidate scorer and record candidate states |
+| `gliner2_base_explicit_fp32.tflite` | 2.1 MB | explicit-span scorer (entity attributes, choice fields) |
 | `gliner2_base_relation_fp32.tflite` | 23.7 MB | relation scorer over head/tail pairs |
 | `gliner2_base_record_assignment_fp32.tflite` | 1.2 MB | record decoder: instance-to-field assignment, object and latent-seed scores |
-| `gliner2_base_record_anchorless_fp32.tflite` | 2.9 MB | record decoder: learned instances attending over the field candidates (anchorless records) |
-| `gliner2_base_embeddings_f16.bin` | 197 MB | float16 [128011, 768], little-endian, row-major |
-| `runtime/` | | `litert_boundary_runtime.py` (the reference runtime), `verify_parity.py`, `parity_corpus.json` |
-
-## What it runs
-
-`runtime/litert_boundary_runtime.py` is gliner2's BoundaryExtractor with every trained module on LiteRT: the schema
-handling, the candidate pool and the decoders are the library's own code, and no PyTorch weights are loaded. Every
-public call works: `extract`, `extract_entities`, `classify_text`, `extract_json`, `extract_relations`, and their
-`batch_*` and `*_long` forms.
-
-- Entities, with label descriptions, thresholds and the null head's abstention; entity attributes.
-- Classification, single- and multi-label.
-- Structures (`extract_json`), with choice fields and the natural, latent and anchorless record modes.
-- Relations.
-
-**Fixed sizes.** One request holds at most 128 subword tokens (schema and text), 64 text words, 8 queries (entity
-labels, structure fields, relation roles) and 8 classification choices, with a candidate pool of 192 (the checkpoint's
-own pool size). Relations take up to 4 types and 256 head/tail pairs per request; explicit spans up to 64 per query;
-records up to 8 fields with 192 candidates each. These are FluidInference's bucket sizes. A larger request is refused
-with a ValueError, never truncated: run long text through the `*_long` calls with a chunk that fits, e.g.
-`extract_entities_long(text, labels, chunk_size=40, chunk_overlap=8)`. The original runs with dynamic shapes.
+| `gliner2_base_record_anchorless_fp32.tflite` | 2.9 MB | record decoder for anchorless records |
+| `gliner2_base_embeddings_f16.bin` | 197 MB | token table, float16 [128011, 768], little-endian, row-major |
+| `runtime/` | | `litert_boundary_runtime.py` (reference runtime), `verify_parity.py`, `parity_corpus.json` |
 
-**Weights:** only their storage and compute precision change; nothing is retrained or pruned.
+`config.json` lists every graph's signatures and shapes.
 
 ## Use
 
@@ -71,101 +53,72 @@ schema = Schema().entities(["person", "organization"]).relations(["works_for"])
 model.extract("Ravi Shah is an engineer at Tidewater Systems.", schema, include_confidence=True)
 ```
 
-On a phone, run the same graphs with LiteRT's `CompiledModel` and port the library's decoding steps the runtime uses
-(`config.json` lists every graph's signature).
+Every public gliner2 call works: `extract`, `extract_entities`, `classify_text`, `extract_json` and `extract_relations`,
+plus their `batch_*` and `*_long` forms. The runtime runs gliner2's BoundaryExtractor with every trained module on
+LiteRT. Schema handling, the candidate pool and the decoders are the library's own code.
 
-## Validation
+**On a phone,** run the same graphs with LiteRT's `CompiledModel`; the runtime shows the host steps.
+- **The encoder** is the only graph worth an accelerator: the NPU with `fp16hw`, the GPU at FP32 with `wfp16` (NORMAL
+  priority), or the CPU.
+- **The six head graphs** run on the CPU in 1–16 ms each.
+
+**Fixed sizes.** The original runs with dynamic shapes; these are FluidInference's bucket sizes. One request holds at most:
+- 128 subword tokens (schema and text) and 64 text words;
+- 8 queries (entity labels, structure fields, relation roles) and 8 classification choices;
+- a candidate pool of 192 (the checkpoint's own);
+- 4 relation types with 256 head/tail pairs;
+- 64 explicit spans per query;
+- records of up to 8 fields with 192 candidates each.
 
-### Every call type against the library
+A larger request is refused with a ValueError, never truncated. Run long text through the `*_long` calls, for example
+`extract_entities_long(text, labels, chunk_size=40, chunk_overlap=8)`. The weights change only in storage and compute
+precision.
+
+## Validation
 
 `runtime/verify_parity.py` runs each request through gliner2 with the original weights and through the reference runtime
-(LiteRT 2.2.0 on the CPU, `wfp16` encoder), and compares the structured results. The corpus
-(`runtime/parity_corpus.json`) is 48 short synthetic texts with schema templates for every call type, 296 requests.
-
-| Call type | Requests | Identical results | Span F1 (lowest) | Largest confidence difference |
-|---|---|---|---|---|
-| `entities` | 40 | 40 | 1.0 | 0.000228 |
-| `entities_single` | 8 | 8 | 1.0 | 5e-06 |
-| `relations_work` | 24 | 24 | 1.0 | 0.000883 |
-| `record_natural` | 16 | 16 | 1.0 | 0.00013 |
-| `record_latent` | 24 | 24 | 1.0 | 0.000641 |
-| `attributes` | 16 | 16 | 1.0 | 0.000298 |
-| `classification_topic` | 40 | 40 | 1.0 | 0.001305 |
-| `mixed` | 16 | 16 | 1.0 | 0.001358 |
-| `relations_family` | 8 | 8 | 1.0 | 0.00033 |
-| `classification_described` | 24 | 24 | 1.0 | 0.001187 |
-| `record_anchorless` | 8 | 8 | 1.0 | 0.000258 |
-| `structure_event` | 24 | 24 | 1.0 | 0.000897 |
-| `classification_multi` | 16 | 16 | 1.0 | 0.000356 |
-| `entities_described` | 16 | 16 | 1.0 | 0.000411 |
-| `structure_order` | 8 | 8 | 1.0 | 0.001252 |
-| `relations_event` | 8 | 8 | 1.0 | 0.0002 |
-
-All 296 of 296 requests give identical results. FluidInference's eleven fixtures (one per schema path: entities, relations,
-the three record modes, attributes, mixed and plain classification, choice fields) also match, 11 of 11, with
-confidences within 0.0004. `validation-full.json` holds the numbers.
-
-### The recorded calls (Mac, not a phone)
-
-2,521 base-v1 calls recorded from the gliner2 2.0.0 library on a synthetic English corpus of short conversational statements
-(`facts`: structure extraction; `keywords.a` and `keywords.b`: entity extraction with two label sets; `relation`:
-classification), packed by the library's collator, padded to the bucket and decoded by a Python mirror of an on-device host. ai-edge-litert
-2.2.0 on arm64 Mac (Apple M5 Max, macOS 27.0.1): the encoder on the CPU and on the Mac's GPU through the same ML Drift
-accelerator the phone uses (Metal here); head and scorer on the CPU. Mac measurements, not phone numbers.
-
-| Encoder | Runtime | Same answers as the library | Span F1 (min) | Drift p99 (max) | Mac ms p50 per graph | Gate |
-|---|---|---|---|---|---|---|
-| `wfp16` | CPU (XNNPACK; encoder 2 threads, head and scorer 1) | facts 287/287, keywords.a 287/287, keywords.b 285/287, relation 1660/1660 | 0.9966 | 0.00135 | encoder 26.28, head 0.52, scorer 2.13 | pass |
-| `wfp16` | Mac GPU (Metal, ML Drift), FP32 precision | facts 287/287, keywords.a 287/287, keywords.b 285/287, relation 1660/1660 | 0.9966 | 0.00135 | encoder 10.03, head 0.65, scorer 2.49 | pass |
-| `wfp16` | Mac GPU (Metal, ML Drift), default precision (FP16) | facts 13/287, keywords.a 112/287, keywords.b 68/287, relation 1259/1660 | 0.0 | 0.58571 | encoder 8.09, head 0.72, scorer 3.47 | fail |
-| `fp16hw` | CPU (XNNPACK; encoder 2 threads, head and scorer 1) | facts 287/287, keywords.a 287/287, keywords.b 285/287, relation 1660/1660 | 0.9966 | 0.00135 | encoder 25.4, head 0.5, scorer 2.44 | pass |
-| `fp16hw` | Mac GPU (Metal, ML Drift), FP32 precision | facts 287/287, keywords.a 287/287, keywords.b 285/287, relation 1660/1660 | 0.9966 | 0.00135 | encoder 10.33, head 0.61, scorer 3.54 | pass |
-| `fp16hw` | Mac GPU (Metal, ML Drift), default precision (FP16) | facts 286/287, keywords.a 286/287, keywords.b 283/287, relation 1654/1660 | 0.9932 | 0.03023 | encoder 8.35, head 1.96, scorer 4.3 | pass |
-| `int8` | CPU (XNNPACK; encoder 2 threads, head and scorer 1) | facts 207/287, keywords.a 229/287, keywords.b 205/287, relation 1488/1660 | 0.8348 | 0.38601 | encoder 29.58, head 0.76, scorer 3.43 | fail |
-
-The acceptance gate: per call type, same answers ≥ 95 %, span F1 ≥ 0.95, confidence drift p99 ≤ 0.05. `validation.json` holds every number.
-
-
-## On phones
-
-### Galaxy S23 Ultra (SM8550, 2026-11-02)
-
-Android 16, LiteRT 2.3.2 with `qnn-runtime` 2.50.0; each graph on a request recorded from the corpus, 10 calls after a
-30 s rest, thermal status 0 throughout; 4 CPU threads (XNNPACK). For the GPU the time is the whole call (it runs
-asynchronously).
-
-| Graph | CPU | GPU FP32, LOW priority | GPU FP32, NORMAL priority | NPU (compiled on the phone) |
-|---|---|---|---|---|
-| encoder, 128 tokens | 143 ms | 156 ms, 0.85 GB dirty | 67 ms, 0.85 GB dirty | 13 ms (`fp16hw`, 5.7 s compile) |
-| head | 1.2 ms | does not compile | does not compile | 1.2 ms |
-| scorer | 2.9 ms | does not compile | does not compile | 3.4 ms |
-| explicit spans | 5.6 ms | does not compile | does not compile | fails to run |
-| relations | 16 ms | does not compile | does not compile | 2.8 ms |
-| record assignment | 8.9 ms | 21.5 ms | not measured | 4.0 ms |
-| anchorless records | 10.8 ms | does not compile | does not compile | 1.3 ms |
-
-- **The encoder is the cost:** a request on the CPU takes about 150 ms (relations add 16 ms); with the encoder on the NPU,
-  about 20 ms.
-- **Answers on the phone's accelerators** (32 recorded requests, two per call type; the phone's encoder outputs run
-  through the rest of the pipeline): the GPU at FP32 changed none; the NPU changed two, both latent-record requests (span
-  F1 0.98, confidence up to 0.28), and none of the other call types.
-- **`fp16hw` on this phone's GPU at FP16 gives non-finite outputs** (LiteRT 2.3.2, Adreno 740), with or without FP32
-  accumulation, so it is not an accumulation overflow: use the CPU, the GPU at FP32 or the NPU there.
-- **GPU priority:** LOW is meant to let other GPU work go first, but on this phone and LiteRT version it does not let an app's
-  rendering in. An app rendering on the same GPU stopped for each whole call at either priority, and LOW made each call
-  2–3 times longer. That was measured with GLiNER2.5-Decide's encoder: see `spybyscript/gliner25-decide-litert`, "GPU
-  priority beside a rendering app". On the GPU use NORMAL; the NPU keeps an app's frames.
-
-### Galaxy Z Fold 8 (SM8850, 2026-10-29)
-
-Snapdragon 8 Elite Gen 5 (SM8850), Android 17, LiteRT 2.2.0. The encoder runs wfp16 on 4 CPU threads (XNNPACK weight cache); the head and the scorer run on one thread each:
-
-- the feature outputs and pair logits equal the Mac's LiteRT run (difference 0.0), with the same candidate pools (3 / 3);
-- the graphs take 39–61 ms a call, against 70 ms for the same call on ONNX Runtime (an FP32 export with an int8 token table);
-- a whole entity call through the host takes 46 ms.
+(LiteRT CPU), then compares the structured results:
+- **The corpus** (`runtime/parity_corpus.json`): 296 requests over 16 call types (entities, relations, structures, the
+  three record modes, attributes, single- and multi-label classification, mixed schemas). All 296 are identical, with
+  span F1 1.0 and confidences within 0.0014.
+- **FluidInference's eleven schema-path fixtures:** 11 of 11.
+
+`validation-full.json` and `validation.json` hold the numbers.
+
+## Galaxy S23 Ultra (SM8550)
+
+Snapdragon 8 Gen 2, Android 16, LiteRT 2.3.2 with `qnn-runtime` 2.50.0. Each graph ran on a request recorded from the
+corpus: 10 calls after a 30 s rest, at thermal status 0. The CPU ran 4 threads; the GPU ran at NORMAL priority, timed as
+whole calls. The NPU compiled the graphs on the phone; for the encoder that takes 5.7 s, once.
+
+| Graph | CPU | GPU FP32 | NPU |
+|---|---|---|---|
+| encoder, 128 tokens | 143 ms | 67 ms (0.85 GB dirty) | 13 ms (`fp16hw`) |
+| head | 1.2 ms | not supported by LiteRT's GPU | 1.2 ms |
+| scorer | 2.9 ms | not supported | 3.4 ms |
+| explicit spans | 5.6 ms | not supported | does not run |
+| relations | 16 ms | not supported | 2.8 ms |
+| record assignment | 8.9 ms | runs (21.5 ms at LOW priority) | 4.0 ms |
+| anchorless records | 10.8 ms | not supported | 1.3 ms |
+
+- **A request** takes about 150 ms on the CPU. With the encoder on the NPU and the heads on the CPU, it takes about
+  20 ms; relations add 16 ms on the CPU, or 3 ms on the NPU.
+- **Answers with the phone's encoder outputs.** 32 recorded requests, two per call type, ran through the rest of the
+  pipeline. At GPU FP32 they were all unchanged. On the NPU, 30 were unchanged; the two that changed were latent-record
+  requests (span F1 0.98).
+- **`fp16hw` at GPU FP16 gives non-finite outputs on this phone's Adreno 740**, with or without FP32 accumulation. Use
+  the GPU at FP32 there.
+- **GPU priority:** use NORMAL. On this phone, LOW priority does not let an app's rendering run any sooner; it only makes
+  each call 2–3 times longer. The measurement is with GLiNER2.5-Decide's encoder: see
+  [spybyscript/gliner25-decide-litert](https://huggingface.co/spybyscript/gliner25-decide-litert).
 
 ## Files
 
-`LICENSE` (Apache-2.0), `NOTICE.txt`, `model-card.md` (the original card), `config.json` (contract), `validation.json`
-(the recorded-call validation), `validation-full.json` (every call type), `SHA256SUMS`, `tokenizer/`, `source/` (the
-checkpoint's configs), `runtime/`.
+- `LICENSE` (Apache-2.0) and `NOTICE.txt`
+- `model-card.md`, the original card
+- `config.json`, the contract
+- `validation.json` and `validation-full.json`
+- `SHA256SUMS`
+- `tokenizer/`
+- `source/` (the checkpoint's configs)
+- `runtime/`

SHA256SUMS

@@ -1,6 +1,6 @@
 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30  LICENSE
 a4134d01e6b89380d84cfcde424328aaa717edadef0a6958c3129f206d91dc37  NOTICE.txt
-be689aee1556e9966f5476887227777c2c9ea296a9f395028771dc97b3b8bad6  README.md
+de378412e30d392c2437c17ab94c8a953cd192d4d8c2e7e8e93fd2a57d025b7d  README.md
 251e71c11dd57b4106d0d5eb102a984abebc5415f4e0eb392a4966e3293b189f  config.json
 22c2e937cfc17e1ee9c8f3ec6f1449d44dc9230cfcbb8737c9bae8492628a3e8  gliner2_base_embeddings_f16.bin
 6f5241586176d6b123f75392bedd66a24c63f4105f8a45181dbc02385e95e5bb  gliner2_base_encoder_fp16hw.tflite