Every trained head: explicit spans, relations and records; reference runtime; phone results

NOTICE.txt

@@ -1,5 +1,5 @@
 GLiNER2.5 base (gliner2.5-base-v1, fastino/gliner2.5-base-v1 @ 1a8bc24e00dc7300b9017c81d63e3dcdabb26596), by Fastino, licensed under the Apache License 2.0 (LICENSE). Its encoder is DeBERTa-v3-base (Microsoft, MIT License).
 
-Changes: converted to LiteRT by spybyscript (encoder wfp16, fp16hw; head and scorer FP32). The head and scorer wrappers and trace patches are adapted from FluidInference's Core ML conversion (https://huggingface.co/FluidInference/gliner2-5-base-coreml @ ca8ddf6c, Apache-2.0); the encoder rewrite is ported from litert-community/GLiNER2.5-Decide-LiteRT @ c687e899 (Apache-2.0). The token embedding table ships as a float16 file outside the graphs. Weights are the original apart from the precision each variant states. Only the encoder, the boundary head and the candidate scorer are exported; the relation-pair, explicit-span and record heads are not.
+Changes: converted to LiteRT by spybyscript (encoder wfp16, fp16hw; head and scorer FP32). The head and scorer wrappers and trace patches are adapted from FluidInference's Core ML conversion (https://huggingface.co/FluidInference/gliner2-5-base-coreml @ ca8ddf6c, Apache-2.0); the encoder rewrite is ported from litert-community/GLiNER2.5-Decide-LiteRT @ c687e899 (Apache-2.0). The token embedding table ships as a float16 file outside the graphs. Weights are the original apart from the precision each variant states. Every trained module is exported: the encoder, the boundary head with the classifier, the candidate scorer, the explicit-span scorer, the relation scorer and the record decoder; the wrappers for the last three are adapted from FluidInference's extraction_export.py. runtime/litert_boundary_runtime.py is ported from FluidInference's extraction_runtime.py, preprocessing.py and extraction_pool.py (Apache-2.0).
 
 model-card.md is the original model card; README.md describes this conversion.

README.md

@@ -14,44 +14,98 @@ tags:
 
 # GLiNER2.5 base (gliner2.5-base-v1) — LiteRT (Android)
 
-An on-device conversion of gliner2.5-base-v1 for Android's LiteRT, validated against the gliner2 library and measured on a phone. We know of no other LiteRT conversion of gliner2.5-base-v1.
+An on-device conversion of gliner2.5-base-v1 for Android's LiteRT with every trained head: entities, classification,
+structures, relations and records, decoded by gliner2's own code, validated against the library and measured on phones.
+We know of no other LiteRT conversion of gliner2.5-base-v1.
 
 Converted from [fastino/gliner2.5-base-v1](https://huggingface.co/fastino/gliner2.5-base-v1) at revision `1a8bc24e00dc7300b9017c81d63e3dcdabb26596` (Apache-2.0, by Fastino).
-It keeps the contract of FluidInference's Core ML port, with the token table on the
-host: an encoder graph (`inputs_embeds`, `attention_mask` → `hidden_states`), a head graph (the features graph's other inputs and
-its thirteen outputs) and the candidate scorer; the host's pool and decoders between and after them. `config.json` is the full contract.
+The split into graphs follows FluidInference's Core ML port ([gliner2-5-base-coreml](https://huggingface.co/FluidInference/gliner2-5-base-coreml), Apache-2.0),
+with the token table on the host. `config.json` is the full contract.
 
 | File | Size | Contents |
 |---|---|---|
-| `gliner2_base_encoder_wfp16.tflite` | 189 MB | encoder: encoder FULLY_CONNECTED weights stored float16 behind DEQUANTIZE (ai-edge-quantizer FLOAT_CASTING); FP32 activations, position tables and norms; finite -1e4 mask; head and scorer FP32 |
-| `gliner2_base_encoder_fp16hw.tflite` | 189 MB | encoder: wfp16's storage over the FP16-hardware encoder: LayerNorm power-of-two pre-scale (x*2^-k, eps*4^-k, k from the corpus' FP32 ranges), first feed-forward FULLY_CONNECTED split in two halves; head and scorer FP32 |
-| `gliner2_base_head_fp32.tflite` | 8.5 MB | head, FP32 |
-| `gliner2_base_scorer_fp32.tflite` | 2.1 MB | scorer, FP32 |
+| `gliner2_base_encoder_wfp16.tflite` | 189 MB | encoder (`inputs_embeds`, `attention_mask` → `hidden_states`): FULLY_CONNECTED weights stored float16 behind DEQUANTIZE; FP32 activations, position tables and norms; finite -1e4 mask |
+| `gliner2_base_encoder_fp16hw.tflite` | 189 MB | the same encoder rewritten for FP16 hardware: LayerNorm power-of-two pre-scale (x*2^-k, eps*4^-k), first feed-forward FULLY_CONNECTED split in two halves |
+| `gliner2_base_head_fp32.tflite` | 8.5 MB | boundary head: routed text and query states, boundary marginals, pool projections, null and count heads, classification logits |
+| `gliner2_base_scorer_fp32.tflite` | 2.1 MB | candidate scorer and the record candidate states |
+| `gliner2_base_explicit_fp32.tflite` | 2.1 MB | explicit-span scorer: the boundary proposer's prior and the pair scorer over forced spans (entity attributes, choice fields) |
+| `gliner2_base_relation_fp32.tflite` | 23.7 MB | relation scorer over head/tail pairs |
+| `gliner2_base_record_assignment_fp32.tflite` | 1.2 MB | record decoder: instance-to-field assignment, object and latent-seed scores |
+| `gliner2_base_record_anchorless_fp32.tflite` | 2.9 MB | record decoder: learned instances attending over the field candidates (anchorless records) |
 | `gliner2_base_embeddings_f16.bin` | 197 MB | float16 [128011, 768], little-endian, row-major |
-
-## What this port does, and what it leaves out
-
-The original checkpoint has an encoder, a boundary head, a classifier, a record decoder and a relation scorer. This
-port exports the encoder, a head graph (the boundary head's extraction outputs, the null and count outputs and the
-classifier's choice logits) and the candidate scorer. The host runs the library's candidate pool and decoding between
-and after them.
-
-- **Entities**, as `extract_entities` returns them: labels with descriptions, thresholds, and the null head's
-  abstention.
-- **Classification**, as `classify_text`: up to 8 choices.
-- **Structures whose fields are plain spans**, as `extract_json` returns them through the library's structure decoder
-  without record mode.
-- These three were validated on 2,521 recorded calls (see Validation).
-- **Not available:** relation extraction (`extract_relations`; the original's `relation_scorer`) and record-mode
-  structures (the `record_decoder`, which groups fields into several records), nor the explicit-span head of
-  FluidInference's Core ML port. Requests that need them cannot run on this port.
-- **Fixed sizes:** one call holds at most 128 subword tokens (prompt and text), 64 text words, 8 queries (entity labels
-  or structure fields) and 8 classification choices, with a candidate pool of 192. The host refuses larger requests
-  rather than truncating them. The original runs with dynamic shapes, so long texts and large label sets need several
-  calls here.
-- **Weights:** only their storage and compute precision change (see the file table); nothing is retrained or pruned.
-
-## Validation (Mac, not a phone)
+| `runtime/` | | `litert_boundary_runtime.py` (the reference runtime), `verify_parity.py`, `parity_corpus.json` |
+
+## What it runs
+
+`runtime/litert_boundary_runtime.py` is gliner2's BoundaryExtractor with every trained module on LiteRT: the schema
+handling, the candidate pool and the decoders are the library's own code, and no PyTorch weights are loaded. Every
+public call works: `extract`, `extract_entities`, `classify_text`, `extract_json`, `extract_relations`, and their
+`batch_*` and `*_long` forms.
+
+- Entities, with label descriptions, thresholds and the null head's abstention; entity attributes.
+- Classification, single- and multi-label.
+- Structures (`extract_json`), with choice fields and the natural, latent and anchorless record modes.
+- Relations.
+
+**Fixed sizes.** One request holds at most 128 subword tokens (schema and text), 64 text words, 8 queries (entity
+labels, structure fields, relation roles) and 8 classification choices, with a candidate pool of 192 (the checkpoint's
+own pool size). Relations take up to 4 types and 256 head/tail pairs per request; explicit spans up to 64 per query;
+records up to 8 fields with 192 candidates each. These are FluidInference's bucket sizes. A larger request is refused
+with a ValueError, never truncated: run long text through the `*_long` calls with a chunk that fits, e.g.
+`extract_entities_long(text, labels, chunk_size=40, chunk_overlap=8)`. The original runs with dynamic shapes.
+
+**Weights:** only their storage and compute precision change; nothing is retrained or pruned.
+
+## Use
+
+```python
+# pip install gliner2==2.0.0 torch numpy ai-edge-litert
+import sys; sys.path.insert(0, "runtime")
+from litert_boundary_runtime import LiteRTBoundaryExtractor
+from gliner2 import Schema
+
+model = LiteRTBoundaryExtractor(".")          # this repository's folder
+model.extract_entities("Alice founded Acme in Toronto in 2020.", ["person", "organization", "city"])
+model.extract_relations("Alice founded Acme in Toronto.", ["founded", "located_in"])
+schema = Schema().entities(["person", "organization"]).relations(["works_for"])
+model.extract("Ravi Shah is an engineer at Tidewater Systems.", schema, include_confidence=True)
+```
+
+On a phone, run the same graphs with LiteRT's `CompiledModel` and port the library's decoding steps the runtime uses
+(`config.json` lists every graph's signature).
+
+## Validation
+
+### Every call type against the library
+
+`runtime/verify_parity.py` runs each request through gliner2 with the original weights and through the reference runtime
+(LiteRT 2.2.0 on the CPU, `wfp16` encoder), and compares the structured results. The corpus
+(`runtime/parity_corpus.json`) is 48 short synthetic texts with schema templates for every call type, 296 requests.
+
+| Call type | Requests | Identical results | Span F1 (lowest) | Largest confidence difference |
+|---|---|---|---|---|
+| `entities` | 40 | 40 | 1.0 | 0.000228 |
+| `entities_single` | 8 | 8 | 1.0 | 5e-06 |
+| `relations_work` | 24 | 24 | 1.0 | 0.000883 |
+| `record_natural` | 16 | 16 | 1.0 | 0.00013 |
+| `record_latent` | 24 | 24 | 1.0 | 0.000641 |
+| `attributes` | 16 | 16 | 1.0 | 0.000298 |
+| `classification_topic` | 40 | 40 | 1.0 | 0.001305 |
+| `mixed` | 16 | 16 | 1.0 | 0.001358 |
+| `relations_family` | 8 | 8 | 1.0 | 0.00033 |
+| `classification_described` | 24 | 24 | 1.0 | 0.001187 |
+| `record_anchorless` | 8 | 8 | 1.0 | 0.000258 |
+| `structure_event` | 24 | 24 | 1.0 | 0.000897 |
+| `classification_multi` | 16 | 16 | 1.0 | 0.000356 |
+| `entities_described` | 16 | 16 | 1.0 | 0.000411 |
+| `structure_order` | 8 | 8 | 1.0 | 0.001252 |
+| `relations_event` | 8 | 8 | 1.0 | 0.0002 |
+
+All 296 of 296 requests give identical results. FluidInference's eleven fixtures (one per schema path: entities, relations,
+the three record modes, attributes, mixed and plain classification, choice fields) also match, 11 of 11, with
+confidences within 0.0004. `validation-full.json` holds the numbers.
+
+### The recorded calls (Mac, not a phone)
 
 2,521 base-v1 calls recorded from the gliner2 2.0.0 library on a synthetic English corpus of short conversational statements
 (`facts`: structure extraction; `keywords.a` and `keywords.b`: entity extraction with two label sets; `relation`:
@@ -71,7 +125,34 @@ accelerator the phone uses (Metal here); head and scorer on the CPU. Mac measure
 
 The acceptance gate: per call type, same answers ≥ 95 %, span F1 ≥ 0.95, confidence drift p99 ≤ 0.05. `validation.json` holds every number.
 
-## On a phone (Galaxy Z Fold 8, 2026-10-29)
+
+## On phones
+
+### Galaxy S23 Ultra (SM8550, 2026-11-02)
+
+Android 16, LiteRT 2.3.2 with `qnn-runtime` 2.50.0; each graph on a request recorded from the corpus, 10 calls after a
+30 s rest, thermal status 0 throughout; 4 CPU threads (XNNPACK). For the GPU the time is the whole call (it runs
+asynchronously).
+
+| Graph | CPU | GPU (FP32, low priority) | NPU (compiled on the phone) |
+|---|---|---|---|
+| encoder, 128 tokens | 143 ms | 156 ms, 0.85 GB dirty | 13 ms (`fp16hw`, 5.7 s compile) |
+| head | 1.2 ms | does not compile | 1.2 ms |
+| scorer | 2.9 ms | does not compile | 3.4 ms |
+| explicit spans | 5.6 ms | does not compile | fails to run |
+| relations | 16 ms | does not compile | 2.8 ms |
+| record assignment | 8.9 ms | 21.5 ms | 4.0 ms |
+| anchorless records | 10.8 ms | does not compile | 1.3 ms |
+
+- **The encoder is the cost:** a request on the CPU takes about 150 ms (relations add 16 ms); with the encoder on the NPU,
+  about 20 ms.
+- **Answers on the phone's accelerators** (32 recorded requests, two per call type; the phone's encoder outputs run
+  through the rest of the pipeline): the GPU at FP32 changed none; the NPU changed two, both latent-record requests (span
+  F1 0.98, confidence up to 0.28), and none of the other call types.
+- **`fp16hw` on this phone's GPU at FP16 gives non-finite outputs** (LiteRT 2.3.2, Adreno 740): use the CPU, the GPU at
+  FP32 or the NPU there.
+
+### Galaxy Z Fold 8 (SM8850, 2026-10-29)
 
 Snapdragon 8 Elite Gen 5 (SM8850), Android 17, LiteRT 2.2.0. The encoder runs wfp16 on 4 CPU threads (XNNPACK weight cache); the head and the scorer run on one thread each:
 
@@ -81,5 +162,6 @@ Snapdragon 8 Elite Gen 5 (SM8850), Android 17, LiteRT 2.2.0. The encoder runs wf
 
 ## Files
 
-`LICENSE` (Apache-2.0), `NOTICE.txt`, `model-card.md` (the original card), `config.json` (contract), `validation.json`,
-`SHA256SUMS`, `tokenizer/`, `source/` (the checkpoint's configs).
+`LICENSE` (Apache-2.0), `NOTICE.txt`, `model-card.md` (the original card), `config.json` (contract), `validation.json`
+(the recorded-call validation), `validation-full.json` (every call type), `SHA256SUMS`, `tokenizer/`, `source/` (the
+checkpoint's configs), `runtime/`.

SHA256SUMS

@@ -1,15 +1,24 @@
 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30  LICENSE
-5d22fdeddc1ceccba0576afa514f74b43411afe769562ad1ac480c529d7cf0a1  NOTICE.txt
-c61268a2b9b76631c2c630a2cf626242a753983211f2bd305aa7300955a6253c  README.md
-4c7cc23f44739e7a2633861f820df1576ffe9456d0613916f602ffbcf44f2fda  config.json
+a4134d01e6b89380d84cfcde424328aaa717edadef0a6958c3129f206d91dc37  NOTICE.txt
+f3d539b48e161356bae4ee2b7acf9e0126a0bce850743cf3179bc84130f6fe13  README.md
+251e71c11dd57b4106d0d5eb102a984abebc5415f4e0eb392a4966e3293b189f  config.json
 22c2e937cfc17e1ee9c8f3ec6f1449d44dc9230cfcbb8737c9bae8492628a3e8  gliner2_base_embeddings_f16.bin
 6f5241586176d6b123f75392bedd66a24c63f4105f8a45181dbc02385e95e5bb  gliner2_base_encoder_fp16hw.tflite
 053cc6376843ee052a01f2c78e813e1f273ad73e71b95558a4b29804d6250e3c  gliner2_base_encoder_wfp16.tflite
+03a08c79fdd6ffc6de3e881af5cea7a369d7b3205017915396b9203d7a982051  gliner2_base_explicit_fp32.tflite
 1dd500e1c321acd7e84c1bec7fba5afb4211f41081f8f6dc0f8a4a78bffc3bd9  gliner2_base_head_fp32.tflite
+dcc066dce8b721f39f4870459efb0abd3c5366f58498a9ff062c4065f9ee24d3  gliner2_base_record_anchorless_fp32.tflite
+e6169de37236a4d056edefcaabfb1e9bba4df1ee34d45f8d6436dc94e22195f5  gliner2_base_record_assignment_fp32.tflite
+dcf3528768b742dc6328c4441ae58694606a3fe15b3500ab429e13bf1caf4ed3  gliner2_base_relation_fp32.tflite
 b828fed73d474ef01f4608f6f6a892cb9a1619c1ca3899f86f6ce33c2ebbc4ee  gliner2_base_scorer_fp32.tflite
 b498995df84d4b6780cc25b2be763dc554eab173fed80134e738b2105be2bce6  model-card.md
+14811c155fd1fe6caf235276c83952ec4934225cfcf9eb3765ecf7fa2cef29ab  runtime/litert_boundary_runtime.py
+a683dc6408f8ea3ec8fbe7a925f1c4bd61ee387944a5ee4793153bedbaeba057  runtime/parity_corpus.json
+f84dbbdf154d3dbcfdb6e979a4ed8c6d2ab2cded4ed9d6f6990df0828c60d9a5  runtime/requirements.txt
+b530aaa4b5dc9fdddcae4549457b8d0b2eb534044f10a1e2600515e475b9ac52  runtime/verify_parity.py
 0eb92d00584d613aab32b2178f84a85176b62c87ae3689ce9084e83f6eba64d1  source/config.json
 d36a845b9f25dcaf1ec45a1c4bdf65ea4ac20596537e14530ec9f660a63aeca4  source/encoder_config/config.json
 cbc8ae6037812709c9c26f2a160f8dc48b0440bcb79c8141804259ae2d6adac3  tokenizer/tokenizer.json
 0bf3ea0873234bd9bfdd3853c440395009ac6365a925b91654daed5396d655e1  tokenizer/tokenizer_config.json
+81973c2e176ab0e6e3bd283185f46a2daf73bd83b7dff75aaca9dc1581ee58df  validation-full.json
 fcdbb9325f3a84f338156b04e0def258fd16efe02195ec12bdb96c185cfc253b  validation.json

config.json

@@ -106,7 +106,7 @@
   {
    "path": "gliner2_base_head_fp32.tflite",
    "role": "head",
-   "about": "the ONNX features graph after its encoder; FP32; CPU",
+   "about": "the boundary head after the encoder (FluidInference's Core ML features graph without its encoder): FP32, CPU",
    "bytes": 8517024,
    "signatures": [
     {
@@ -291,7 +291,7 @@
   {
    "path": "gliner2_base_scorer_fp32.tflite",
    "role": "scorer",
-   "about": "the ONNX scorer unchanged; FP32; CPU",
+   "about": "the candidate scorer and the record candidate states (FluidInference's Core ML scorer): FP32, CPU",
    "bytes": 2126928,
    "signatures": [
     {
@@ -424,6 +424,325 @@
      ]
     }
    ]
+  },
+  {
+   "path": "gliner2_base_explicit_fp32.tflite",
+   "role": "explicit",
+   "about": "explicit-span scorer: the boundary proposer's prior and the pair scorer over up to 64 forced spans per query (attributes, choice fields); FP32, CPU",
+   "bytes": 2051844,
+   "signatures": [
+    {
+     "key": "w64_q8_s64",
+     "inputs": [
+      {
+       "name": "text_states",
+       "dtype": "float32",
+       "shape": [
+        1,
+        64,
+        768
+       ]
+      },
+      {
+       "name": "text_mask",
+       "dtype": "float32",
+       "shape": [
+        1,
+        64
+       ]
+      },
+      {
+       "name": "query_states",
+       "dtype": "float32",
+       "shape": [
+        1,
+        8,
+        768
+       ]
+      },
+      {
+       "name": "query_mask",
+       "dtype": "float32",
+       "shape": [
+        1,
+        8
+       ]
+      },
+      {
+       "name": "boundary_states",
+       "dtype": "float32",
+       "shape": [
+        1,
+        65,
+        128
+       ]
+      },
+      {
+       "name": "start_logits",
+       "dtype": "float32",
+       "shape": [
+        1,
+        8,
+        65
+       ]
+      },
+      {
+       "name": "end_logits",
+       "dtype": "float32",
+       "shape": [
+        1,
+        8,
+        65
+       ]
+      },
+      {
+       "name": "inside_prefix",
+       "dtype": "float32",
+       "shape": [
+        1,
+        8,
+        65
+       ]
+      },
+      {
+       "name": "inside_prefix_mean",
+       "dtype": "float32",
+       "shape": [
+        1,
+        8,
+        1
+       ]
+      },
+      {
+       "name": "span_indices",
+       "dtype": "int32",
+       "shape": [
+        1,
+        8,
+        64,
+        2
+       ]
+      },
+      {
+       "name": "span_mask",
+       "dtype": "float32",
+       "shape": [
+        1,
+        8,
+        64
+       ]
+      }
+     ],
+     "outputs": [
+      {
+       "name": "span_logits",
+       "dtype": "float32",
+       "shape": [
+        1,
+        8,
+        64
+       ]
+      }
+     ]
+    }
+   ]
+  },
+  {
+   "path": "gliner2_base_relation_fp32.tflite",
+   "role": "relation",
+   "about": "relation scorer over up to 256 head/tail pairs of up to 4 relation types (pairs chosen by gliner2's typed pair generator); FP32, CPU",
+   "bytes": 23658524,
+   "signatures": [
+    {
+     "key": "w64_r4_p256",
+     "inputs": [
+      {
+       "name": "text_states",
+       "dtype": "float32",
+       "shape": [
+        1,
+        64,
+        768
+       ]
+      },
+      {
+       "name": "text_length",
+       "dtype": "int32",
+       "shape": [
+        1
+       ]
+      },
+      {
+       "name": "relation_states",
+       "dtype": "float32",
+       "shape": [
+        1,
+        4,
+        1536
+       ]
+      },
+      {
+       "name": "batch_index",
+       "dtype": "int32",
+       "shape": [
+        256
+       ]
+      },
+      {
+       "name": "relation_index",
+       "dtype": "int32",
+       "shape": [
+        256
+       ]
+      },
+      {
+       "name": "head_start",
+       "dtype": "int32",
+       "shape": [
+        256
+       ]
+      },
+      {
+       "name": "head_end",
+       "dtype": "int32",
+       "shape": [
+        256
+       ]
+      },
+      {
+       "name": "tail_start",
+       "dtype": "int32",
+       "shape": [
+        256
+       ]
+      },
+      {
+       "name": "tail_end",
+       "dtype": "int32",
+       "shape": [
+        256
+       ]
+      },
+      {
+       "name": "pair_mask",
+       "dtype": "float32",
+       "shape": [
+        256
+       ]
+      }
+     ],
+     "outputs": [
+      {
+       "name": "relation_logits",
+       "dtype": "float32",
+       "shape": [
+        256
+       ]
+      }
+     ]
+    }
+   ]
+  },
+  {
+   "path": "gliner2_base_record_assignment_fp32.tflite",
+   "role": "record_assignment",
+   "about": "record decoder: instance-to-field assignment over 8 fields with up to 192 candidates each, for up to 1536 instances, plus object and latent-seed scores; FP32, CPU",
+   "bytes": 1193288,
+   "signatures": [
+    {
+     "key": "f8_c192_i1536",
+     "inputs": [
+      {
+       "name": "instance_states",
+       "dtype": "float32",
+       "shape": [
+        1536,
+        768
+       ]
+      },
+      {
+       "name": "field_queries",
+       "dtype": "float32",
+       "shape": [
+        8,
+        768
+       ]
+      },
+      {
+       "name": "field_candidate_states",
+       "dtype": "float32",
+       "shape": [
+        8,
+        192,
+        768
+       ]
+      }
+     ],
+     "outputs": [
+      {
+       "name": "assignment_logits",
+       "dtype": "float32",
+       "shape": [
+        1536,
+        8,
+        193
+       ]
+      },
+      {
+       "name": "object_logits",
+       "dtype": "float32",
+       "shape": [
+        1536
+       ]
+      },
+      {
+       "name": "latent_seed_logits",
+       "dtype": "float32",
+       "shape": [
+        1536
+       ]
+      }
+     ]
+    }
+   ]
+  },
+  {
+   "path": "gliner2_base_record_anchorless_fp32.tflite",
+   "role": "record_anchorless",
+   "about": "record decoder: its 32 learned instances attending over up to 1536 field candidates (anchorless records); FP32, CPU",
+   "bytes": 2875604,
+   "signatures": [
+    {
+     "key": "c1536",
+     "inputs": [
+      {
+       "name": "context_states",
+       "dtype": "float32",
+       "shape": [
+        1536,
+        768
+       ]
+      },
+      {
+       "name": "context_mask",
+       "dtype": "float32",
+       "shape": [
+        1536
+       ]
+      }
+     ],
+     "outputs": [
+      {
+       "name": "instance_states",
+       "dtype": "float32",
+       "shape": [
+        32,
+        768
+       ]
+      }
+     ]
+    }
+   ]
   }
  ],
  "table": {
@@ -446,16 +765,16 @@
   "choices": 8,
   "pool": 192,
   "boundaries": 65,
-  "about": "the host pads to T=128 tokens, W=64 words, Q=8 queries, C=8 choices, P=192 candidates, as BaseExtractor does, and refuses larger requests"
+  "about": "the host pads to T=128 tokens, W=64 words, Q=8 queries, C=8 choices, P=192 candidates and refuses larger requests"
  },
  "host": {
   "steps": "as the gliner2 library does: pack and pad the request, then encoder -> head -> the shared candidate pool on the host -> scorer -> thresholds, flat overlap resolution and spans; entities abstain when sigmoid(null_logits) > 0.5",
   "changes": [
    "input_ids no longer enter a graph: the host looks them up in the float16 table into inputs_embeds",
    "attention_mask is float32 (1.0 / 0.0) instead of int32",
-   "the ONNX features graph is two graphs: encoder (inputs_embeds, attention_mask -> hidden_states) and head (hidden_states plus the six other ONNX feature inputs -> the same thirteen outputs)",
-   "the scorer is the ONNX scorer (same inputs and outputs), FP32",
-   "bind inputs by name; signatures: encoder `t128`, head `t128`, scorer `p192`"
+   "FluidInference's features graph is two graphs: encoder (inputs_embeds, attention_mask -> hidden_states) and head (hidden_states plus the six other feature inputs -> the same thirteen outputs)",
+   "the scorer, explicit-span, relation and record graphs take FluidInference's inputs and return its outputs, FP32",
+   "bind inputs by name; signatures: encoder `t128`, head `t128`, scorer `p192`, explicit `w64_q8_s64`, relation `w64_r4_p256`, record assignment `f8_c192_i1536`, anchorless records `c1536`"
   ]
  },
  "accelerators": {
@@ -484,5 +803,6 @@
  },
  "provenance": {
   "graphs": "encoder graph code ported from litert-community/GLiNER2.5-Decide-LiteRT (Apache-2.0); head and scorer wrappers adapted from FluidInference/gliner2-5-base-coreml (Apache-2.0)"
- }
+ },
+ "referenceRuntime": "runtime/litert_boundary_runtime.py (gliner2 2.0.0's BoundaryExtractor over these graphs)"
 }

gliner2_base_explicit_fp32.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03a08c79fdd6ffc6de3e881af5cea7a369d7b3205017915396b9203d7a982051
+size 2051844

gliner2_base_record_anchorless_fp32.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dcc066dce8b721f39f4870459efb0abd3c5366f58498a9ff062c4065f9ee24d3
+size 2875604

gliner2_base_record_assignment_fp32.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6169de37236a4d056edefcaabfb1e9bba4df1ee34d45f8d6436dc94e22195f5
+size 1193288

gliner2_base_relation_fp32.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dcf3528768b742dc6328c4441ae58694606a3fe15b3500ab429e13bf1caf4ed3
+size 23658524

runtime/litert_boundary_runtime.py

@@ -0,0 +1,490 @@
+"""gliner2.5-base-v1 on LiteRT with every trained head: gliner2's own schema handling and decoders over seven graphs.
+
+Ported from FluidInference/gliner2-5-base-coreml@ca8ddf6c (Apache-2.0): extraction_runtime.py (the Core ML runtime),
+preprocessing.py (prepare_extraction) and extraction_pool.py (select_candidates). Changes: LiteRT CompiledModel graphs in
+place of Core ML packages; the features stage is two graphs (the encoder over the host-mapped float16 token table, then
+the head); the bucket sizes come from the graphs.
+
+    from litert_boundary_runtime import LiteRTBoundaryExtractor
+    from gliner2 import Schema
+    model = LiteRTBoundaryExtractor("path/to/bundle")
+    model.extract("Alice founded Acme in Toronto.", Schema().entities(["person", "organization"]).relations(["founded"]))
+    model.extract_relations("Alice founded Acme in Toronto.", ["founded", "located_in"])
+
+Every public call of gliner2's extractor works: extract, batch_extract, extract_entities, classify_text, extract_json,
+extract_relations and their batch_* and *_long forms. The graphs have fixed sizes (128 subword tokens, 64 text words,
+8 queries and 8 classification choices per request); a larger request is refused with a ValueError, never truncated.
+For long text use the *_long calls with a chunk_size that fits, e.g. extract_entities_long(text, labels, chunk_size=40,
+chunk_overlap=8).
+
+Needs gliner2==2.0.0 (schema, collator and decoders; no PyTorch weights are loaded), torch (the library's tensor
+types), numpy and ai-edge-litert. The bundle folder holds the .tflite graphs, the float16 table, tokenizer/ and
+source/config.json (the checkpoint's configuration).
+"""
+from __future__ import annotations
+
+import json
+import math
+from contextvars import ContextVar
+from pathlib import Path
+
+import numpy as np
+import torch
+from gliner2.configuration import BoundaryHeadSettings
+from gliner2.models.boundary.constants import MASK_LOGIT
+from gliner2.models.boundary.engine import BoundaryExtractor
+from gliner2.models.boundary.indexing import gather_rows
+from gliner2.models.boundary.pool import PooledCandidates, _deduplicate_pool
+from gliner2.models.boundary.proposal import select_top_boundaries
+from gliner2.models.boundary.records import RecordGroupOutput
+from gliner2.models.boundary.relations import RelationProposalSettings, RelationTypeSpec, TypedRelationPairGenerator
+from gliner2.models.base import load_extractor_tokenizer
+from gliner2.models.outputs import CandidateTensorBatch, ExtractorOutput
+from gliner2.processor import SchemaTransformer
+from gliner2.training.trainer import ExtractorCollator
+
+FEATURE_NAMES = ("text_states", "query_states", "boundary_states", "boundary_mask", "start_logits", "end_logits", "inside_prefix",
+                 "inside_prefix_mean", "pool_start_projection", "pool_end_projection", "null_logits", "count_log_rates",
+                 "classification_logits")
+HEAD_INPUTS = ("hidden_states", "text_indices", "text_mask", "query_indices", "query_mask", "cls_indices", "cls_mask")
+SCORER_INPUTS = ("text_states", "text_mask", "query_states", "query_mask", "boundary_states", "start_logits", "end_logits",
+                 "inside_prefix", "inside_prefix_mean", "candidate_indices", "candidate_mask", "candidate_compatibility")
+TABLE = "gliner2_base_embeddings_f16.bin"
+FILES = {
+    "head": "gliner2_base_head_fp32.tflite", "scorer": "gliner2_base_scorer_fp32.tflite",
+    "explicit": "gliner2_base_explicit_fp32.tflite", "relation": "gliner2_base_relation_fp32.tflite",
+    "record_assignment": "gliner2_base_record_assignment_fp32.tflite", "record_anchorless": "gliner2_base_record_anchorless_fp32.tflite",
+}
+
+
+class Graph:
+    """One signature of a LiteRT CompiledModel: run(feed) -> {output name: float32 array}."""
+
+    def __init__(self, path: Path, signature: str | None = None, threads: int = 4, accelerator: str = "cpu", cache: str = ""):
+        from ai_edge_litert.compiled_model import CompiledModel, CpuOptions, GpuOptions, HardwareAccelerator, Options
+        if accelerator == "cpu":
+            options = Options(HardwareAccelerator.CPU, cpu_options=CpuOptions(num_threads=threads, xnnpack_weight_cache_path=cache))
+        else:  # "gpu" (FP32) or "gpu-fp16"
+            options = Options(HardwareAccelerator.GPU,
+                              gpu_options=GpuOptions(enforce_f32=accelerator == "gpu", constant_tensor_sharing=True))
+        self.model = CompiledModel.from_file(str(path), options=options)
+        if signature is None:
+            signature = self.model.get_signature_list()[0] if hasattr(self.model, "get_signature_list") else None
+        self.index = self.model.get_signature_index(signature) if signature else 0
+        sig = self.model.get_signature_by_index(self.index)
+        self.signature = signature or sig.get("key")
+        self.inputs, self.outputs = sig["inputs"], sig["outputs"]
+        self.ins, self.outs = self.model.create_input_buffers(self.index), self.model.create_output_buffers(self.index)
+        details = self.model.get_input_tensor_details(self.signature)
+        self.dtypes = {k: (np.int32 if "int" in str(v.get("dtype", "")).lower() else np.float32) for k, v in details.items()}
+        self.input_shapes = {k: tuple(int(x) for x in v["shape"]) for k, v in details.items()}
+        self.shapes = {k: tuple(int(x) for x in v["shape"]) for k, v in self.model.get_output_tensor_details(self.signature).items()}
+
+    def __call__(self, feed: dict) -> dict:
+        for buffer, name in zip(self.ins, self.inputs):
+            buffer.write(np.ascontiguousarray(np.asarray(feed[name]), self.dtypes[name]))
+        self.model.run_by_index(self.index, self.ins, self.outs)
+        return {name: buffer.read(int(np.prod(self.shapes[name])), np.float32).reshape(self.shapes[name])
+                for buffer, name in zip(self.outs, self.outputs)}
+
+
+def signature_of(path: Path) -> str:
+    from ai_edge_litert.compiled_model import CompiledModel
+    model = CompiledModel.from_file(str(path))
+    key = model.get_signature_by_index(0)["key"]
+    model.close()
+    return key
+
+
+def to_tensor(value):
+    return torch.from_numpy(np.asarray(value).copy())
+
+
+def padded(value, size):
+    if value.shape[0] > size:
+        raise ValueError(f"Request exceeds the bucket capacity {size}")
+    result = value.new_zeros((size, *value.shape[1:]))
+    result[: value.shape[0]] = value
+    return result
+
+
+def prepare_extraction(processor, text: str, schema, length: int, max_words: int, max_queries: int, max_choices: int):
+    """gliner2's collator on one request, padded to the bucket; refuses requests that do not fit (never truncates)."""
+    built = schema.build() if hasattr(schema, "build") else schema
+    batch = ExtractorCollator(processor, is_training=False, max_len=None, architecture="boundary")([(text, built)])
+    if batch.input_ids.shape[1] > length:
+        raise ValueError(f"Schema and text require {batch.input_ids.shape[1]} subwords; bucket holds {length}")
+    if batch.text_word_indices.shape[1] > max_words:
+        raise ValueError(f"Text requires {batch.text_word_indices.shape[1]} words; bucket holds {max_words}")
+    if batch.query_marker_indices.shape[1] > max_queries:
+        raise ValueError(f"Schema requires {batch.query_marker_indices.shape[1]} queries; bucket holds {max_queries}")
+    if batch.cls_marker_indices.shape[1] > max_choices:
+        raise ValueError(f"Schema requires {batch.cls_marker_indices.shape[1]} choices; bucket holds {max_choices}")
+    if batch.query_marker_indices.shape[1] == 0 and batch.cls_marker_indices.shape[1] == 0:
+        raise ValueError("Schema has no extraction or classification queries")
+
+    def pad(values, width, fill=0):
+        array = values.numpy()
+        return np.pad(array, ((0, 0), (0, width - array.shape[1])), constant_values=fill)
+
+    arrays = {
+        "input_ids": pad(batch.input_ids, length, processor.tokenizer.pad_token_id).astype(np.int32),
+        "attention_mask": pad(batch.attention_mask, length).astype(np.int32),
+        "text_indices": pad(batch.text_word_indices, max_words).astype(np.int32),
+        "text_mask": pad(batch.text_word_mask, max_words).astype(np.float32),
+        "query_indices": pad(batch.query_marker_indices, max_queries).astype(np.int32),
+        "query_mask": pad(batch.query_marker_mask, max_queries).astype(np.float32),
+        "cls_indices": pad(batch.cls_marker_indices, max_choices).astype(np.int32),
+        "cls_mask": pad(batch.cls_marker_mask, max_choices).astype(np.float32),
+    }
+    return arrays, batch
+
+
+def select_candidates(start_projection, end_projection, boundary_mask, query_mask, start_logits, end_logits, *, boundary_top_k: int,
+                      pool_size: int, min_pool_per_query: int) -> PooledCandidates:
+    """gliner2 2.0.0's shared candidate pool (stable ranking and deduplication) from the head graph's projections."""
+    batch, n_boundaries, dim = start_projection.shape
+    n_queries = query_mask.shape[1]
+    floor = torch.full_like(start_logits, MASK_LOGIT)
+    valid_boundary = boundary_mask.unsqueeze(1) & query_mask.unsqueeze(-1)
+    union_start = torch.where(valid_boundary, start_logits, floor).amax(1)
+    union_end = torch.where(valid_boundary, end_logits, floor).amax(1)
+    union_valid = boundary_mask & query_mask.any(-1, keepdim=True)
+    _, starts, starts_valid = select_top_boundaries(union_start.unsqueeze(1), union_valid.unsqueeze(1), boundary_top_k)
+    _, ends, ends_valid = select_top_boundaries(union_end.unsqueeze(1), union_valid.unsqueeze(1), boundary_top_k)
+    starts, ends = starts[:, 0], ends[:, 0]
+    starts_valid, ends_valid = starts_valid[:, 0], ends_valid[:, 0]
+    n_starts, n_ends = starts.shape[1], ends.shape[1]
+    pair_start = starts.unsqueeze(-1).expand(batch, n_starts, n_ends).reshape(batch, -1)
+    pair_end = ends.unsqueeze(1).expand(batch, n_starts, n_ends).reshape(batch, -1)
+    pair_valid = (starts_valid.unsqueeze(-1) & ends_valid.unsqueeze(1) & (ends.unsqueeze(1) > starts.unsqueeze(-1))).reshape(batch, -1)
+    compatibility = (gather_rows(start_projection, pair_start) * gather_rows(end_projection, pair_end)).sum(-1) / math.sqrt(dim)
+    union_pair_score = (compatibility + union_start.gather(1, pair_start.clamp(0, n_boundaries - 1))
+                        + union_end.gather(1, pair_end.clamp(0, n_boundaries - 1)))
+    quota = min(min_pool_per_query, pair_start.shape[-1])
+    quota_keys = pair_start.new_zeros((batch, 0))
+    quota_scores = union_pair_score.new_zeros((batch, 0))
+    quota_valid = pair_valid.new_zeros((batch, 0))
+    if quota:
+        start_idx = pair_start.clamp(0, start_logits.shape[2] - 1).unsqueeze(1).expand(batch, n_queries, -1)
+        end_idx = pair_end.clamp(0, end_logits.shape[2] - 1).unsqueeze(1).expand(batch, n_queries, -1)
+        per_query = start_logits.gather(2, start_idx) + end_logits.gather(2, end_idx) + compatibility.unsqueeze(1)
+        per_query_valid = pair_valid.unsqueeze(1) & query_mask.unsqueeze(-1)
+        ranked = torch.argsort(per_query.masked_fill(~per_query_valid, MASK_LOGIT), dim=-1, descending=True, stable=True)[..., :quota]
+        quota_start, quota_end = start_idx.gather(-1, ranked), end_idx.gather(-1, ranked)
+        quota_valid = per_query_valid.gather(-1, ranked).reshape(batch, -1)
+        quota_keys = (quota_start * n_boundaries + quota_end).reshape(batch, -1)
+        rank_bonus = torch.arange(quota, 0, -1, device=start_projection.device, dtype=union_pair_score.dtype)
+        quota_scores = (union_pair_score.new_full((batch, n_queries, quota), -MASK_LOGIT * 0.5) + rank_bonus.view(1, 1, quota)).reshape(batch, -1)
+    all_keys = torch.cat((quota_keys, pair_start * n_boundaries + pair_end), -1)
+    all_scores = torch.cat((quota_scores, union_pair_score.detach()), -1)
+    all_valid = torch.cat((quota_valid, pair_valid), -1)
+    selected_keys, selected_valid = _deduplicate_pool(all_keys, all_scores, all_valid, pool_size, n_boundaries)
+    selected_keys = torch.where(selected_valid, selected_keys, torch.zeros_like(selected_keys))
+    selected_s = torch.div(selected_keys, n_boundaries, rounding_mode="floor")
+    selected_e = selected_keys - selected_s * n_boundaries
+    indices = torch.where(selected_valid.unsqueeze(-1), torch.stack((selected_s, selected_e), -1), torch.zeros_like(torch.stack((selected_s, selected_e), -1)))
+    selected_compat = (gather_rows(start_projection, selected_s) * gather_rows(end_projection, selected_e)).sum(-1) / math.sqrt(dim)
+    selected_score = (selected_compat + union_start.gather(1, selected_s.clamp(0, n_boundaries - 1))
+                      + union_end.gather(1, selected_e.clamp(0, n_boundaries - 1))).masked_fill(~selected_valid, MASK_LOGIT)
+    selected_compat = torch.where(selected_valid, selected_compat, torch.zeros_like(selected_compat))
+    return PooledCandidates(indices=indices, mask=selected_valid, proposal_logits=selected_score, gold_mask=None,
+                            compat_logits=selected_compat, stats=None)
+
+
+class LiteRTBoundaryHead(torch.nn.Module):
+    """Gives gliner2's decoder the tensors the boundary head would have produced, from the LiteRT stages."""
+
+    def __init__(self, context: ContextVar, explicit: Graph):
+        super().__init__()
+        self.context, self.explicit = context, explicit
+        self.max_queries, self.max_spans = explicit.input_shapes["span_indices"][1], explicit.input_shapes["span_indices"][2]
+
+    def forward(self, text, text_mask, query, query_mask, return_candidates=True):
+        state = self.context.get()
+        features, full, count = state["features"], state["candidates"], query.shape[1]
+        selected = CandidateTensorBatch(indices=full.indices[:, :count], proposal_logits=full.proposal_logits[:, :count],
+                                        pair_logits=full.pair_logits[:, :count], valid_mask=full.valid_mask[:, :count],
+                                        query_mask=full.query_mask[:, :count], candidate_states=full.candidate_states[:, :count])
+        return ExtractorOutput(candidates=selected if return_candidates else None, start_logits=features["start_logits"][:, :count],
+                               end_logits=features["end_logits"][:, :count], null_logits=features["null_logits"][:, :count],
+                               count_log_rates=features["count_log_rates"][:, :count], batch_size=1)
+
+    def score_explicit_spans(self, text, text_mask, query, query_mask, indices, valid_mask=None):
+        state = self.context.get()
+        features = state["features"]
+        full_queries = features["query_states"][0]
+        active = int(state["arrays"]["query_mask"].sum())
+        query_count, span_count = query.shape[1], indices.shape[2]
+        if query_count > self.max_queries or span_count > self.max_spans:
+            raise ValueError("Explicit-span request exceeds the bucket capacity")
+        selected = []
+        for row in query[0]:
+            matches = torch.isclose(full_queries[:active], row, atol=1e-6, rtol=0).all(-1).nonzero(as_tuple=False).flatten()
+            if matches.numel() != 1:
+                raise ValueError("Explicit-span query cannot be mapped to the encoded schema")
+            selected.append(int(matches[0]))
+        selection = torch.tensor(selected, dtype=torch.long)
+
+        def feature(name):
+            return padded(features[name][0].index_select(0, selection), self.max_queries).unsqueeze(0)
+
+        span_indices = torch.zeros(1, self.max_queries, self.max_spans, 2, dtype=torch.int32)
+        span_mask = torch.zeros(1, self.max_queries, self.max_spans, dtype=torch.float32)
+        span_indices[:, :query_count, :span_count] = indices.int()
+        span_mask[:, :query_count, :span_count] = valid_mask.float() if valid_mask is not None else 1.0
+        feed = {"text_states": text, "text_mask": text_mask.float(), "query_states": padded(query[0], self.max_queries).unsqueeze(0),
+                "query_mask": padded(query_mask[0].float(), self.max_queries).unsqueeze(0), "boundary_states": features["boundary_states"],
+                "start_logits": feature("start_logits"), "end_logits": feature("end_logits"), "inside_prefix": feature("inside_prefix"),
+                "inside_prefix_mean": feature("inside_prefix_mean"), "span_indices": span_indices, "span_mask": span_mask}
+        output = self.explicit({k: v.numpy() for k, v in feed.items()})["span_logits"]
+        return to_tensor(output)[:, :query_count, :span_count]
+
+
+class LiteRTRelationScorer(torch.nn.Module):
+    """Runs the relation graph after gliner2's own typed pair selection."""
+
+    def __init__(self, context: ContextVar, graph: Graph):
+        super().__init__()
+        self.context, self.graph = context, graph
+        self.max_words = graph.input_shapes["text_states"][1]
+        self.max_relations = graph.input_shapes["relation_states"][1]
+        self.pair_cap = graph.input_shapes["pair_mask"][0]
+
+    def forward(self, text, relation, candidates, pairs):
+        count = len(pairs)
+        if text.shape[1] != self.max_words or relation.shape[1] > self.max_relations or count > self.pair_cap:
+            raise ValueError("Relation request exceeds the bucket capacity")
+        relation_states = torch.zeros(1, self.max_relations, relation.shape[-1], dtype=relation.dtype)
+        relation_states[:, : relation.shape[1]] = relation
+        text_length = int(self.context.get()["arrays"]["text_mask"].sum())
+        feed = {"text_states": text, "text_length": torch.tensor([text_length], dtype=torch.int32), "relation_states": relation_states,
+                "batch_index": padded(pairs.batch_index.int(), self.pair_cap), "relation_index": padded(pairs.relation_index.int(), self.pair_cap),
+                "head_start": padded(pairs.head_start.int(), self.pair_cap), "head_end": padded(pairs.head_end.int(), self.pair_cap),
+                "tail_start": padded(pairs.tail_start.int(), self.pair_cap), "tail_end": padded(pairs.tail_end.int(), self.pair_cap),
+                "pair_mask": padded(pairs.pair_mask.float(), self.pair_cap)}
+        return to_tensor(self.graph({k: v.numpy() for k, v in feed.items()})["relation_logits"])[:count]
+
+
+class LiteRTRecordHead(torch.nn.Module):
+    """Keeps gliner2's instance ordering while the record decoder's learned layers run on LiteRT."""
+
+    def __init__(self, assignment: Graph, anchorless: Graph):
+        super().__init__()
+        self.assignment, self.anchorless = assignment, anchorless
+        self.max_instances = assignment.input_shapes["instance_states"][0]
+        self.max_fields, self.max_candidates = assignment.input_shapes["field_candidate_states"][:2]
+
+    def forward_group(self, spec, query_states, candidates, sample_index):
+        field_specs = list(spec.fields)
+        field_query_ids = [field.query_id for field in field_specs]
+        if len(field_specs) > self.max_fields:
+            raise ValueError("Record has more fields than the bucket holds")
+        if any(q < 0 or q >= query_states.shape[0] for q in field_query_ids):
+            raise ValueError("Record field query is outside the encoded schema")
+        field_states, field_spans, field_logits, field_masks = [], [], [], []
+        for query_id in field_query_ids:
+            mask = candidates.valid_mask[sample_index, query_id]
+            if int(mask.sum()) > self.max_candidates:
+                raise ValueError("Record candidate count exceeds the bucket")
+            field_states.append(candidates.candidate_states[sample_index, query_id][mask])
+            field_spans.append(candidates.indices[sample_index, query_id][mask])
+            field_logits.append(candidates.pair_logits[sample_index, query_id][mask])
+            field_masks.append(torch.ones(int(mask.sum()), dtype=torch.bool))
+        instance_seed, instance_spans = [], []
+        if spec.mode == "natural":
+            anchor = field_query_ids.index(spec.anchor_query_id)
+            instances = field_states[anchor]
+            for index, span in enumerate(field_spans[anchor]):
+                instance_seed.append((anchor, index))
+                instance_spans.append((int(span[0]), int(span[1])))
+        elif spec.mode == "latent":
+            instances = torch.cat(field_states, 0) if field_states else query_states.new_zeros((0, query_states.shape[-1]))
+            for field_index, spans in enumerate(field_spans):
+                for index, span in enumerate(spans):
+                    instance_seed.append((field_index, index))
+                    instance_spans.append((int(span[0]), int(span[1])))
+        else:
+            context = torch.cat(field_states, 0) if field_states else query_states.new_zeros((0, query_states.shape[-1]))
+            capacity = self.anchorless.input_shapes["context_mask"][0]
+            context_mask = torch.zeros(capacity, dtype=torch.float32)
+            context_mask[: context.shape[0]] = 1.0
+            out = self.anchorless({"context_states": padded(context, capacity).numpy(), "context_mask": context_mask.numpy()})
+            instances = to_tensor(out["instance_states"])
+            instance_seed = [None] * instances.shape[0]
+            instance_spans = [None] * instances.shape[0]
+        count = instances.shape[0]
+        if count > self.max_instances:
+            raise ValueError("Record instance count exceeds the bucket")
+        candidate_states = torch.zeros(self.max_fields, self.max_candidates, query_states.shape[-1])
+        for index, states in enumerate(field_states):
+            candidate_states[index, : states.shape[0]] = states
+        predicted = self.assignment({"instance_states": padded(instances, self.max_instances).numpy(),
+                                     "field_queries": padded(query_states[field_query_ids], self.max_fields).numpy(),
+                                     "field_candidate_states": candidate_states.numpy()})
+        assignment = to_tensor(predicted["assignment_logits"])
+        assignment_by_field = [assignment[:count, i, : 1 + states.shape[0]] for i, states in enumerate(field_states)]
+        if spec.mode == "natural":
+            object_logits = field_logits[field_query_ids.index(spec.anchor_query_id)]
+        elif spec.mode == "latent":
+            object_logits = to_tensor(predicted["latent_seed_logits"])[:count]
+        else:
+            object_logits = to_tensor(predicted["object_logits"])[:count]
+        return RecordGroupOutput(spec=spec, object_logits=object_logits, assign_logits=assignment_by_field, field_query_ids=field_query_ids,
+                                 field_specs=field_specs, field_spans=field_spans, field_cand_mask=field_masks, field_cand_logits=field_logits,
+                                 instance_seed=instance_seed, instance_spans=instance_spans)
+
+
+class LiteRTBoundaryExtractor(BoundaryExtractor):
+    """gliner2's BoundaryExtractor (schemas, candidate selection, decoders) with every trained module on LiteRT."""
+
+    def __init__(self, bundle: str, encoder: str = "gliner2_base_encoder_wfp16.tflite", accelerator: str = "cpu", threads: int = 4,
+                 cache: str = ""):
+        torch.nn.Module.__init__(self)
+        folder = Path(bundle)
+        config = json.loads((folder / "source" / "config.json").read_text())
+        self.boundary_settings = BoundaryHeadSettings(**config["boundary_head"])
+        self.processor = SchemaTransformer(tokenizer=load_extractor_tokenizer(str(folder / "tokenizer")), token_pooling="first")
+        self.enable_records = self.boundary_settings.enable_records
+        self.enable_relations = self.boundary_settings.enable_relations
+        self.strict_extraction = True
+        self._device_anchor = torch.nn.Parameter(torch.zeros(1), requires_grad=False)  # the runtime reads device and dtype from parameters()
+        self._context = ContextVar("gliner2_litert_extraction_context")
+        self.encoder_graph = Graph(folder / encoder, signature_of(folder / encoder), threads, accelerator, cache)
+        self.head_graph = Graph(folder / FILES["head"], signature_of(folder / FILES["head"]), 1)
+        self.scorer_graph = Graph(folder / FILES["scorer"], signature_of(folder / FILES["scorer"]), 1)
+        graphs = {k: Graph(folder / FILES[k], signature_of(folder / FILES[k]), 1)
+                  for k in ("explicit", "relation", "record_assignment", "record_anchorless")}
+        self.length = self.encoder_graph.input_shapes["attention_mask"][1]
+        self.max_words = self.head_graph.input_shapes["text_indices"][1]
+        self.max_queries = self.head_graph.input_shapes["query_indices"][1]
+        self.max_choices = self.head_graph.input_shapes["cls_indices"][1]
+        hidden = self.encoder_graph.input_shapes["inputs_embeds"][2]
+        rows = (folder / TABLE).stat().st_size // (2 * hidden)
+        self.table = np.memmap(folder / TABLE, dtype="<f2", mode="r", shape=(rows, hidden))
+        self.boundary_head = LiteRTBoundaryHead(self._context, graphs["explicit"])
+        self.relation_scorer = LiteRTRelationScorer(self._context, graphs["relation"])
+        self.record_decoder = LiteRTRecordHead(graphs["record_assignment"], graphs["record_anchorless"])
+        self.relation_pair_generator = TypedRelationPairGenerator(RelationProposalSettings(
+            heads_per_relation=self.boundary_settings.relation_heads_per_type, tails_per_relation=self.boundary_settings.relation_tails_per_type,
+            pair_cap=self.boundary_settings.relation_pair_cap, argument_threshold=self.boundary_settings.relation_argument_proposal_threshold))
+
+    def features(self, arrays: dict) -> dict:
+        ids = arrays["input_ids"].reshape(-1).astype(np.int64)
+        hidden = self.encoder_graph({"inputs_embeds": self.table[ids].astype(np.float32)[None],
+                                     "attention_mask": arrays["attention_mask"].astype(np.float32).reshape(1, -1)})["hidden_states"]
+        feed = {k: arrays[k] for k in HEAD_INPUTS if k != "hidden_states"}
+        feed["hidden_states"] = hidden
+        return {name: to_tensor(value) for name, value in self.head_graph(feed).items()}
+
+    def _encode_core(self, batch):
+        state = self._context.get()
+        features = state["features"]
+        query_mask = to_tensor(state["arrays"]["query_mask"]).bool()
+        query_count = int(query_mask.sum())
+        query_states = features["query_states"][:, :query_count]
+        query_mask = query_mask[:, :query_count]
+        text_mask = to_tensor(state["arrays"]["text_mask"]).bool()
+        ext_specs, cls_specs, rel_specs, word_offsets = [], [], [], []
+        for sample in range(len(batch)):
+            specs = [{"group_index": q.task_index, "field_index": q.role_index, "task_type": q.task_type, "task_name": q.task_name,
+                      "field_name": q.role_name} for q in batch.query_layouts[sample].queries]
+            ext_specs.append(specs)
+            classifications, choice_offset = [], 0
+            for group in range(batch.schema_counts[sample]):
+                if batch.task_types[sample][group] != "classifications":
+                    continue
+                count = max(len(batch.schema_special_indices[sample][group]) - 1, 0)
+                tokens = batch.schema_tokens_list[sample][group]
+                if count:
+                    classifications.append({"group_index": group, "task_name": tokens[2], "schema_tokens": tokens,
+                                            "group_embs": features["classification_logits"][sample, choice_offset: choice_offset + count]})
+                choice_offset += count
+            cls_specs.append(classifications)
+            word_offsets.append(max(int(batch.text_word_counts[sample]) - len(batch.start_mappings[sample]), 0))
+            groups = {}
+            for query_id, spec in enumerate(specs):
+                groups.setdefault(spec["group_index"], []).append(query_id)
+            relations = []
+            for group, role_ids in groups.items():
+                if batch.task_types[sample][group] != "relations" or len(role_ids) < 2:
+                    continue
+                head_id, tail_id = role_ids[:2]
+                roles = query_states[sample, [head_id, tail_id]]
+                state_vec = torch.cat((roles[0], roles[1]), -1) if self.boundary_settings.directional_relation_states else roles.mean(0)
+                relations.append({"group_index": group, "relation_type": specs[head_id]["task_name"],
+                                  "spec": RelationTypeSpec(specs[head_id]["task_name"], head_query_ids=(head_id,), tail_query_ids=(tail_id,)),
+                                  "query_state": state_vec})
+            rel_specs.append(relations)
+        return {"text_states": features["text_states"], "text_mask": text_mask, "text_lengths": text_mask.sum(-1).long(),
+                "query_states": query_states, "query_mask": query_mask, "ext_specs": ext_specs, "cls_specs": cls_specs,
+                "rel_specs": rel_specs, "word_offsets": word_offsets}
+
+    def _extract_classification_result(self, results, schema_name, schema, embs, schema_tokens, temperature=1.0):
+        cls_config = self._resolve_classification_config(schema_tokens[2], schema.get("classifications", []))
+        if cls_config is None:
+            return
+        if temperature <= 0:
+            raise ValueError("Classification temperature must be positive")
+        logits = embs / temperature
+        activation, multi_label = cls_config.get("class_act", "auto"), cls_config.get("multi_label", False)
+        probabilities = torch.sigmoid(logits) if activation == "sigmoid" or (activation != "softmax" and multi_label) else torch.softmax(logits, dim=-1)
+        labels = cls_config["labels"]
+        if multi_label:
+            threshold = cls_config.get("cls_threshold", 0.5)
+            chosen = [(labels[i], float(probabilities[i])) for i in range(len(labels)) if float(probabilities[i]) >= threshold]
+            if not chosen:
+                best = int(probabilities.argmax())
+                chosen = [(labels[best], float(probabilities[best]))]
+            results[cls_config["task"]] = chosen
+            return
+        best = int(probabilities.argmax())
+        results[cls_config["task"]] = (labels[best], float(probabilities[best]))
+
+    def batch_extract(self, texts, schemas, batch_size: int = 8, threshold: float = 0.5, num_workers: int = 0, format_results: bool = True,
+                      include_confidence: bool = False, include_spans: bool = False, max_len=None, overlap_policy=None):
+        """One request at a time through the fixed-shape graphs (gliner2's batch API, so the *_long calls work too)."""
+        if isinstance(schemas, list):
+            if len(schemas) != len(texts):
+                raise ValueError(f"Schema count ({len(schemas)}) != text count ({len(texts)})")
+            schema_list = schemas
+        else:
+            schema_list = [schemas] * len(texts)
+        return [self.extract(text, schema, threshold=threshold, format_results=format_results, include_confidence=include_confidence,
+                             include_spans=include_spans, overlap_policy=overlap_policy) for text, schema in zip(texts, schema_list)]
+
+    def extract(self, text: str, schema, threshold: float = 0.5, format_results: bool = True, include_confidence: bool = False,
+                include_spans: bool = False, max_len=None, overlap_policy=None):
+        schema_dicts, metadata_list = self._build_schema_dicts_and_metadata([schema])
+        if overlap_policy is not None:
+            metadata_list[0]["_overlap_policy"] = self._resolved_overlap_policy(overlap_policy)
+        arrays, batch = prepare_extraction(self.processor, text, schema_dicts[0], self.length, self.max_words, self.max_queries, self.max_choices)
+        features = self.features(arrays)
+        query_mask = to_tensor(arrays["query_mask"]).bool()
+        candidates = None
+        if bool(query_mask.any()):
+            head = self.boundary_settings
+            pool = select_candidates(features["pool_start_projection"], features["pool_end_projection"], features["boundary_mask"].bool(),
+                                     query_mask, features["start_logits"], features["end_logits"], boundary_top_k=head.pool_boundary_top_k,
+                                     pool_size=head.pool_size, min_pool_per_query=head.min_pool_per_query)
+            feed = {"text_states": features["text_states"], "text_mask": to_tensor(arrays["text_mask"]), "query_states": features["query_states"],
+                    "query_mask": to_tensor(arrays["query_mask"]), "boundary_states": features["boundary_states"],
+                    "start_logits": features["start_logits"], "end_logits": features["end_logits"], "inside_prefix": features["inside_prefix"],
+                    "inside_prefix_mean": features["inside_prefix_mean"], "candidate_indices": pool.indices.int(),
+                    "candidate_mask": pool.mask.float(), "candidate_compatibility": pool.compat_logits}
+            scored = self.scorer_graph({k: v.numpy() for k, v in feed.items()})
+            q = self.max_queries
+            candidates = CandidateTensorBatch(indices=pool.indices.unsqueeze(1).expand(1, q, -1, 2),
+                                              proposal_logits=pool.proposal_logits.unsqueeze(1).expand(1, q, -1),
+                                              pair_logits=to_tensor(scored["pair_logits"]), valid_mask=pool.mask.unsqueeze(1).expand(1, q, -1),
+                                              query_mask=query_mask,
+                                              candidate_states=to_tensor(scored["candidate_states"]).unsqueeze(1).expand(1, q, -1, -1))
+        token = self._context.set({"arrays": arrays, "features": features, "candidates": candidates})
+        try:
+            raw = self._extract_from_batch(batch, threshold, metadata_list, include_confidence, include_spans)[0]
+            if format_results:
+                return self.format_results(raw, include_confidence, metadata_list[0].get("relation_order", []),
+                                           metadata_list[0].get("classification_tasks", []))
+            return raw
+        finally:
+            self._context.reset(token)

runtime/parity_corpus.json

@@ -0,0 +1,101 @@
+{
+ "about": "Parity corpus for LiteRT ports of GLiNER2 checkpoints: short general-domain texts with invented people and companies, and schema templates covering every call type gliner2 2.0.0 offers (entities with and without descriptions, relations, plain structures, choice fields, natural/latent/anchorless records, entity attributes, single- and multi-label classification, mixed schemas). Only agreement with the library is measured; there are no gold labels. Each text stays under 40 words so a call fits a 128-token bucket with its schema.",
+ "templates": {
+  "entities": {"entities": ["person", "organization", "location", "date"]},
+  "entities_described": {"entities": {"person": "a named individual", "product": "a named product, device or service", "money": "an amount of money"}},
+  "entities_single": {"entities": ["person", "organization"], "dtype": "str"},
+  "relations_work": {"relations": ["works_for", "founded", "located_in"]},
+  "relations_family": {"relations": {"parent_of": "the head is a parent of the tail", "married_to": "the two people are married"}},
+  "relations_event": {"relations": ["won", "held_in"]},
+  "structure_event": {"structures": [{"name": "event", "fields": [{"name": "name", "dtype": "str"}, {"name": "date", "dtype": "str"}, {"name": "place", "dtype": "str"}]}]},
+  "structure_order": {"structures": [{"name": "order", "fields": [{"name": "item", "dtype": "str"}, {"name": "category", "dtype": "str", "choices": ["electronics", "clothing", "furniture", "food"]}, {"name": "quantity", "dtype": "str"}]}]},
+  "record_natural": {"structures": [{"name": "employment", "mode": "natural", "anchor": "person", "fields": [{"name": "person", "dtype": "str"}, {"name": "company", "dtype": "str"}, {"name": "role", "dtype": "str"}]}]},
+  "record_latent": {"structures": [{"name": "employment", "mode": "latent", "fields": [{"name": "person", "dtype": "str"}, {"name": "company", "dtype": "str"}]}]},
+  "record_anchorless": {"structures": [{"name": "trip", "mode": "anchorless", "fields": [{"name": "traveler", "dtype": "str"}, {"name": "destination", "dtype": "str"}]}]},
+  "attributes": {"entities": ["person", "organization"], "attributes": {"role": {"labels": ["founder", "employee", "customer"]}}},
+  "classification_topic": {"classifications": [{"task": "topic", "labels": ["business", "family", "travel", "shopping", "science", "sports"]}]},
+  "classification_multi": {"classifications": [{"task": "tags", "labels": ["money", "people", "places", "dates", "products"], "multi_label": true}]},
+  "classification_described": {"classifications": [{"task": "tone", "labels": {"positive": "good news or praise", "negative": "bad news or complaints", "neutral": "plain facts"}}]},
+  "mixed": {"entities": ["person", "organization"], "relations": ["works_for"], "classifications": [{"task": "sentiment", "labels": ["positive", "negative", "neutral"]}]}
+ },
+ "groups": {
+  "business": {
+   "templates": ["entities", "entities_single", "relations_work", "record_natural", "record_latent", "attributes", "classification_topic", "mixed"],
+   "texts": [
+    "Nora Lindqvist founded Brightwell Labs in Oslo in 2019 and still runs it as chief executive.",
+    "Tomas Ferreira joined Halcyon Freight in March as head of logistics, moving from a rival in Lisbon.",
+    "Priya Raman works at Cobalt Analytics in Pune, where she leads a team of twelve engineers.",
+    "Marco Bellini and Ines Duarte started Verde Foods together; Marco handles sales while Ines manages the kitchen in Porto.",
+    "Westgate Mills hired Daniel Okafor as an accountant last week, and Sara Klein left the company after ten years.",
+    "Kwame Mensah is the founder of Sunline Solar, a startup based in Accra that installs rooftop panels.",
+    "After the merger, Lena Hoffmann became chief financial officer of Northpeak Group in Munich.",
+    "Ravi Shah, an engineer at Tidewater Systems, presented the new billing platform to customers in Boston on Tuesday."
+   ]
+  },
+  "family": {
+   "templates": ["entities", "relations_family", "record_latent", "classification_topic", "classification_described"],
+   "texts": [
+    "Helen Ward married Peter Ward in 1998, and their daughter Emma was born two years later in Leeds.",
+    "Carlos Mendes is the father of Lucia and Mateo, who both moved to Madrid for university.",
+    "My grandmother Rosa raised four children on a farm near Cork before moving to Dublin.",
+    "Aiko Tanaka and her husband Kenji celebrated their anniversary in Kyoto with their son Haruto.",
+    "Fatima Diallo is the mother of twins, Amina and Yusuf, who turned nine on Sunday.",
+    "James Porter's sister, Claire, lives in Toronto with her partner Mark and their dog.",
+    "Olga Petrova married Ivan Sokolov last spring; Ivan's brother Dmitri was the best man.",
+    "The Nguyen family, Linh, Bao and their son Minh, spent the holidays with relatives in Hanoi."
+   ]
+  },
+  "travel": {
+   "templates": ["entities", "relations_work", "record_anchorless", "structure_event", "classification_topic", "classification_multi"],
+   "texts": [
+    "Last July, Elena flew from Rome to Reykjavik and spent a week hiking near Vik.",
+    "Ben and Chloe drove from Denver to Santa Fe, stopping overnight in Taos on the way.",
+    "Our tour group arrived in Marrakech on Friday and leaves for Fez on Monday morning.",
+    "Hugo booked a train from Paris to Geneva for the conference on 14 May.",
+    "Mia visited her cousin in Cape Town and then took a ferry to Robben Island.",
+    "The museum in Bilbao reopened in April after a two-year renovation led by the city council.",
+    "Sam and Priya plan to backpack through Peru, starting in Cusco and ending in Lima.",
+    "Grace missed her connection in Frankfurt, so she reached Warsaw a day late."
+   ]
+  },
+  "shopping": {
+   "templates": ["entities_described", "structure_order", "record_natural", "classification_multi", "classification_described", "attributes"],
+   "texts": [
+    "I ordered two oak chairs from Hearthwood for 180 dollars, but only one arrived.",
+    "Lucas bought a Zenith X2 laptop and a wireless mouse at Volt Electronics for 1,240 euros.",
+    "Please send three boxes of green tea and a ceramic teapot to the Seattle store.",
+    "The customer returned a wool coat to Northline Apparel and asked for a refund of 95 pounds.",
+    "Amara paid 12 dollars for a loaf of sourdough and a jar of honey at the Saturday market.",
+    "Our office ordered forty desk lamps from Brightspace, and the invoice came to 2,000 dollars.",
+    "Jonah's new Aurora headphones broke after a week, and the shop offered him a replacement.",
+    "Ten kilograms of rice and five bags of lentils were delivered to the community kitchen."
+   ]
+  },
+  "science": {
+   "templates": ["entities", "entities_described", "relations_work", "structure_event", "classification_topic", "classification_described"],
+   "texts": [
+    "Dr. Yusuf Kaya and his team at the Halden Institute published a study on coral bleaching in June.",
+    "The Meridian telescope in Chile recorded a supernova on 3 February, according to astronomer Lucia Ortiz.",
+    "Researchers at Brookfield University tested a new battery that charges in six minutes.",
+    "Marta Silva, a biologist at the Lisbon Aquarium, tagged forty sea turtles last summer.",
+    "The probe Helios-4 entered orbit around Mars after a seven-month journey from Earth.",
+    "Engineers at Polaris Robotics in Seoul unveiled a robot that sorts recycling by material.",
+    "A trial run by Kestrel Health in Nairobi found the vaccine safe for children over five.",
+    "Professor Anil Mehta left Crestview College to lead the climate lab at Northgate University."
+   ]
+  },
+  "sports": {
+   "templates": ["entities", "relations_event", "structure_event", "record_latent", "classification_topic", "mixed"],
+   "texts": [
+    "Sofia Marin won the Vienna Marathon in April, finishing ahead of last year's champion Ruth Abebe.",
+    "The Harbor City Hawks beat the Ridgeview Bears 3 to 1 in the final held in Glasgow.",
+    "Coach Andre Lopes signed a new contract with Atletico Verde until 2028.",
+    "Kenji Watanabe took gold in the 200 metres at the championships in Nagoya on Saturday.",
+    "The tennis open in Melbourne ended with Clara Weiss lifting her first trophy.",
+    "Liam O'Brien scored twice as Dunmore Rovers won the cup at their home ground.",
+    "The cycling race from Lyon to Annecy was won by Pieter de Vries after a late sprint.",
+    "Amira Haddad set a national record in the long jump at the meet in Doha."
+   ]
+  }
+ }
+}

runtime/requirements.txt

@@ -0,0 +1,5 @@
+gliner2==2.0.0
+torch
+numpy
+ai-edge-litert>=2.2.0
+huggingface_hub  # verify_parity.py

runtime/verify_parity.py

@@ -0,0 +1,139 @@
+#!/usr/bin/env python3
+"""Check a LiteRT GLiNER2 port against the original checkpoint on the bundled parity corpus.
+
+    pip install gliner2==2.0.0 torch numpy ai-edge-litert
+    python runtime/verify_parity.py --bundle .            # from the repository folder
+
+Runs every corpus request twice, through gliner2 with the original weights (downloaded from Fastino at the pinned
+revision) and through this repository's LiteRT runtime, and prints per call type how many structured results are
+identical, the lowest span F1 and the largest confidence difference. The corpus (runtime/parity_corpus.json) is short
+synthetic texts with schema templates for every call type gliner2 2.0.0 offers; it has no gold labels.
+"""
+from __future__ import annotations
+
+import argparse
+import collections
+import json
+import pathlib
+import sys
+
+HERE = pathlib.Path(__file__).resolve().parent
+SOURCES = {"boundary": ("fastino/gliner2.5-base-v1", "1a8bc24e00dc7300b9017c81d63e3dcdabb26596"),
+           "span": ("fastino/GLiNER2.5-Decide", "bbe10ff77ebb238777c17d3a8ac9260e30929057")}
+
+
+def build_schema(t: dict):
+    from gliner2 import AttributeGroup, Schema
+    s = Schema()
+    if "entities" in t:
+        s.entities(t["entities"], dtype=t.get("dtype", "list"))
+    if "attributes" in t:
+        s.entity_attributes({k: AttributeGroup(labels=v["labels"], multi_label=v.get("multi_label", False)) for k, v in t["attributes"].items()})
+    if "relations" in t:
+        s.relations(t["relations"])
+    for c in t.get("classifications", []):
+        s.classification(c["task"], c["labels"], multi_label=c.get("multi_label", False))
+    for st in t.get("structures", []):
+        builder = s.structure(st["name"], mode=st.get("mode"), anchor=st.get("anchor"))
+        for f in st["fields"]:
+            builder.field(f["name"], dtype=f.get("dtype", "list"), choices=f.get("choices"))
+    return s
+
+
+def without_confidence(value):
+    if isinstance(value, dict):
+        return {k: without_confidence(v) for k, v in value.items() if k != "confidence"}
+    if isinstance(value, list):
+        return [without_confidence(v) for v in value]
+    return value
+
+
+def confidence_errors(reference, actual):
+    if isinstance(reference, dict) and isinstance(actual, dict):
+        out = []
+        for key, value in reference.items():
+            if key == "confidence" and key in actual:
+                out.append(abs(float(value) - float(actual[key])))
+            elif key in actual:
+                out.extend(confidence_errors(value, actual[key]))
+        return out
+    if isinstance(reference, (list, tuple)) and isinstance(actual, (list, tuple)):
+        return [e for a, b in zip(reference, actual) for e in confidence_errors(a, b)]
+    return []
+
+
+def answers(value, path: str = "") -> list[str]:
+    out = []
+    if isinstance(value, dict):
+        if {"text", "start", "end"} <= set(value):
+            return [f"{path}|{value['text']}|{value['start']}|{value['end']}"]
+        if "head" in value and "tail" in value:
+            return [f"{path}|H:{'/'.join(answers(value['head']))}|T:{'/'.join(answers(value['tail']))}"]
+        if "label" in value:
+            return [f"{path}|{value['label']}"]
+        for key, item in value.items():
+            if key != "confidence":
+                out += answers(item, f"{path}.{key}" if path else key)
+    elif isinstance(value, (list, tuple)):
+        if len(value) == 2 and isinstance(value[0], str) and isinstance(value[1], float):
+            return [f"{path}|{value[0]}"]
+        for item in value:
+            out += answers(item, path)
+    elif value is not None:
+        out.append(f"{path}|{value}")
+    return out
+
+
+def f1(reference: list[str], actual: list[str]) -> float:
+    a, b = collections.Counter(reference), collections.Counter(actual)
+    if not a and not b:
+        return 1.0
+    return 2 * sum((a & b).values()) / (sum(a.values()) + sum(b.values()))
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    parser.add_argument("--bundle", default=str(HERE.parent))
+    parser.add_argument("--corpus", default=str(HERE / "parity_corpus.json"))
+    args = parser.parse_args()
+    sys.path.insert(0, str(HERE))
+    import torch
+    from gliner2 import AutoExtractor
+    from huggingface_hub import snapshot_download
+    config = json.loads((pathlib.Path(args.bundle) / "source" / "config.json").read_text())
+    architecture = config.get("architecture", "span")
+    if architecture == "boundary":
+        from litert_boundary_runtime import LiteRTBoundaryExtractor as Runtime
+    else:
+        from litert_span_runtime import LiteRTSpanExtractor as Runtime
+    repo, revision = SOURCES[architecture]
+    native = AutoExtractor.from_pretrained(snapshot_download(repo, revision=revision), map_location="cpu").eval()
+    torch.set_num_threads(4)
+    runtime = Runtime(args.bundle)
+    corpus = json.loads(pathlib.Path(args.corpus).read_text())
+    by_kind = {}
+    for group in corpus["groups"].values():
+        for text in group["texts"]:
+            for kind in group["templates"]:
+                schema = build_schema(corpus["templates"][kind])
+                expected = native.extract(text, schema, include_confidence=True, include_spans=True)
+                k = by_kind.setdefault(kind, {"calls": 0, "identical": 0, "refused": 0, "minSpanF1": 1.0, "maxConfidenceDifference": 0.0})
+                k["calls"] += 1
+                try:
+                    actual = runtime.extract(text, schema, include_confidence=True, include_spans=True)
+                except ValueError:  # larger than the graphs' fixed sizes: refused, never truncated
+                    k["refused"] += 1
+                    continue
+                k["identical"] += without_confidence(expected) == without_confidence(actual)
+                k["minSpanF1"] = round(min(k["minSpanF1"], f1(answers(expected), answers(actual))), 4)
+                k["maxConfidenceDifference"] = round(max([k["maxConfidenceDifference"], *confidence_errors(expected, actual)]), 6)
+    for kind, k in by_kind.items():
+        print(f"{kind:26s} {json.dumps(k)}")
+    total = sum(k["calls"] - k["refused"] for k in by_kind.values())
+    same = sum(k["identical"] for k in by_kind.values())
+    print(f"identical structured results: {same} / {total}")
+    return 0 if same == total else 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

validation-full.json

@@ -0,0 +1,201 @@
+{
+ "about": "Every call type against gliner2 2.0.0 with the original weights (runtime/verify_parity.py): LiteRT 2.2.0 on the CPU, wfp16 encoder.",
+ "source": "fastino/gliner2.5-base-v1@1a8bc24e00dc7300b9017c81d63e3dcdabb26596",
+ "corpus": {
+  "file": "runtime/parity_corpus.json",
+  "requests": 296,
+  "identical": 296,
+  "byCallType": {
+   "entities": {
+    "calls": 40,
+    "refused": 0,
+    "matched": 40,
+    "items": 148,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.000228
+   },
+   "entities_single": {
+    "calls": 8,
+    "refused": 0,
+    "matched": 8,
+    "items": 16,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 5e-06
+   },
+   "relations_work": {
+    "calls": 24,
+    "refused": 0,
+    "matched": 24,
+    "items": 62,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.000883
+   },
+   "record_natural": {
+    "calls": 16,
+    "refused": 0,
+    "matched": 16,
+    "items": 28,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.00013
+   },
+   "record_latent": {
+    "calls": 24,
+    "refused": 0,
+    "matched": 24,
+    "items": 429,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.000641
+   },
+   "attributes": {
+    "calls": 16,
+    "refused": 0,
+    "matched": 16,
+    "items": 31,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.000298
+   },
+   "classification_topic": {
+    "calls": 40,
+    "refused": 0,
+    "matched": 40,
+    "items": 40,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.001305
+   },
+   "mixed": {
+    "calls": 16,
+    "refused": 0,
+    "matched": 16,
+    "items": 68,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.001358
+   },
+   "relations_family": {
+    "calls": 8,
+    "refused": 0,
+    "matched": 8,
+    "items": 15,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.00033
+   },
+   "classification_described": {
+    "calls": 24,
+    "refused": 0,
+    "matched": 24,
+    "items": 24,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.001187
+   },
+   "record_anchorless": {
+    "calls": 8,
+    "refused": 0,
+    "matched": 8,
+    "items": 2,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.000258
+   },
+   "structure_event": {
+    "calls": 24,
+    "refused": 0,
+    "matched": 24,
+    "items": 45,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.000897
+   },
+   "classification_multi": {
+    "calls": 16,
+    "refused": 0,
+    "matched": 16,
+    "items": 22,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.000356
+   },
+   "entities_described": {
+    "calls": 16,
+    "refused": 0,
+    "matched": 16,
+    "items": 40,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.000411
+   },
+   "structure_order": {
+    "calls": 8,
+    "refused": 0,
+    "matched": 8,
+    "items": 22,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.001252
+   },
+   "relations_event": {
+    "calls": 8,
+    "refused": 0,
+    "matched": 8,
+    "items": 15,
+    "minSpanF1": 1.0,
+    "maxConfidenceDifference": 0.0002
+   }
+  }
+ },
+ "fixtures": {
+  "about": "FluidInference's eleven verify-full-extraction fixtures",
+  "identical": 11,
+  "total": 11,
+  "rows": [
+   {
+    "case": "entities",
+    "structureMatch": true,
+    "maxConfidenceDifference": 1e-06
+   },
+   {
+    "case": "entities_multi",
+    "structureMatch": true,
+    "maxConfidenceDifference": 3e-06
+   },
+   {
+    "case": "relations",
+    "structureMatch": true,
+    "maxConfidenceDifference": 2.2e-05
+   },
+   {
+    "case": "relations_two",
+    "structureMatch": true,
+    "maxConfidenceDifference": 6e-06
+   },
+   {
+    "case": "record_natural",
+    "structureMatch": true,
+    "maxConfidenceDifference": 0.000134
+   },
+   {
+    "case": "record_latent",
+    "structureMatch": true,
+    "maxConfidenceDifference": 0.000352
+   },
+   {
+    "case": "record_anchorless",
+    "structureMatch": true,
+    "maxConfidenceDifference": null
+   },
+   {
+    "case": "attributes",
+    "structureMatch": true,
+    "maxConfidenceDifference": 2.8e-05
+   },
+   {
+    "case": "mixed_classification",
+    "structureMatch": true,
+    "maxConfidenceDifference": 1e-06
+   },
+   {
+    "case": "classification_only",
+    "structureMatch": true,
+    "maxConfidenceDifference": 6.4e-05
+   },
+   {
+    "case": "enum_choice",
+    "structureMatch": true,
+    "maxConfidenceDifference": 0.0
+   }
+  ]
+ }
+}