alexwengg commited on
Commit
fcf4209
·
verified ·
1 Parent(s): 626b551

Add validated GLiNER2 extraction Core ML stages

Browse files

FP32 full extraction stages with selected native parity; base also includes FP16 with documented confidence drift. Classification artifacts remain available.

Files changed (42) hide show
  1. README.md +30 -30
  2. benchmark-extraction.py +76 -0
  3. config.json +99 -0
  4. convert-explicit-coreml.py +170 -0
  5. convert-extraction-coreml.py +217 -0
  6. convert-record-coreml.py +222 -0
  7. convert-relation-coreml.py +165 -0
  8. convert_extraction_names.py +31 -0
  9. extraction-assets.lock.json +149 -0
  10. extraction_export.py +408 -0
  11. extraction_pool.py +117 -0
  12. extraction_runtime.py +525 -0
  13. gliner2_multi_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/model.mlmodel +3 -0
  14. gliner2_multi_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/weights/weight.bin +3 -0
  15. gliner2_multi_explicit_fp32_W64_Q8_S64.mlpackage/Manifest.json +18 -0
  16. gliner2_multi_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel +3 -0
  17. gliner2_multi_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin +3 -0
  18. gliner2_multi_extraction_features_fp32_L128_W64_Q8.mlpackage/Manifest.json +18 -0
  19. gliner2_multi_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel +3 -0
  20. gliner2_multi_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin +3 -0
  21. gliner2_multi_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Manifest.json +18 -0
  22. gliner2_multi_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel +3 -0
  23. gliner2_multi_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin +3 -0
  24. gliner2_multi_record_anchorless_fp32_F8_C192_I1536.mlpackage/Manifest.json +18 -0
  25. gliner2_multi_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel +3 -0
  26. gliner2_multi_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin +3 -0
  27. gliner2_multi_record_assignment_fp32_F8_C192_I1536.mlpackage/Manifest.json +18 -0
  28. gliner2_multi_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/model.mlmodel +3 -0
  29. gliner2_multi_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/weights/weight.bin +3 -0
  30. gliner2_multi_relation_fp32_W64_R4_P256.mlpackage/Manifest.json +18 -0
  31. preprocessing.py +46 -0
  32. pyproject.toml +3 -1
  33. reports/extraction-validation.json +82 -0
  34. reports/extraction-verify-fp16.json +849 -0
  35. reports/extraction-verify-fp32.json +858 -0
  36. tests/test_extraction_export.py +67 -0
  37. tests/test_extraction_pool.py +62 -0
  38. tests/test_extraction_preprocessing.py +43 -0
  39. tests/test_extraction_record.py +66 -0
  40. uv.lock +76 -0
  41. verify-extraction.py +170 -0
  42. verify-full-extraction.py +144 -0
README.md CHANGED
@@ -1,7 +1,7 @@
1
  ---
2
  license: apache-2.0
3
  library_name: coremltools
4
- pipeline_tag: text-classification
5
  tags:
6
  - coreml
7
  - gliner2
@@ -9,37 +9,37 @@ tags:
9
  - apple-silicon
10
  ---
11
 
12
- # GLiNER2.5 multilingual Core ML classification
13
-
14
- Core ML conversion of the classification path of
15
- [Fastino GLiNER2.5 multilingual](https://huggingface.co/fastino/gliner2.5-multi-v1),
16
- revision `a221b77a8baf4a613b8f8652661d41fa10a5641e` (Apache-2.0).
17
- The original checkpoint has 287,355,159 parameters. This export includes the
18
- encoder and trained classification head (278,719,489 parameters); it does not
19
- include entity, relation, record or span extraction. Use the original checkpoint
20
- for those tasks.
21
-
22
- The L128/K8 FP16 package is 576,528,829 bytes. It matched native choices on
23
- 299 of 300 eligible fixed application requests, with one near tie reversing
24
- "sports" and "world news". Maximum chosen-confidence difference was 0.009249;
25
- median Python Core ML call was 8.81 ms on an M5 Pro running macOS 27.0.
26
- The FP32 reference package is 1,152,027,886 bytes; it matched all 100 sampled
27
- native choices, with maximum confidence difference 0.00000805 and median call
28
- 11.24 ms. Four French, Spanish, Chinese and German smoke phrases gave the same
29
- chosen labels in native, FP16 and FP32. See the report JSON files. These are
30
- parity checks, not a complete Decision Index run or ANE-only timings.
31
-
32
- The model uses the upstream GLiNER2 schema and tokenizer supplied here.
33
- The Core ML minimum target is iOS 17/macOS 14. At most eight labels fit this
34
- bucket; inputs over capacity require a larger model bucket rather than
35
- truncation. The included runtime loads only Core ML and tokenizer files:
36
 
37
  ```bash
38
  uv sync
39
- uv run python runtime.py --model-dir . --text '科学家发现了一颗新的行星。' \
40
- --task topic --labels '["science","sports","politics"]' --precision fp16
 
 
 
 
 
 
41
  ```
42
 
43
- Use `--precision fp32` when exact native decisions around narrow margins matter.
44
- The conversion scripts, pinned dependencies and asset hashes are included.
45
- Fluid Inference converted the model; Fastino authored the original checkpoint.
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  library_name: coremltools
4
+ pipeline_tag: token-classification
5
  tags:
6
  - coreml
7
  - gliner2
 
9
  - apple-silicon
10
  ---
11
 
12
+ # GLiNER2.5 multilingual for Core ML
13
+
14
+ This repository contains fixed-shape Core ML exports of the trained classification and extraction paths from [Fastino/gliner2.5-multi-v1](https://huggingface.co/fastino/gliner2.5-multi-v1) at revision `a221b77a8baf4a613b8f8652661d41fa10a5641e`. The original Apache-2.0 checkpoint has 287,355,159 parameters. Fastino authored the source model; Fluid Inference converted it. The model packages include the learned encoder, classification, boundary, relation, explicit-span, and record heads. The Python runtime keeps the source GLiNER2 schema, candidate selection, and decoder semantics.
15
+
16
+ ## Extraction
17
+
18
+ The FP32 extraction stage packages support entities, relations, entity attributes, enum choices, natural/latent/anchorless records, and schemas mixed with classification. In a small fixed manifest of real text and schema fixtures, FP32 matched the native structured output on **15/15** cases. The largest FP32 confidence difference was 0.00000489. The multilingual manifest includes Spanish, French, Chinese, and German text. These checks are selected parity fixtures, not a Decision Index score or a full dataset evaluation.
19
+
20
+ FP16 result: 14/15 structures matched; the latent-record fixture differed (10 native records versus 9 Core ML records), so FP16 extraction is not published. The multilingual extraction FP16 packages remain local experiments because one latent-record fixture differed. They are not in this repository.
21
+
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
 
23
  ```bash
24
  uv sync
25
+ uv run python - <<'PYCODE'
26
+ from gliner2 import Schema
27
+ from extraction_runtime import CoreMLBoundaryExtractor
28
+
29
+ model = CoreMLBoundaryExtractor('.', precision='fp32')
30
+ schema = Schema().entities(['person', 'organization', 'location'])
31
+ print(model.extract('Alice founded Acme in Toronto.', schema, include_spans=True))
32
+ PYCODE
33
  ```
34
 
35
+ The extraction bucket holds up to 128 combined subword tokens, 64 text words, 8 extraction queries, and 8 classification choices. Candidate, explicit span, relation pair, and record capacities are fixed in the package names. Requests beyond the bucket raise `ValueError`; they require a larger exported bucket. The runtime uses the original tokenizer files and `gliner2==2.0.0`, but loads no PyTorch model weights. Run `verify-full-extraction.py` with the pinned native checkpoint for the selected parity check.
36
+
37
+ On an M5 Pro with macOS 27.0, FP32 end-to-end median 10.56 ms with All, 10.02 ms with CPU+GPU, 36.08 ms with CPU+Neural Engine for a selected three-label entity request after 20 warmups and over 200 Python calls. Those are local end-to-end measurements for this shape, not ANE-only latency or a device-wide benchmark.
38
+
39
+ Under CPU+Neural Engine, the FP32 feature graph compute plan assigned all operations to CPU on this machine; CPU+GPU is the faster measured setting. A local FP32-source LUT8 attempt stalled in k-means with numerical warnings; no compressed extraction package is included.
40
+
41
+ ## Classification
42
+
43
+ The original L128/K8 classification packages remain available, with a separate `runtime.py` entry point. Up to eight labels fit that bucket. 299/300 selected choices matched in FP16; 100/100 matched in an FP32 control. See the classification report JSONs for the exact selected samples and limits. The full checkpoint's task and dataset scores have not been reproduced here.
44
+
45
+ The Core ML deployment target is iOS 17/macOS 14. Conversion scripts, pinned dependencies, asset hashes, and selected verification reports are included. The source revision is a current pinned snapshot; identity with the historical Decision Index evaluation checkpoint has not been established.
benchmark-extraction.py ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Selected fixed-bucket end-to-end Core ML extraction latency on this Mac."""
2
+
3
+ import argparse
4
+ import json
5
+ import platform
6
+ import statistics
7
+ import time
8
+ from pathlib import Path
9
+
10
+ import coremltools as ct
11
+ import psutil
12
+ from gliner2 import Schema
13
+
14
+ from extraction_runtime import CoreMLBoundaryExtractor
15
+
16
+ UNITS = {
17
+ "cpu_only": ct.ComputeUnit.CPU_ONLY,
18
+ "cpu_and_gpu": ct.ComputeUnit.CPU_AND_GPU,
19
+ "cpu_and_neural_engine": ct.ComputeUnit.CPU_AND_NE,
20
+ "all": ct.ComputeUnit.ALL,
21
+ }
22
+
23
+
24
+ def percentile(values, fraction):
25
+ ordered = sorted(values)
26
+ return ordered[min(round(fraction * (len(ordered) - 1)), len(ordered) - 1)]
27
+
28
+
29
+ def main():
30
+ parser = argparse.ArgumentParser()
31
+ parser.add_argument("--model-dir", required=True)
32
+ parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32")
33
+ parser.add_argument("--units", choices=list(UNITS), default="all")
34
+ parser.add_argument("--warmup", type=int, default=20)
35
+ parser.add_argument("--iterations", type=int, default=200)
36
+ args = parser.parse_args()
37
+ text = "Alice founded Acme in Toronto in 2020."
38
+ schema = Schema().entities(["person", "organization", "location"])
39
+ started = time.perf_counter()
40
+ runtime = CoreMLBoundaryExtractor(args.model_dir, precision=args.precision, compute_units=UNITS[args.units])
41
+ load_ms = (time.perf_counter() - started) * 1000
42
+ for _ in range(args.warmup):
43
+ runtime.extract(text, schema)
44
+ process = psutil.Process()
45
+ latencies = []
46
+ peak_rss = process.memory_info().rss
47
+ for _ in range(args.iterations):
48
+ start = time.perf_counter()
49
+ runtime.extract(text, schema)
50
+ latencies.append((time.perf_counter() - start) * 1000)
51
+ peak_rss = max(peak_rss, process.memory_info().rss)
52
+ report = {
53
+ "purpose": "selected end-to-end entity extraction latency, no benchmark scoring",
54
+ "fixture": text,
55
+ "shape": "L128/W64/Q8/C192",
56
+ "precision": args.precision,
57
+ "compute_units": args.units,
58
+ "warmup": args.warmup,
59
+ "iterations": args.iterations,
60
+ "load_ms": load_ms,
61
+ "p50_ms": statistics.median(latencies),
62
+ "p95_ms": percentile(latencies, 0.95),
63
+ "mean_ms": statistics.mean(latencies),
64
+ "peak_process_rss_bytes": peak_rss,
65
+ "macos": platform.mac_ver()[0],
66
+ "machine": platform.machine(),
67
+ "coremltools": ct.__version__,
68
+ }
69
+ folder = Path(args.model_dir)
70
+ path = folder / f"benchmark-{args.precision}-{args.units}.json"
71
+ path.write_text(json.dumps(report, indent=2) + "\n")
72
+ print(json.dumps(report, indent=2))
73
+
74
+
75
+ if __name__ == "__main__":
76
+ main()
config.json ADDED
@@ -0,0 +1,99 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architecture": "boundary",
3
+ "architecture_version": 1,
4
+ "architectures": [
5
+ "BoundaryExtractor"
6
+ ],
7
+ "attn_implementation": "sdpa",
8
+ "boundary_head": {
9
+ "abstention_loss_weight": 0.2,
10
+ "abstention_threshold": 0.5,
11
+ "adaptive_threshold": false,
12
+ "bidirectional_proposals": true,
13
+ "boundary_attention_heads": 4,
14
+ "boundary_attention_layers": 2,
15
+ "boundary_attention_window": 128,
16
+ "boundary_dim": 128,
17
+ "boundary_ffn_multiplier": 2.0,
18
+ "boundary_focal_clip": 0.05,
19
+ "boundary_focal_gamma_negative": 2.0,
20
+ "boundary_focal_gamma_positive": 0.0,
21
+ "boundary_marginal_loss": "asymmetric_focal",
22
+ "boundary_negative_weight": 0.5,
23
+ "boundary_refinement_layers": 1,
24
+ "boundary_top_k_alpha": 0.08,
25
+ "boundary_top_k_bucket": 8,
26
+ "boundary_top_k_max": 128,
27
+ "candidate_attention_heads": 4,
28
+ "candidate_attention_layers": 0,
29
+ "candidate_budget": 192,
30
+ "candidate_pool": "shared",
31
+ "classification_loss_weight": 1.0,
32
+ "classification_temperature": 1.0,
33
+ "consistency_loss_weight": 0.1,
34
+ "consistency_warmup_steps": 2000,
35
+ "content_dim": 64,
36
+ "content_soft_max_pool": false,
37
+ "count_loss_weight": 0.2,
38
+ "directional_relation_states": true,
39
+ "dropout": 0.1,
40
+ "enable_abstention": true,
41
+ "enable_count_head": true,
42
+ "enable_records": true,
43
+ "enable_relations": true,
44
+ "enable_rotary_endpoints": true,
45
+ "enable_span_content": true,
46
+ "end_block_size": 256,
47
+ "end_top_k": 24,
48
+ "endpoint_difference_features": true,
49
+ "ends_per_start": 12,
50
+ "export_mode": "auto",
51
+ "hard_negative_keep_all_when_absent": true,
52
+ "hard_negatives_per_positive": 20,
53
+ "loss_reduction": "sum",
54
+ "max_gold_per_query": 64,
55
+ "max_negative_queries_per_batch": 64,
56
+ "min_pool_per_query": 8,
57
+ "minimum_hard_negatives": 16,
58
+ "multihead_pair_compat_heads": 8,
59
+ "negative_query_ratio": 1.0,
60
+ "overlap_policy": "flat",
61
+ "pair_dim": 128,
62
+ "pair_temperature": 1.0,
63
+ "pool_boundary_top_k": 32,
64
+ "pool_size": 192,
65
+ "proposal_loss_weight": 0.3,
66
+ "query_attention_layers": 0,
67
+ "query_conditioned_inside_weight": true,
68
+ "record_anchor_proposal_threshold": 0.2,
69
+ "record_anchor_threshold": 0.5,
70
+ "record_dim": 128,
71
+ "record_field_threshold": 0.5,
72
+ "record_instance_queries": 32,
73
+ "record_loss_weight": 1.0,
74
+ "record_temperature": 1.0,
75
+ "relation_argument_proposal_threshold": 0.2,
76
+ "relation_biaffine_content": true,
77
+ "relation_heads_per_type": 32,
78
+ "relation_loss_weight": 1.0,
79
+ "relation_pair_cap": 64,
80
+ "relation_tails_per_type": 32,
81
+ "relation_temperature": 1.0,
82
+ "rerank_listwise_weight": 0.3,
83
+ "reranker_endpoint_compat": true,
84
+ "rotary_base": 10000.0,
85
+ "soft_iou_anneal_steps": 20000,
86
+ "soft_iou_aux_weight": 0.2,
87
+ "start_top_k": 24,
88
+ "starts_per_end": 12,
89
+ "training_candidate_budget": 192,
90
+ "use_inside_evidence": true,
91
+ "vectorized_pair_elements": 16777216
92
+ },
93
+ "config_version": 3,
94
+ "max_len": 4096,
95
+ "model_name": "microsoft/mdeberta-v3-base",
96
+ "model_type": "extractor",
97
+ "token_pooling": "first",
98
+ "transformers_version": "5.8.0"
99
+ }
convert-explicit-coreml.py ADDED
@@ -0,0 +1,170 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Export GLiNER2.5 multilingual trained explicit-span scorer for attributes and enums."""
2
+
3
+ import argparse
4
+ import json
5
+ from pathlib import Path
6
+
7
+ import coremltools as ct
8
+ import numpy as np
9
+ import torch
10
+ from gliner2 import AutoExtractor, Schema
11
+ from huggingface_hub import snapshot_download
12
+
13
+ from extraction_export import ExtractionExplicitSpanExport, ExtractionFeaturesExport, coreml_trace_patches
14
+ from preprocessing import prepare_extraction
15
+
16
+ MODEL_ID = "fastino/gliner2.5-multi-v1"
17
+ MODEL_REVISION = "a221b77a8baf4a613b8f8652661d41fa10a5641e"
18
+ INPUT_NAMES = (
19
+ "text_states",
20
+ "text_mask",
21
+ "query_states",
22
+ "query_mask",
23
+ "boundary_states",
24
+ "start_logits",
25
+ "end_logits",
26
+ "inside_prefix",
27
+ "inside_prefix_mean",
28
+ "span_indices",
29
+ "span_mask",
30
+ )
31
+
32
+
33
+ def main():
34
+ parser = argparse.ArgumentParser()
35
+ parser.add_argument("--output-dir", default="build/extraction")
36
+ parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32")
37
+ parser.add_argument("--length", type=int, default=128)
38
+ parser.add_argument("--max-words", type=int, default=64)
39
+ parser.add_argument("--max-queries", type=int, default=8)
40
+ parser.add_argument("--max-spans", type=int, default=64)
41
+ args = parser.parse_args()
42
+ torch.set_num_threads(4)
43
+ source = snapshot_download(
44
+ MODEL_ID,
45
+ revision=MODEL_REVISION,
46
+ allow_patterns=[
47
+ "config.json",
48
+ "encoder_config/*",
49
+ "model.safetensors",
50
+ "tokenizer.json",
51
+ "tokenizer_config.json",
52
+ ],
53
+ )
54
+ native = AutoExtractor.from_pretrained(source, map_location="cpu").eval()
55
+ text = "Alice founded Acme in Toronto in 2020."
56
+ schema = Schema().entities(["person", "organization", "location"])
57
+ arrays, batch = prepare_extraction(native.processor, text, schema, args.length, args.max_words, args.max_queries)
58
+ tensors = tuple(torch.from_numpy(value) for value in arrays.values())
59
+ with torch.no_grad(), coreml_trace_patches():
60
+ features = ExtractionFeaturesExport(native).eval()(*tensors)
61
+ with torch.no_grad():
62
+ core = native._encode_core(batch)
63
+ candidates = native.boundary_head(
64
+ core["text_states"], core["text_mask"], core["query_states"], core["query_mask"]
65
+ ).candidates
66
+ query_count = core["query_states"].shape[1]
67
+ indices = torch.zeros(1, args.max_queries, args.max_spans, 2, dtype=torch.int32)
68
+ mask = torch.zeros(1, args.max_queries, args.max_spans, dtype=torch.float32)
69
+ count = min(args.max_spans, candidates.indices.shape[2])
70
+ indices[:, :query_count, :count] = candidates.indices[:, :query_count, :count].int()
71
+ mask[:, :query_count, :count] = candidates.valid_mask[:, :query_count, :count].float()
72
+ wrapper = ExtractionExplicitSpanExport(native).eval()
73
+ arguments = (
74
+ features[0],
75
+ tensors[3],
76
+ features[1],
77
+ tensors[5],
78
+ features[2],
79
+ features[4],
80
+ features[5],
81
+ features[6],
82
+ features[7],
83
+ indices,
84
+ mask,
85
+ )
86
+ with torch.no_grad():
87
+ reference = wrapper(*arguments)
88
+ native_reference = native.boundary_head.score_explicit_spans(
89
+ core["text_states"],
90
+ core["text_mask"],
91
+ core["query_states"],
92
+ core["query_mask"],
93
+ indices[:, :query_count].long(),
94
+ mask[:, :query_count].bool(),
95
+ )
96
+ wrapper_error = float(
97
+ (reference[:, :query_count][mask[:, :query_count].bool()] - native_reference[mask[:, :query_count].bool()])
98
+ .abs()
99
+ .max()
100
+ )
101
+ traced = torch.jit.trace(wrapper, arguments, check_trace=False)
102
+ if wrapper_error > 1e-4:
103
+ raise RuntimeError(f"Explicit span wrapper differs from native: {wrapper_error}")
104
+ precision = ct.precision.FLOAT16 if args.precision == "fp16" else ct.precision.FLOAT32
105
+ converted = ct.convert(
106
+ traced,
107
+ convert_to="mlprogram",
108
+ minimum_deployment_target=ct.target.iOS17,
109
+ compute_precision=precision,
110
+ compute_units=ct.ComputeUnit.CPU_ONLY,
111
+ inputs=[
112
+ ct.TensorType(name=name, shape=tuple(value.shape), dtype=np.int32 if name == "span_indices" else np.float32)
113
+ for name, value in zip(INPUT_NAMES, arguments)
114
+ ],
115
+ outputs=[ct.TensorType(name="span_logits", dtype=np.float32)],
116
+ )
117
+ converted.short_description = "GLiNER2.5 multilingual trained explicit-span extraction scorer"
118
+ converted.author = "Fastino (original); Fluid Inference (Core ML conversion)"
119
+ converted.license = "Apache-2.0"
120
+ converted.user_defined_metadata.update(
121
+ {
122
+ "source_model": MODEL_ID,
123
+ "source_revision": MODEL_REVISION,
124
+ "stage": "trained explicit-span proposal and reranker",
125
+ "word_capacity": str(args.max_words),
126
+ "query_capacity": str(args.max_queries),
127
+ "span_capacity": str(args.max_spans),
128
+ }
129
+ )
130
+ out = Path(args.output_dir)
131
+ out.mkdir(parents=True, exist_ok=True)
132
+ suffix = f"{args.precision}_W{args.max_words}_Q{args.max_queries}_S{args.max_spans}"
133
+ package = out / f"gliner2_multi_explicit_{suffix}.mlpackage"
134
+ converted.save(str(package))
135
+ runtime = ct.models.MLModel(str(package), compute_units=ct.ComputeUnit.CPU_ONLY)
136
+ predicted = runtime.predict(
137
+ {
138
+ name: value.detach().numpy().astype(np.int32 if name == "span_indices" else np.float32)
139
+ for name, value in zip(INPUT_NAMES, arguments)
140
+ }
141
+ )["span_logits"]
142
+ runtime_error = float(
143
+ np.max(
144
+ np.abs(
145
+ predicted[:, :query_count][mask[:, :query_count].bool().numpy()]
146
+ - reference.numpy()[:, :query_count][mask[:, :query_count].bool().numpy()]
147
+ )
148
+ )
149
+ )
150
+ if not np.isfinite(runtime_error):
151
+ raise RuntimeError("Explicit-span scorer produced non-finite logits")
152
+ report = {
153
+ "source_model": MODEL_ID,
154
+ "source_revision": MODEL_REVISION,
155
+ "precision": args.precision,
156
+ "fixture": text,
157
+ "valid_spans": int(mask.sum()),
158
+ "wrapper_max_absolute_error": wrapper_error,
159
+ "coreml_max_absolute_error": runtime_error,
160
+ "package": str(package),
161
+ "package_bytes": sum(file.stat().st_size for file in package.rglob("*") if file.is_file()),
162
+ "coremltools": ct.__version__,
163
+ "torch": torch.__version__,
164
+ }
165
+ (out / f"explicit-{suffix}.json").write_text(json.dumps(report, indent=2) + "\n")
166
+ print(json.dumps(report, indent=2))
167
+
168
+
169
+ if __name__ == "__main__":
170
+ main()
convert-extraction-coreml.py ADDED
@@ -0,0 +1,217 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Export the trained GLiNER2.5 multilingual boundary extraction stages to Core ML."""
2
+
3
+ import argparse
4
+ import json
5
+ import shutil
6
+ from pathlib import Path
7
+
8
+ import coremltools as ct
9
+ import numpy as np
10
+ import torch
11
+ from gliner2 import AutoExtractor, Schema
12
+ from huggingface_hub import snapshot_download
13
+
14
+ from convert_extraction_names import FEATURE_NAMES, SCORE_INPUT_NAMES
15
+ from extraction_export import ExtractionFeaturesExport, ExtractionScoreExport, coreml_trace_patches
16
+ from extraction_pool import select_candidates
17
+ from preprocessing import prepare_extraction
18
+
19
+ MODEL_ID = "fastino/gliner2.5-multi-v1"
20
+ MODEL_REVISION = "a221b77a8baf4a613b8f8652661d41fa10a5641e"
21
+ FIXTURE_TEXT = "Alice founded Acme in Toronto in 2020."
22
+ FIXTURE_SCHEMA = Schema().entities(["person", "organization", "location"])
23
+
24
+
25
+ def package_bytes(path: Path) -> int:
26
+ return sum(file.stat().st_size for file in path.rglob("*") if file.is_file())
27
+
28
+
29
+ def main():
30
+ parser = argparse.ArgumentParser()
31
+ parser.add_argument("--output-dir", default="build/extraction")
32
+ parser.add_argument("--length", type=int, default=128, help="Subword capacity")
33
+ parser.add_argument("--max-words", type=int, default=64)
34
+ parser.add_argument("--max-queries", type=int, default=8)
35
+ parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32")
36
+ args = parser.parse_args()
37
+ torch.set_num_threads(4)
38
+ source = snapshot_download(
39
+ MODEL_ID,
40
+ revision=MODEL_REVISION,
41
+ allow_patterns=[
42
+ "config.json",
43
+ "encoder_config/*",
44
+ "model.safetensors",
45
+ "tokenizer.json",
46
+ "tokenizer_config.json",
47
+ ],
48
+ )
49
+ native = AutoExtractor.from_pretrained(source, map_location="cpu").eval()
50
+ arrays, batch = prepare_extraction(
51
+ native.processor, FIXTURE_TEXT, FIXTURE_SCHEMA, args.length, args.max_words, args.max_queries
52
+ )
53
+ tensors = tuple(torch.from_numpy(value) for value in arrays.values())
54
+ features_wrapper = ExtractionFeaturesExport(native).eval()
55
+ with torch.no_grad(), coreml_trace_patches():
56
+ features_reference = features_wrapper(*tensors)
57
+ traced_features = torch.jit.trace(features_wrapper, tensors, check_trace=False)
58
+ with torch.no_grad():
59
+ native_core = native._encode_core(batch)
60
+ valid_words = native_core["text_states"].shape[1]
61
+ valid_queries = native_core["query_states"].shape[1]
62
+ routing_error = max(
63
+ float((features_reference[0][:, :valid_words] - native_core["text_states"]).abs().max()),
64
+ float((features_reference[1][:, :valid_queries] - native_core["query_states"]).abs().max()),
65
+ )
66
+ if routing_error > 1e-4:
67
+ raise RuntimeError(f"Traced routing differs from native: {routing_error}")
68
+
69
+ precision = ct.precision.FLOAT16 if args.precision == "fp16" else ct.precision.FLOAT32
70
+ input_names = tuple(arrays)
71
+ features_model = ct.convert(
72
+ traced_features,
73
+ convert_to="mlprogram",
74
+ minimum_deployment_target=ct.target.iOS17,
75
+ compute_precision=precision,
76
+ compute_units=ct.ComputeUnit.CPU_ONLY,
77
+ inputs=[ct.TensorType(name=name, shape=arrays[name].shape, dtype=arrays[name].dtype) for name in input_names],
78
+ outputs=[ct.TensorType(name=name, dtype=np.float32) for name in FEATURE_NAMES],
79
+ )
80
+ features_model.short_description = "GLiNER2.5 multilingual trained boundary extraction features"
81
+ features_model.author = "Fastino (original); Fluid Inference (Core ML conversion)"
82
+ features_model.license = "Apache-2.0"
83
+ features_model.user_defined_metadata.update(
84
+ {
85
+ "source_model": MODEL_ID,
86
+ "source_revision": MODEL_REVISION,
87
+ "stage": "extraction features and trained boundary heads",
88
+ "subword_capacity": str(args.length),
89
+ "word_capacity": str(args.max_words),
90
+ "query_capacity": str(args.max_queries),
91
+ }
92
+ )
93
+ out = Path(args.output_dir)
94
+ out.mkdir(parents=True, exist_ok=True)
95
+ shutil.copy2(Path(source) / "config.json", out / "config.json")
96
+ tokenizer_dir = out / "tokenizer"
97
+ tokenizer_dir.mkdir(exist_ok=True)
98
+ for name in ("tokenizer.json", "tokenizer_config.json"):
99
+ shutil.copy2(Path(source) / name, tokenizer_dir / name)
100
+ suffix = f"{args.precision}_L{args.length}_W{args.max_words}_Q{args.max_queries}"
101
+ features_path = out / f"gliner2_multi_extraction_features_{suffix}.mlpackage"
102
+ if features_path.exists():
103
+ shutil.rmtree(features_path)
104
+ features_model.save(str(features_path))
105
+ print(f"Saved {features_path}", flush=True)
106
+
107
+ head = native.boundary_head
108
+ pooled = select_candidates(
109
+ features_reference[8],
110
+ features_reference[9],
111
+ features_reference[3].bool(),
112
+ tensors[5].bool(),
113
+ features_reference[4],
114
+ features_reference[5],
115
+ boundary_top_k=head.shared_pool_builder.pool_boundary_top_k,
116
+ pool_size=head.shared_pool_builder.pool_size,
117
+ min_pool_per_query=head.shared_pool_builder.min_pool_per_query,
118
+ )
119
+ score_tensors = (
120
+ features_reference[0],
121
+ tensors[3],
122
+ features_reference[1],
123
+ tensors[5],
124
+ features_reference[2],
125
+ features_reference[4],
126
+ features_reference[5],
127
+ features_reference[6],
128
+ features_reference[7],
129
+ pooled.indices.int(),
130
+ pooled.mask.float(),
131
+ pooled.compat_logits,
132
+ )
133
+ scorer_wrapper = ExtractionScoreExport(native).eval()
134
+ with torch.no_grad():
135
+ scores_reference = scorer_wrapper(*score_tensors)
136
+ traced_scores = torch.jit.trace(scorer_wrapper, score_tensors, check_trace=False)
137
+ scorer_model = ct.convert(
138
+ traced_scores,
139
+ convert_to="mlprogram",
140
+ minimum_deployment_target=ct.target.iOS17,
141
+ compute_precision=precision,
142
+ compute_units=ct.ComputeUnit.CPU_ONLY,
143
+ inputs=[
144
+ ct.TensorType(
145
+ name=name, shape=tuple(value.shape), dtype=np.int32 if name == "candidate_indices" else np.float32
146
+ )
147
+ for name, value in zip(SCORE_INPUT_NAMES, score_tensors)
148
+ ],
149
+ outputs=[
150
+ ct.TensorType(name="pair_logits", dtype=np.float32),
151
+ ct.TensorType(name="candidate_states", dtype=np.float32),
152
+ ],
153
+ )
154
+ scorer_model.short_description = "GLiNER2.5 multilingual trained shared-pool extraction scorer"
155
+ scorer_model.author = "Fastino (original); Fluid Inference (Core ML conversion)"
156
+ scorer_model.license = "Apache-2.0"
157
+ scorer_model.user_defined_metadata.update(
158
+ {
159
+ "source_model": MODEL_ID,
160
+ "source_revision": MODEL_REVISION,
161
+ "stage": "trained extraction candidate scorer",
162
+ "candidate_capacity": str(head.shared_pool_builder.pool_size),
163
+ }
164
+ )
165
+ scorer_path = out / f"gliner2_multi_extraction_scorer_{suffix}.mlpackage"
166
+ if scorer_path.exists():
167
+ shutil.rmtree(scorer_path)
168
+ scorer_model.save(str(scorer_path))
169
+ print(f"Saved {scorer_path}", flush=True)
170
+
171
+ # The first runtime check uses the same selected real fixture as the trace.
172
+ runtime_features = ct.models.MLModel(str(features_path), compute_units=ct.ComputeUnit.CPU_ONLY)
173
+ predicted_features = runtime_features.predict(arrays)
174
+ errors = {
175
+ name: float(np.max(np.abs(np.asarray(predicted_features[name]) - reference.detach().numpy())))
176
+ for name, reference in zip(FEATURE_NAMES, features_reference)
177
+ }
178
+ if any(not np.isfinite(value) for value in errors.values()):
179
+ raise RuntimeError("Extraction features contain non-finite values")
180
+ runtime_scorer = ct.models.MLModel(str(scorer_path), compute_units=ct.ComputeUnit.CPU_ONLY)
181
+ score_arrays = {
182
+ name: value.detach().numpy().astype(np.int32 if name == "candidate_indices" else np.float32)
183
+ for name, value in zip(SCORE_INPUT_NAMES, score_tensors)
184
+ }
185
+ predicted_scores = runtime_scorer.predict(score_arrays)
186
+ errors["pair_logits"] = float(np.max(np.abs(predicted_scores["pair_logits"] - scores_reference[0].numpy())))
187
+ errors["candidate_states"] = float(
188
+ np.max(np.abs(predicted_scores["candidate_states"] - scores_reference[1].numpy()))
189
+ )
190
+ if any(not np.isfinite(value) for value in errors.values()):
191
+ raise RuntimeError("Extraction scorer contains non-finite values")
192
+ report = {
193
+ "source_model": MODEL_ID,
194
+ "source_revision": MODEL_REVISION,
195
+ "precision": args.precision,
196
+ "fixture": FIXTURE_TEXT,
197
+ "shape": {
198
+ "subwords": args.length,
199
+ "words": args.max_words,
200
+ "queries": args.max_queries,
201
+ "candidates": head.shared_pool_builder.pool_size,
202
+ },
203
+ "routing_max_absolute_error": routing_error,
204
+ "runtime_max_absolute_errors": errors,
205
+ "packages": {
206
+ "features": {"path": str(features_path), "bytes": package_bytes(features_path)},
207
+ "scorer": {"path": str(scorer_path), "bytes": package_bytes(scorer_path)},
208
+ },
209
+ "coremltools": ct.__version__,
210
+ "torch": torch.__version__,
211
+ }
212
+ (out / f"conversion-{suffix}.json").write_text(json.dumps(report, indent=2) + "\n")
213
+ print(json.dumps(report, indent=2))
214
+
215
+
216
+ if __name__ == "__main__":
217
+ main()
convert-record-coreml.py ADDED
@@ -0,0 +1,222 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Export GLiNER2.5 multilingual trained record assignment and anchorless heads."""
2
+
3
+ import argparse
4
+ import json
5
+ import shutil
6
+ from pathlib import Path
7
+
8
+ import coremltools as ct
9
+ import numpy as np
10
+ import torch
11
+ from gliner2 import AutoExtractor, Schema
12
+ from gliner2.training.trainer import ExtractorCollator
13
+ from huggingface_hub import snapshot_download
14
+
15
+ from extraction_export import ExtractionRecordAnchorlessExport, ExtractionRecordAssignmentExport
16
+
17
+ MODEL_ID = "fastino/gliner2.5-multi-v1"
18
+ MODEL_REVISION = "a221b77a8baf4a613b8f8652661d41fa10a5641e"
19
+ FIXTURE_TEXT = "Alice works at Acme. Bob works at Beta."
20
+
21
+
22
+ def record_fixture(native, mode):
23
+ schema = Schema()
24
+ builder = schema.structure("employment", mode=mode, anchor="person" if mode == "natural" else None)
25
+ builder.field("person", dtype="str")
26
+ builder.field("company", dtype="str")
27
+ batch = ExtractorCollator(native.processor, is_training=False, max_len=None, architecture="boundary")(
28
+ [(FIXTURE_TEXT, schema.build())]
29
+ )
30
+ with torch.no_grad():
31
+ core = native._encode_core(batch)
32
+ candidates = native.boundary_head(
33
+ core["text_states"], core["text_mask"], core["query_states"], core["query_mask"]
34
+ ).candidates
35
+ spec = next(iter(batch.record_specs[0].values()))
36
+ group = native.record_decoder.forward_group(spec, core["query_states"][0], candidates, 0)
37
+ field_states = [
38
+ candidates.candidate_states[0, query_id][candidates.valid_mask[0, query_id]]
39
+ for query_id in group.field_query_ids
40
+ ]
41
+ queries = core["query_states"][0][group.field_query_ids]
42
+ return group, spec, field_states, queries
43
+
44
+
45
+ def pad_first(value, size: int):
46
+ if value.shape[0] > size:
47
+ raise ValueError(f"Real record fixture exceeds bucket capacity {size}")
48
+ result = value.new_zeros((size, *value.shape[1:]))
49
+ result[: value.shape[0]] = value
50
+ return result
51
+
52
+
53
+ def package_bytes(path):
54
+ return sum(file.stat().st_size for file in path.rglob("*") if file.is_file())
55
+
56
+
57
+ def main():
58
+ parser = argparse.ArgumentParser()
59
+ parser.add_argument("--output-dir", default="build/extraction")
60
+ parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32")
61
+ parser.add_argument("--max-fields", type=int, default=8)
62
+ parser.add_argument("--max-candidates", type=int, default=192)
63
+ parser.add_argument("--max-instances", type=int, default=1536)
64
+ args = parser.parse_args()
65
+ torch.set_num_threads(4)
66
+ source = snapshot_download(
67
+ MODEL_ID,
68
+ revision=MODEL_REVISION,
69
+ allow_patterns=[
70
+ "config.json",
71
+ "encoder_config/*",
72
+ "model.safetensors",
73
+ "tokenizer.json",
74
+ "tokenizer_config.json",
75
+ ],
76
+ )
77
+ native = AutoExtractor.from_pretrained(source, map_location="cpu").eval()
78
+ head = native.record_decoder
79
+ if args.max_instances < args.max_fields * args.max_candidates:
80
+ raise ValueError("Instance bucket must hold all latent field candidates")
81
+
82
+ group, spec, field_states, queries = record_fixture(native, "natural")
83
+ anchor_index = group.field_query_ids.index(spec.anchor_query_id)
84
+ instances = field_states[anchor_index]
85
+ hidden = instances.shape[-1]
86
+ field_candidates = torch.zeros(args.max_fields, args.max_candidates, hidden)
87
+ for field_index, states in enumerate(field_states):
88
+ if states.shape[0] > args.max_candidates:
89
+ raise ValueError("Record candidate count exceeds bucket")
90
+ field_candidates[field_index, : states.shape[0]] = states
91
+ assignment_args = (pad_first(instances, args.max_instances), pad_first(queries, args.max_fields), field_candidates)
92
+ assignment_wrapper = ExtractionRecordAssignmentExport(native).eval()
93
+ with torch.no_grad():
94
+ assignment_reference = assignment_wrapper(*assignment_args)
95
+ for field_index, expected in enumerate(group.assign_logits):
96
+ actual = assignment_reference[0][: instances.shape[0], field_index, : expected.shape[1]]
97
+ if not torch.allclose(actual, expected, atol=1e-4):
98
+ raise RuntimeError("Record assignment wrapper differs from native")
99
+ assignment_trace = torch.jit.trace(assignment_wrapper, assignment_args, check_trace=False)
100
+
101
+ precision = ct.precision.FLOAT16 if args.precision == "fp16" else ct.precision.FLOAT32
102
+ assignment_model = ct.convert(
103
+ assignment_trace,
104
+ convert_to="mlprogram",
105
+ minimum_deployment_target=ct.target.iOS17,
106
+ compute_precision=precision,
107
+ compute_units=ct.ComputeUnit.CPU_ONLY,
108
+ inputs=[
109
+ ct.TensorType(name=name, shape=tuple(value.shape), dtype=np.float32)
110
+ for name, value in zip(("instance_states", "field_queries", "field_candidate_states"), assignment_args)
111
+ ],
112
+ outputs=[
113
+ ct.TensorType(name="assignment_logits", dtype=np.float32),
114
+ ct.TensorType(name="object_logits", dtype=np.float32),
115
+ ct.TensorType(name="latent_seed_logits", dtype=np.float32),
116
+ ],
117
+ )
118
+ assignment_model.short_description = "GLiNER2.5 multilingual trained record assignment and object heads"
119
+ assignment_model.author = "Fastino (original); Fluid Inference (Core ML conversion)"
120
+ assignment_model.license = "Apache-2.0"
121
+ assignment_model.user_defined_metadata.update(
122
+ {
123
+ "source_model": MODEL_ID,
124
+ "source_revision": MODEL_REVISION,
125
+ "stage": "trained record assignment, object and latent seed heads",
126
+ "field_capacity": str(args.max_fields),
127
+ "candidate_capacity": str(args.max_candidates),
128
+ "instance_capacity": str(args.max_instances),
129
+ }
130
+ )
131
+ out = Path(args.output_dir)
132
+ out.mkdir(parents=True, exist_ok=True)
133
+ suffix = f"{args.precision}_F{args.max_fields}_C{args.max_candidates}_I{args.max_instances}"
134
+ assignment_path = out / f"gliner2_multi_record_assignment_{suffix}.mlpackage"
135
+ if assignment_path.exists():
136
+ shutil.rmtree(assignment_path)
137
+ assignment_model.save(str(assignment_path))
138
+ assignment_runtime = ct.models.MLModel(str(assignment_path), compute_units=ct.ComputeUnit.CPU_ONLY)
139
+ assignment_prediction = assignment_runtime.predict(
140
+ {
141
+ name: value.numpy().astype(np.float32)
142
+ for name, value in zip(("instance_states", "field_queries", "field_candidate_states"), assignment_args)
143
+ }
144
+ )
145
+ assignment_errors = {
146
+ name: float(np.max(np.abs(assignment_prediction[name] - expected.numpy())))
147
+ for name, expected in zip(("assignment_logits", "object_logits", "latent_seed_logits"), assignment_reference)
148
+ }
149
+
150
+ _, _, anchorless_field_states, _ = record_fixture(native, "anchorless")
151
+ context = torch.cat(anchorless_field_states, 0)
152
+ context_size = args.max_fields * args.max_candidates
153
+ context_states = pad_first(context, context_size)
154
+ context_mask = torch.zeros(context_size, dtype=torch.float32)
155
+ context_mask[: context.shape[0]] = 1.0
156
+ anchorless_wrapper = ExtractionRecordAnchorlessExport(native).eval()
157
+ with torch.no_grad():
158
+ anchorless_reference = anchorless_wrapper(context_states, context_mask)
159
+ native_states = head._anchorless_states(anchorless_field_states)
160
+ wrapper_error = float((anchorless_reference - native_states).abs().max())
161
+ if wrapper_error > 1e-4:
162
+ raise RuntimeError(f"Anchorless wrapper differs from native: {wrapper_error}")
163
+ anchorless_trace = torch.jit.trace(anchorless_wrapper, (context_states, context_mask), check_trace=False)
164
+ anchorless_model = ct.convert(
165
+ anchorless_trace,
166
+ convert_to="mlprogram",
167
+ minimum_deployment_target=ct.target.iOS17,
168
+ compute_precision=precision,
169
+ compute_units=ct.ComputeUnit.CPU_ONLY,
170
+ inputs=[
171
+ ct.TensorType(name="context_states", shape=tuple(context_states.shape), dtype=np.float32),
172
+ ct.TensorType(name="context_mask", shape=tuple(context_mask.shape), dtype=np.float32),
173
+ ],
174
+ outputs=[ct.TensorType(name="instance_states", dtype=np.float32)],
175
+ )
176
+ anchorless_model.short_description = "GLiNER2.5 multilingual trained anchorless record instance head"
177
+ anchorless_model.author = "Fastino (original); Fluid Inference (Core ML conversion)"
178
+ anchorless_model.license = "Apache-2.0"
179
+ anchorless_model.user_defined_metadata.update(
180
+ {
181
+ "source_model": MODEL_ID,
182
+ "source_revision": MODEL_REVISION,
183
+ "stage": "trained anchorless record instance head",
184
+ "context_capacity": str(context_size),
185
+ }
186
+ )
187
+ anchorless_path = out / f"gliner2_multi_record_anchorless_{suffix}.mlpackage"
188
+ if anchorless_path.exists():
189
+ shutil.rmtree(anchorless_path)
190
+ anchorless_model.save(str(anchorless_path))
191
+ anchorless_runtime = ct.models.MLModel(str(anchorless_path), compute_units=ct.ComputeUnit.CPU_ONLY)
192
+ anchorless_prediction = anchorless_runtime.predict(
193
+ {
194
+ "context_states": context_states.numpy().astype(np.float32),
195
+ "context_mask": context_mask.numpy().astype(np.float32),
196
+ }
197
+ )["instance_states"]
198
+ anchorless_error = float(np.max(np.abs(anchorless_prediction - anchorless_reference.numpy())))
199
+ if not all(np.isfinite(value) for value in (*assignment_errors.values(), anchorless_error)):
200
+ raise RuntimeError("Record head produced non-finite values")
201
+ report = {
202
+ "source_model": MODEL_ID,
203
+ "source_revision": MODEL_REVISION,
204
+ "precision": args.precision,
205
+ "fixture": FIXTURE_TEXT,
206
+ "shape": {"fields": args.max_fields, "candidates": args.max_candidates, "instances": args.max_instances},
207
+ "assignment_max_absolute_errors": assignment_errors,
208
+ "anchorless_wrapper_max_absolute_error": wrapper_error,
209
+ "anchorless_coreml_max_absolute_error": anchorless_error,
210
+ "packages": {
211
+ "assignment": {"path": str(assignment_path), "bytes": package_bytes(assignment_path)},
212
+ "anchorless": {"path": str(anchorless_path), "bytes": package_bytes(anchorless_path)},
213
+ },
214
+ "coremltools": ct.__version__,
215
+ "torch": torch.__version__,
216
+ }
217
+ (out / f"record-{suffix}.json").write_text(json.dumps(report, indent=2) + "\n")
218
+ print(json.dumps(report, indent=2))
219
+
220
+
221
+ if __name__ == "__main__":
222
+ main()
convert-relation-coreml.py ADDED
@@ -0,0 +1,165 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Export the trained GLiNER2.5 multilingual sparse relation scorer to Core ML."""
2
+
3
+ import argparse
4
+ import json
5
+ from pathlib import Path
6
+
7
+ import coremltools as ct
8
+ import numpy as np
9
+ import torch
10
+ from gliner2 import AutoExtractor, Schema
11
+ from gliner2.models.base import QueryLayout
12
+ from gliner2.training.trainer import ExtractorCollator
13
+ from huggingface_hub import snapshot_download
14
+
15
+ from extraction_export import ExtractionRelationExport
16
+
17
+ MODEL_ID = "fastino/gliner2.5-multi-v1"
18
+ MODEL_REVISION = "a221b77a8baf4a613b8f8652661d41fa10a5641e"
19
+ INPUT_NAMES = (
20
+ "text_states",
21
+ "text_length",
22
+ "relation_states",
23
+ "batch_index",
24
+ "relation_index",
25
+ "head_start",
26
+ "head_end",
27
+ "tail_start",
28
+ "tail_end",
29
+ "pair_mask",
30
+ )
31
+
32
+
33
+ def pad(value, size: int, fill=0):
34
+ if value.shape[0] > size:
35
+ raise ValueError(f"Relation fixture exceeds capacity {size}")
36
+ output = value.new_full((size, *value.shape[1:]), fill)
37
+ output[: value.shape[0]] = value
38
+ return output
39
+
40
+
41
+ def main():
42
+ parser = argparse.ArgumentParser()
43
+ parser.add_argument("--output-dir", default="build/extraction")
44
+ parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32")
45
+ parser.add_argument("--max-words", type=int, default=64)
46
+ parser.add_argument("--max-relations", type=int, default=4)
47
+ args = parser.parse_args()
48
+ torch.set_num_threads(4)
49
+ source = snapshot_download(
50
+ MODEL_ID,
51
+ revision=MODEL_REVISION,
52
+ allow_patterns=[
53
+ "config.json",
54
+ "encoder_config/*",
55
+ "model.safetensors",
56
+ "tokenizer.json",
57
+ "tokenizer_config.json",
58
+ ],
59
+ )
60
+ native = AutoExtractor.from_pretrained(source, map_location="cpu").eval()
61
+ text = "Alice founded Acme in Toronto."
62
+ schema = Schema().relations(["founded"])
63
+ batch = ExtractorCollator(native.processor, is_training=False, max_len=None, architecture="boundary")(
64
+ [(text, schema.build())]
65
+ )
66
+ with torch.no_grad():
67
+ core = native._encode_core(batch)
68
+ output = native.boundary_head(core["text_states"], core["text_mask"], core["query_states"], core["query_mask"])
69
+ sample = native._single_sample_candidates(output.candidates, 0)
70
+ relation_specs = core["rel_specs"][0]
71
+ pairs = native.relation_pair_generator.generate_batched(
72
+ sample, [QueryLayout(queries=())], [[entry["spec"] for entry in relation_specs]], compact=False
73
+ )
74
+ relation_states = torch.stack([entry["query_state"] for entry in relation_specs]).unsqueeze(0)
75
+ native_scores = native.relation_scorer(core["text_states"], relation_states, sample, pairs)
76
+ pair_cap = args.max_relations * native.boundary_settings.relation_pair_cap
77
+ text_states = torch.zeros(1, args.max_words, core["text_states"].shape[-1])
78
+ text_states[:, : core["text_states"].shape[1]] = core["text_states"]
79
+ relation_padded = torch.zeros(1, args.max_relations, relation_states.shape[-1])
80
+ relation_padded[:, : relation_states.shape[1]] = relation_states
81
+ arguments = (
82
+ text_states,
83
+ torch.tensor([core["text_states"].shape[1]], dtype=torch.int32),
84
+ relation_padded,
85
+ pad(pairs.batch_index.int(), pair_cap),
86
+ pad(pairs.relation_index.int(), pair_cap),
87
+ pad(pairs.head_start.int(), pair_cap),
88
+ pad(pairs.head_end.int(), pair_cap),
89
+ pad(pairs.tail_start.int(), pair_cap),
90
+ pad(pairs.tail_end.int(), pair_cap),
91
+ pad(pairs.pair_mask.float(), pair_cap),
92
+ )
93
+ wrapper = ExtractionRelationExport(native).eval()
94
+ with torch.no_grad():
95
+ reference = wrapper(*arguments)
96
+ wrapper_error = float((reference[: len(pairs)] - native_scores).abs().max())
97
+ traced = torch.jit.trace(wrapper, arguments, check_trace=False)
98
+ if wrapper_error > 1e-4:
99
+ raise RuntimeError(f"Relation wrapper differs from native: {wrapper_error}")
100
+ precision = ct.precision.FLOAT16 if args.precision == "fp16" else ct.precision.FLOAT32
101
+ converted = ct.convert(
102
+ traced,
103
+ convert_to="mlprogram",
104
+ minimum_deployment_target=ct.target.iOS17,
105
+ compute_precision=precision,
106
+ compute_units=ct.ComputeUnit.CPU_ONLY,
107
+ inputs=[
108
+ ct.TensorType(
109
+ name=name,
110
+ shape=tuple(value.shape),
111
+ dtype=np.float32 if name in ("text_states", "relation_states", "pair_mask") else np.int32,
112
+ )
113
+ for name, value in zip(INPUT_NAMES, arguments)
114
+ ],
115
+ outputs=[ct.TensorType(name="relation_logits", dtype=np.float32)],
116
+ )
117
+ converted.short_description = "GLiNER2.5 multilingual trained sparse relation scoring head"
118
+ converted.author = "Fastino (original); Fluid Inference (Core ML conversion)"
119
+ converted.license = "Apache-2.0"
120
+ converted.user_defined_metadata.update(
121
+ {
122
+ "source_model": MODEL_ID,
123
+ "source_revision": MODEL_REVISION,
124
+ "stage": "trained relation scorer",
125
+ "word_capacity": str(args.max_words),
126
+ "relation_capacity": str(args.max_relations),
127
+ "pair_capacity": str(pair_cap),
128
+ }
129
+ )
130
+ out = Path(args.output_dir)
131
+ out.mkdir(parents=True, exist_ok=True)
132
+ suffix = f"{args.precision}_W{args.max_words}_R{args.max_relations}_P{pair_cap}"
133
+ package = out / f"gliner2_multi_relation_{suffix}.mlpackage"
134
+ converted.save(str(package))
135
+ model = ct.models.MLModel(str(package), compute_units=ct.ComputeUnit.CPU_ONLY)
136
+ prediction = model.predict(
137
+ {
138
+ name: value.numpy().astype(
139
+ np.float32 if name in ("text_states", "relation_states", "pair_mask") else np.int32
140
+ )
141
+ for name, value in zip(INPUT_NAMES, arguments)
142
+ }
143
+ )["relation_logits"]
144
+ runtime_error = float(np.max(np.abs(prediction[: len(pairs)] - reference.numpy()[: len(pairs)])))
145
+ if not np.isfinite(runtime_error):
146
+ raise RuntimeError("Relation scorer produced non-finite values")
147
+ report = {
148
+ "source_model": MODEL_ID,
149
+ "source_revision": MODEL_REVISION,
150
+ "precision": args.precision,
151
+ "fixture": text,
152
+ "valid_pairs": int(pairs.pair_mask.sum()),
153
+ "wrapper_max_absolute_error": wrapper_error,
154
+ "coreml_max_absolute_error": runtime_error,
155
+ "package": str(package),
156
+ "package_bytes": sum(file.stat().st_size for file in package.rglob("*") if file.is_file()),
157
+ "coremltools": ct.__version__,
158
+ "torch": torch.__version__,
159
+ }
160
+ (out / f"relation-{suffix}.json").write_text(json.dumps(report, indent=2) + "\n")
161
+ print(json.dumps(report, indent=2))
162
+
163
+
164
+ if __name__ == "__main__":
165
+ main()
convert_extraction_names.py ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Ordered tensor contract shared by extraction conversion and verification."""
2
+
3
+ FEATURE_NAMES = (
4
+ "text_states",
5
+ "query_states",
6
+ "boundary_states",
7
+ "boundary_mask",
8
+ "start_logits",
9
+ "end_logits",
10
+ "inside_prefix",
11
+ "inside_prefix_mean",
12
+ "pool_start_projection",
13
+ "pool_end_projection",
14
+ "null_logits",
15
+ "count_log_rates",
16
+ "classification_logits",
17
+ )
18
+ SCORE_INPUT_NAMES = (
19
+ "text_states",
20
+ "text_mask",
21
+ "query_states",
22
+ "query_mask",
23
+ "boundary_states",
24
+ "start_logits",
25
+ "end_logits",
26
+ "inside_prefix",
27
+ "inside_prefix_mean",
28
+ "candidate_indices",
29
+ "candidate_mask",
30
+ "candidate_compatibility",
31
+ )
extraction-assets.lock.json ADDED
@@ -0,0 +1,149 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "source_model": "fastino/gliner2.5-multi-v1",
3
+ "source_revision": "a221b77a8baf4a613b8f8652661d41fa10a5641e",
4
+ "package_files": [
5
+ {
6
+ "package": "gliner2_multi_explicit_fp32_W64_Q8_S64.mlpackage",
7
+ "bytes": 1152477,
8
+ "files": [
9
+ {
10
+ "path": "gliner2_multi_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/model.mlmodel",
11
+ "bytes": 48820,
12
+ "sha256": "a0a9bfed957616b1d1f9c3ef4794d0cce69de82e364e36e52bb0f5ef078377c7"
13
+ },
14
+ {
15
+ "path": "gliner2_multi_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/weights/weight.bin",
16
+ "bytes": 1103040,
17
+ "sha256": "c2c82c889b6e214f7614a40419cfd7b05d7eeef8ef10c4bef2f950942e4f651b"
18
+ },
19
+ {
20
+ "path": "gliner2_multi_explicit_fp32_W64_Q8_S64.mlpackage/Manifest.json",
21
+ "bytes": 617,
22
+ "sha256": "27ad8fcb013bcabf501620750f4e626fe128007a7d761fe24ce2ca8232330caa"
23
+ }
24
+ ]
25
+ },
26
+ {
27
+ "package": "gliner2_multi_extraction_features_fp32_L128_W64_Q8.mlpackage",
28
+ "bytes": 1155993314,
29
+ "files": [
30
+ {
31
+ "path": "gliner2_multi_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel",
32
+ "bytes": 1012405,
33
+ "sha256": "232fe09e50b91fef67f31015a0e90124e95f1d7084c448d99550d18741d2a81b"
34
+ },
35
+ {
36
+ "path": "gliner2_multi_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin",
37
+ "bytes": 1154980292,
38
+ "sha256": "4a4d2f7aa1395ce3aa72103dbdd295c6d4c3d8f0276d831df7a48c47cce57048"
39
+ },
40
+ {
41
+ "path": "gliner2_multi_extraction_features_fp32_L128_W64_Q8.mlpackage/Manifest.json",
42
+ "bytes": 617,
43
+ "sha256": "2d1774aef4f6279bba281f3f5a1d641330dfbda44a0684eaf08bdd598ccc8af0"
44
+ }
45
+ ]
46
+ },
47
+ {
48
+ "package": "gliner2_multi_extraction_scorer_fp32_L128_W64_Q8.mlpackage",
49
+ "bytes": 1750640,
50
+ "files": [
51
+ {
52
+ "path": "gliner2_multi_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel",
53
+ "bytes": 32583,
54
+ "sha256": "70dd5aa5aee2634c4ceff411dc71d079143ff687679ac763503fb1950c1ff3e7"
55
+ },
56
+ {
57
+ "path": "gliner2_multi_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin",
58
+ "bytes": 1717440,
59
+ "sha256": "10a9fa54e3dbf2550a4636ae2275074c0f68770e60c351df1e3dcfddec6b9dc5"
60
+ },
61
+ {
62
+ "path": "gliner2_multi_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Manifest.json",
63
+ "bytes": 617,
64
+ "sha256": "1290c77b1d40c5d035e4f9b78325c65c175c9ab38672d8acec907f5ad2a7b171"
65
+ }
66
+ ]
67
+ },
68
+ {
69
+ "package": "gliner2_multi_record_anchorless_fp32_F8_C192_I1536.mlpackage",
70
+ "bytes": 2876852,
71
+ "files": [
72
+ {
73
+ "path": "gliner2_multi_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel",
74
+ "bytes": 5003,
75
+ "sha256": "5f32406aebd0d9a907aafd0a2f83863b95a4fdeeaeeedb4ad003aa4d02b7074e"
76
+ },
77
+ {
78
+ "path": "gliner2_multi_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin",
79
+ "bytes": 2871232,
80
+ "sha256": "a2620c168f1638982cd87243c172682dd4fc9ddd9e32f52bceb252b74a6452e1"
81
+ },
82
+ {
83
+ "path": "gliner2_multi_record_anchorless_fp32_F8_C192_I1536.mlpackage/Manifest.json",
84
+ "bytes": 617,
85
+ "sha256": "027e73edffa88e923254044698b103582b06afe1423e6a89ed85a88a90446717"
86
+ }
87
+ ]
88
+ },
89
+ {
90
+ "package": "gliner2_multi_record_assignment_fp32_F8_C192_I1536.mlpackage",
91
+ "bytes": 1196568,
92
+ "files": [
93
+ {
94
+ "path": "gliner2_multi_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel",
95
+ "bytes": 7471,
96
+ "sha256": "9b4d565ca5529e183c51254a16fa2f78be829bc01cb816f5010d158faf09ca85"
97
+ },
98
+ {
99
+ "path": "gliner2_multi_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin",
100
+ "bytes": 1188480,
101
+ "sha256": "6498691d20934d4d65b345bf6c55000e6f72e0bc124493248a822150452bfa6e"
102
+ },
103
+ {
104
+ "path": "gliner2_multi_record_assignment_fp32_F8_C192_I1536.mlpackage/Manifest.json",
105
+ "bytes": 617,
106
+ "sha256": "126f3aa04f8abf7be8aaf1cabeb7d5401fe3f68bf152f2448e4745142350595a"
107
+ }
108
+ ]
109
+ },
110
+ {
111
+ "package": "gliner2_multi_relation_fp32_W64_R4_P256.mlpackage",
112
+ "bytes": 23661041,
113
+ "files": [
114
+ {
115
+ "path": "gliner2_multi_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/model.mlmodel",
116
+ "bytes": 29832,
117
+ "sha256": "18c143798c805f0b00162b069059a1e1af155c2de331d57d5300edb6db845ece"
118
+ },
119
+ {
120
+ "path": "gliner2_multi_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/weights/weight.bin",
121
+ "bytes": 23630592,
122
+ "sha256": "d9cf4fce04c725db699f4ab54dc21335fa0388501f055ca454a5a34ecf0e8955"
123
+ },
124
+ {
125
+ "path": "gliner2_multi_relation_fp32_W64_R4_P256.mlpackage/Manifest.json",
126
+ "bytes": 617,
127
+ "sha256": "ef4b4e152183a41dfab174b1cd5d4471025222e92dd939790cc6211590ed509f"
128
+ }
129
+ ]
130
+ }
131
+ ],
132
+ "runtime_files": [
133
+ {
134
+ "path": "config.json",
135
+ "bytes": 3151,
136
+ "sha256": "8b59a0f426a65859c89cd1ea850c3529c09aa3be3a6fafd8eddfdd17b1bf0146"
137
+ },
138
+ {
139
+ "path": "tokenizer/tokenizer_config.json",
140
+ "bytes": 645,
141
+ "sha256": "0bf3ea0873234bd9bfdd3853c440395009ac6365a925b91654daed5396d655e1"
142
+ },
143
+ {
144
+ "path": "tokenizer/tokenizer.json",
145
+ "bytes": 16035853,
146
+ "sha256": "c62446df87ae18ec98b133f8f84fc449a07cc89bbf8ef192a4cb5f9c53777a7a"
147
+ }
148
+ ]
149
+ }
extraction_export.py ADDED
@@ -0,0 +1,408 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Core ML graph wrappers for GLiNER2.5's trained boundary extraction path."""
2
+
3
+ import math
4
+ from contextlib import contextmanager
5
+
6
+ import torch
7
+ from coremltools.converters.mil import Builder as mb
8
+ from coremltools.converters.mil.frontend.torch.ops import _get_inputs
9
+ from coremltools.converters.mil.frontend.torch.torch_op_registry import register_torch_op
10
+ from coremltools.converters.mil.mil import types
11
+ from gliner2.models.boundary import encoding, heads
12
+ from gliner2.models.boundary.pool import PooledCandidates
13
+ from gliner2.models.boundary.proposal import BoundaryProposals
14
+ from transformers.models.deberta_v2 import modeling_deberta_v2
15
+
16
+ from export_model import coreml_safe_attention_forward
17
+
18
+
19
+ @register_torch_op(override=True)
20
+ def clamp_min(context, node):
21
+ """Preserve the tensor dtype when TorchScript supplied a Python scalar."""
22
+ x, y = _get_inputs(context, node, expected=2)
23
+ if x.dtype != y.dtype:
24
+ y = mb.cast(x=y, dtype=types.builtin_to_string(x.dtype))
25
+ context.add(mb.maximum(x=x, y=y, name=node.name))
26
+
27
+
28
+ @register_torch_op(torch_alias=["clip"], override=True)
29
+ def clamp(context, node):
30
+ """Avoid promoting integer span indices to float for an absent bound."""
31
+ inputs = _get_inputs(context, node, expected=[1, 2, 3])
32
+ x = inputs[0]
33
+ lower = inputs[1] if len(inputs) > 1 and inputs[1] is not None else None
34
+ upper = inputs[2] if len(inputs) > 2 and inputs[2] is not None else None
35
+ result = x
36
+ for bound, op in ((upper, mb.minimum), (lower, mb.maximum)):
37
+ if bound is None:
38
+ continue
39
+ if bound.dtype != x.dtype:
40
+ bound = mb.cast(x=bound, dtype=types.builtin_to_string(x.dtype))
41
+ result = op(x=result, y=bound)
42
+ context.add(mb.identity(x=result, name=node.name))
43
+
44
+
45
+ def shift_left(text_states, bos_state):
46
+ """Functional equivalent of the upstream in-place BOS placement."""
47
+ bos = bos_state.to(text_states.dtype).view(1, 1, -1)
48
+ return torch.cat((bos.expand(text_states.shape[0], 1, -1), text_states), 1)
49
+
50
+
51
+ def shift_right(text_states, text_lengths, eos_state):
52
+ """Functional equivalent of the upstream in-place EOS placement."""
53
+ batch, length, hidden = text_states.shape
54
+ eos = eos_state.to(text_states.dtype).view(1, 1, hidden)
55
+ right = torch.cat((text_states, eos.expand(batch, 1, hidden)), 1)
56
+ positions = torch.arange(length + 1, device=text_states.device).view(1, length + 1, 1)
57
+ return torch.where(
58
+ positions == text_lengths.view(batch, 1, 1),
59
+ eos.expand(batch, length + 1, hidden),
60
+ right,
61
+ )
62
+
63
+
64
+ def safe_boundary_attention(self, states, mask):
65
+ """Explicit scaled attention with the same finite masked result as upstream."""
66
+ batch, length, dim = states.shape
67
+ qkv = self.qkv_projection(self.norm(states)).view(batch, length, 3, self.num_heads, self.head_dim)
68
+ query, key, value = qkv.permute(2, 0, 3, 1, 4)
69
+ allowed = mask.view(batch, 1, 1, length).expand(batch, 1, length, length)
70
+ if self.window > 0:
71
+ positions = torch.arange(length, device=states.device)
72
+ local = (positions.view(length, 1) - positions.view(1, length)).abs() <= self.window
73
+ allowed = allowed & local.view(1, 1, length, length)
74
+ diagonal = torch.eye(length, dtype=torch.bool, device=states.device).view(1, 1, length, length)
75
+ allowed = (allowed.float() + diagonal.float()) > 0.5
76
+ scores = torch.matmul(query, key.transpose(-1, -2)) / math.sqrt(self.head_dim)
77
+ scores = scores.masked_fill(~allowed, -1e4)
78
+ attended = torch.matmul(torch.softmax(scores, dim=-1), value)
79
+ attended = attended.transpose(1, 2).reshape(batch, length, dim)
80
+ return (states + self.dropout(self.output_projection(attended))) * mask.unsqueeze(-1).to(states.dtype)
81
+
82
+
83
+ def safe_query_head(self, boundary, boundary_mask, text, text_mask, query, query_mask):
84
+ """Upstream marginals with a dtype-safe count clamp for Core ML."""
85
+ scale = 1.0 / math.sqrt(self.boundary_dim)
86
+ start = (
87
+ torch.einsum(
88
+ "bld,bqd->bql",
89
+ self.dropout(self.start_boundary_projection(boundary)),
90
+ self.start_query_projection(query),
91
+ )
92
+ * scale
93
+ )
94
+ end = (
95
+ torch.einsum(
96
+ "bld,bqd->bql",
97
+ self.dropout(self.end_boundary_projection(boundary)),
98
+ self.end_query_projection(query),
99
+ )
100
+ * scale
101
+ )
102
+ inside = (
103
+ torch.einsum(
104
+ "bld,bqd->bql",
105
+ self.dropout(self.inside_text_projection(text)),
106
+ self.inside_query_projection(query),
107
+ )
108
+ * scale
109
+ )
110
+ boundary_keep = boundary_mask.unsqueeze(1) & query_mask.unsqueeze(-1)
111
+ text_keep = text_mask.unsqueeze(1) & query_mask.unsqueeze(-1)
112
+ start = heads._masked_fill_min(start, boundary_keep)
113
+ end = heads._masked_fill_min(end, boundary_keep)
114
+ inside = heads._masked_fill_min(inside, text_keep)
115
+ inside_for_prefix = inside.masked_fill(~text_keep, 0.0).float()
116
+ count = torch.clamp(text_keep.sum(-1, keepdim=True).float(), min=1.0)
117
+ mean = (inside_for_prefix.sum(-1, keepdim=True) / count).detach()
118
+ centered = (inside_for_prefix - mean) * text_keep.to(inside_for_prefix.dtype)
119
+ zeros = torch.zeros(centered.shape[0], centered.shape[1], 1, dtype=torch.float32, device=text.device)
120
+ prefix = torch.cat((zeros, centered.cumsum(dim=-1)), dim=-1)
121
+ return heads.BoundaryMarginals(start, end, inside, prefix, mean)
122
+
123
+
124
+ @contextmanager
125
+ def coreml_trace_patches():
126
+ """Apply and restore mathematically equivalent trace-safe operations."""
127
+ saved = (
128
+ modeling_deberta_v2.scaled_size_sqrt,
129
+ modeling_deberta_v2.build_rpos,
130
+ modeling_deberta_v2.DisentangledSelfAttention.forward,
131
+ encoding.shift_left_with_bos,
132
+ encoding.shift_right_with_eos,
133
+ encoding.BoundaryAttentionBlock.forward,
134
+ heads.BoundaryQueryHead.forward,
135
+ )
136
+
137
+ def static_scale(query_layer, scale_factor):
138
+ value = math.sqrt(float(query_layer.shape[-1] * scale_factor))
139
+ return torch.tensor(value, dtype=torch.float32, device=query_layer.device)
140
+
141
+ modeling_deberta_v2.scaled_size_sqrt = static_scale
142
+ modeling_deberta_v2.build_rpos = lambda query, key, relative_pos, buckets, max_pos: relative_pos
143
+ modeling_deberta_v2.DisentangledSelfAttention.forward = coreml_safe_attention_forward
144
+ encoding.shift_left_with_bos = shift_left
145
+ encoding.shift_right_with_eos = shift_right
146
+ encoding.BoundaryAttentionBlock.forward = safe_boundary_attention
147
+ heads.BoundaryQueryHead.forward = safe_query_head
148
+ try:
149
+ yield
150
+ finally:
151
+ (
152
+ modeling_deberta_v2.scaled_size_sqrt,
153
+ modeling_deberta_v2.build_rpos,
154
+ modeling_deberta_v2.DisentangledSelfAttention.forward,
155
+ encoding.shift_left_with_bos,
156
+ encoding.shift_right_with_eos,
157
+ encoding.BoundaryAttentionBlock.forward,
158
+ heads.BoundaryQueryHead.forward,
159
+ ) = saved
160
+
161
+
162
+ class ExtractionFeaturesExport(torch.nn.Module):
163
+ """Trained encoder, boundary marginals, pool projections and null/count heads."""
164
+
165
+ def __init__(self, native):
166
+ super().__init__()
167
+ self.encoder = native.encoder
168
+ self.head = native.boundary_head
169
+ self.classifier = native.classifier
170
+
171
+ def forward(
172
+ self,
173
+ input_ids,
174
+ attention_mask,
175
+ text_indices,
176
+ text_mask,
177
+ query_indices,
178
+ query_mask,
179
+ cls_indices,
180
+ cls_mask,
181
+ ):
182
+ hidden = self.encoder(input_ids=input_ids.long(), attention_mask=attention_mask.long()).last_hidden_state
183
+ text_idx = text_indices.long().unsqueeze(-1).expand(-1, -1, hidden.shape[-1])
184
+ query_idx = query_indices.long().unsqueeze(-1).expand(-1, -1, hidden.shape[-1])
185
+ text = hidden.gather(1, text_idx) * text_mask.unsqueeze(-1)
186
+ query = hidden.gather(1, query_idx) * query_mask.unsqueeze(-1)
187
+ cls_idx = cls_indices.long().unsqueeze(-1).expand(-1, -1, hidden.shape[-1])
188
+ classification_states = hidden.gather(1, cls_idx)
189
+ classification_logits = self.classifier(classification_states).squeeze(-1)
190
+ classification_logits = torch.where(
191
+ cls_mask > 0.5, classification_logits, torch.full_like(classification_logits, -1e4)
192
+ )
193
+ tm, qm = text_mask > 0.5, query_mask > 0.5
194
+ encoded = self.head.boundary_encoder(text, tm)
195
+ marginal = self.head.boundary_query_head(encoded.states, encoded.mask, text, tm, query, qm)
196
+ return (
197
+ text,
198
+ query,
199
+ encoded.states,
200
+ encoded.mask.float(),
201
+ marginal.start_logits,
202
+ marginal.end_logits,
203
+ marginal.inside_prefix,
204
+ marginal.inside_prefix_mean,
205
+ self.head.shared_pool_builder.start_projection(encoded.states),
206
+ self.head.shared_pool_builder.end_projection(encoded.states),
207
+ self.head.null_projection(query).squeeze(-1),
208
+ self.head.count_head(query).squeeze(-1),
209
+ classification_logits,
210
+ )
211
+
212
+
213
+ class ExtractionScoreExport(torch.nn.Module):
214
+ """Trained shared-pool reranker and record candidate state projection."""
215
+
216
+ def __init__(self, native):
217
+ super().__init__()
218
+ self.scorer = native.boundary_head.shared_pool_scorer
219
+ self.candidate_encoder = native.boundary_head.candidate_encoder
220
+
221
+ def forward(
222
+ self,
223
+ text,
224
+ text_mask,
225
+ query,
226
+ query_mask,
227
+ boundary,
228
+ starts,
229
+ ends,
230
+ inside,
231
+ inside_mean,
232
+ indices,
233
+ pool_mask,
234
+ compatibility,
235
+ ):
236
+ tm, qm = text_mask > 0.5, query_mask > 0.5
237
+ pooled = PooledCandidates(indices.long(), pool_mask > 0.5, None, None, compatibility)
238
+ score, _ = self.scorer(
239
+ boundary,
240
+ query,
241
+ qm,
242
+ pooled,
243
+ starts,
244
+ ends,
245
+ inside,
246
+ tm.sum(-1).long(),
247
+ text,
248
+ tm,
249
+ inside_prefix_mean=inside_mean,
250
+ )
251
+ index = indices.long()
252
+ start_states = boundary.gather(1, index[..., 0].unsqueeze(-1).expand(-1, -1, boundary.shape[-1]))
253
+ end_states = boundary.gather(1, index[..., 1].unsqueeze(-1).expand(-1, -1, boundary.shape[-1]))
254
+ candidate_states = self.candidate_encoder(torch.cat((start_states, end_states), -1))
255
+ candidate_states = candidate_states * pool_mask.unsqueeze(-1)
256
+ return score.transpose(1, 2), candidate_states
257
+
258
+
259
+ class ExtractionRelationExport(torch.nn.Module):
260
+ """The trained sparse relation scorer with tensor-only pair routing."""
261
+
262
+ def __init__(self, native):
263
+ super().__init__()
264
+ self.scorer = native.relation_scorer
265
+
266
+ def forward(
267
+ self,
268
+ text,
269
+ text_length,
270
+ relation,
271
+ batch_index,
272
+ relation_index,
273
+ head_start,
274
+ head_end,
275
+ tail_start,
276
+ tail_end,
277
+ pair_mask,
278
+ ):
279
+ scorer = self.scorer
280
+ length = text.shape[1]
281
+ batch_valid = (batch_index >= 0) & (batch_index < text.shape[0])
282
+ relation_valid = (relation_index >= 0) & (relation_index < relation.shape[1])
283
+ valid = batch_valid & relation_valid & (pair_mask > 0.5)
284
+ batch = batch_index.long().clamp(0, text.shape[0] - 1)
285
+ rel_index = relation_index.long().clamp(0, relation.shape[1] - 1)
286
+
287
+ def gather(position):
288
+ return text[batch, position.long().clamp(0, length - 1)]
289
+
290
+ h_start = gather(head_start)
291
+ h_end = gather(head_end - 1)
292
+ t_start = gather(tail_start)
293
+ t_end = gather(tail_end - 1)
294
+ rel = relation[batch, rel_index]
295
+ delta = (tail_start - head_start).to(text.dtype)
296
+ order = torch.sign(delta).unsqueeze(-1)
297
+ distance = (delta.abs() / text_length.float().clamp_min(1.0)).unsqueeze(-1)
298
+ features = torch.cat((h_start, h_end, t_start, t_end, rel, order, distance), -1)
299
+ score = scorer.mlp(features).squeeze(-1)
300
+ if scorer.use_biaffine_content:
301
+ prefix = torch.cat(
302
+ (text.new_zeros(text.shape[0], 1, scorer.hidden_size), text.float().cumsum(1).to(text.dtype)),
303
+ dim=1,
304
+ )
305
+
306
+ def pool(start, end):
307
+ total = prefix[batch, end.long().clamp(0, length)] - prefix[batch, start.long().clamp(0, length)]
308
+ width = (end - start).clamp_min(1).unsqueeze(-1).to(total.dtype)
309
+ return total / width
310
+
311
+ head_content = scorer.head_content_projection(pool(head_start, head_end))
312
+ tail_content = scorer.tail_content_projection(pool(tail_start, tail_end))
313
+ gate = torch.sigmoid(scorer.relation_content_gate(rel))
314
+ biaffine = (head_content * gate * tail_content).sum(-1) / (scorer.hidden_size**0.5)
315
+ linear = scorer.content_linear(torch.cat((head_content, tail_content, rel), -1)).squeeze(-1)
316
+ score = score + biaffine + linear
317
+ return score.masked_fill(~valid, 0.0)
318
+
319
+
320
+ class ExtractionRecordAssignmentExport(torch.nn.Module):
321
+ """All trained natural/latent/anchorless object and field assignment layers."""
322
+
323
+ def __init__(self, native):
324
+ super().__init__()
325
+ self.head = native.record_decoder
326
+
327
+ def forward(self, instances, field_queries, field_candidates):
328
+ head = self.head
329
+ instance_projection = head.inst_proj(instances)
330
+ field_projection = head.field_proj(field_queries)
331
+ query = instance_projection.unsqueeze(1) + field_projection.unsqueeze(0)
332
+ null_scores = torch.einsum("ifd,d->if", query, head.null_embed)
333
+ candidate_scores = torch.einsum("ifd,fcd->ifc", query, head.cand_proj(field_candidates))
334
+ assignment = torch.cat((null_scores.unsqueeze(-1), candidate_scores), -1)
335
+ object_scores = head.object_head(instances).squeeze(-1)
336
+ latent_scores = head.latent_seed_head(instances).squeeze(-1)
337
+ return assignment, object_scores, latent_scores
338
+
339
+
340
+ class ExtractionRecordAnchorlessExport(torch.nn.Module):
341
+ """Trained learned-instance and contextual attention path for records."""
342
+
343
+ def __init__(self, native):
344
+ super().__init__()
345
+ self.head = native.record_decoder
346
+
347
+ def forward(self, context_states, context_mask):
348
+ head = self.head
349
+ instances = head.instance_embed
350
+ query = head.q_proj(instances)
351
+ key = head.k_proj(context_states)
352
+ value = head.v_proj(context_states)
353
+ attention = torch.matmul(query, key.transpose(-1, -2)) / math.sqrt(head.record_dim)
354
+ attention = attention.masked_fill(context_mask.unsqueeze(0) < 0.5, -1e4)
355
+ pooled = torch.matmul(torch.softmax(attention, -1), value)
356
+ return instances + pooled * (context_mask.sum() > 0).to(pooled.dtype)
357
+
358
+
359
+ class ExtractionExplicitSpanExport(torch.nn.Module):
360
+ """Trained proposal prior and reranker for forced attribute/enum spans."""
361
+
362
+ def __init__(self, native):
363
+ super().__init__()
364
+ self.proposer = native.boundary_head.boundary_proposer
365
+ self.scorer = native.boundary_head.pair_scorer
366
+
367
+ def forward(
368
+ self,
369
+ text,
370
+ text_mask,
371
+ query,
372
+ query_mask,
373
+ boundary,
374
+ starts,
375
+ ends,
376
+ inside,
377
+ inside_mean,
378
+ indices,
379
+ valid_mask,
380
+ ):
381
+ tm, qm = text_mask > 0.5, query_mask > 0.5
382
+ idx = indices.long()
383
+ legal = (
384
+ (idx[..., 0] >= 0)
385
+ & (idx[..., 1] > idx[..., 0])
386
+ & (idx[..., 1] <= tm.sum(-1).view(-1, 1, 1))
387
+ & qm.unsqueeze(-1)
388
+ & (valid_mask > 0.5)
389
+ )
390
+ compatibility = self.proposer.score_explicit_pairs(boundary, query, idx, legal)
391
+ proposals = BoundaryProposals(
392
+ indices=idx,
393
+ logits=None,
394
+ valid_mask=legal,
395
+ compat_logits=compatibility,
396
+ )
397
+ return self.scorer(
398
+ boundary,
399
+ query,
400
+ proposals,
401
+ starts,
402
+ ends,
403
+ inside,
404
+ tm.sum(-1).long(),
405
+ text,
406
+ tm,
407
+ inside_prefix_mean=inside_mean,
408
+ )
extraction_pool.py ADDED
@@ -0,0 +1,117 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Weight-free GLiNER2 boundary candidate selection between Core ML stages.
2
+
3
+ The learned start/end projections are outputs of the first Core ML stage. This
4
+ module preserves GLiNER2 2.0.0's stable ranking and deduplication on the host.
5
+ """
6
+
7
+ import math
8
+
9
+ import torch
10
+ from gliner2.models.boundary.constants import MASK_LOGIT
11
+ from gliner2.models.boundary.indexing import gather_rows
12
+ from gliner2.models.boundary.pool import PooledCandidates, _deduplicate_pool
13
+ from gliner2.models.boundary.proposal import select_top_boundaries
14
+
15
+
16
+ def select_candidates(
17
+ start_projection: torch.Tensor,
18
+ end_projection: torch.Tensor,
19
+ boundary_mask: torch.Tensor,
20
+ query_mask: torch.Tensor,
21
+ start_logits: torch.Tensor,
22
+ end_logits: torch.Tensor,
23
+ *,
24
+ boundary_top_k: int,
25
+ pool_size: int,
26
+ min_pool_per_query: int,
27
+ ) -> PooledCandidates:
28
+ """Select the native shared pool using already projected Core ML states."""
29
+ batch, n_boundaries, dim = start_projection.shape
30
+ n_queries = query_mask.shape[1]
31
+ if end_projection.shape != start_projection.shape:
32
+ raise ValueError("Start and end projections must have the same shape")
33
+ if start_logits.shape != (batch, n_queries, n_boundaries):
34
+ raise ValueError("Start logits do not match boundary and query dimensions")
35
+ if end_logits.shape != start_logits.shape:
36
+ raise ValueError("End logits do not match start logits")
37
+ if boundary_mask.shape != (batch, n_boundaries) or query_mask.shape != (batch, n_queries):
38
+ raise ValueError("Boundary or query mask has an unexpected shape")
39
+
40
+ floor = torch.full_like(start_logits, MASK_LOGIT)
41
+ valid_boundary = boundary_mask.unsqueeze(1) & query_mask.unsqueeze(-1)
42
+ union_start = torch.where(valid_boundary, start_logits, floor).amax(1)
43
+ union_end = torch.where(valid_boundary, end_logits, floor).amax(1)
44
+ union_valid = boundary_mask & query_mask.any(-1, keepdim=True)
45
+ _, starts, starts_valid = select_top_boundaries(union_start.unsqueeze(1), union_valid.unsqueeze(1), boundary_top_k)
46
+ _, ends, ends_valid = select_top_boundaries(union_end.unsqueeze(1), union_valid.unsqueeze(1), boundary_top_k)
47
+ starts, ends = starts[:, 0], ends[:, 0]
48
+ starts_valid, ends_valid = starts_valid[:, 0], ends_valid[:, 0]
49
+ n_starts, n_ends = starts.shape[1], ends.shape[1]
50
+ pair_start = starts.unsqueeze(-1).expand(batch, n_starts, n_ends).reshape(batch, -1)
51
+ pair_end = ends.unsqueeze(1).expand(batch, n_starts, n_ends).reshape(batch, -1)
52
+ pair_valid = (
53
+ starts_valid.unsqueeze(-1) & ends_valid.unsqueeze(1) & (ends.unsqueeze(1) > starts.unsqueeze(-1))
54
+ ).reshape(batch, -1)
55
+
56
+ selected_start = gather_rows(start_projection, pair_start)
57
+ selected_end = gather_rows(end_projection, pair_end)
58
+ compatibility = (selected_start * selected_end).sum(-1) / math.sqrt(dim)
59
+ union_pair_score = (
60
+ compatibility
61
+ + union_start.gather(1, pair_start.clamp(0, n_boundaries - 1))
62
+ + union_end.gather(1, pair_end.clamp(0, n_boundaries - 1))
63
+ )
64
+
65
+ quota = min(min_pool_per_query, pair_start.shape[-1])
66
+ quota_keys = pair_start.new_zeros((batch, 0))
67
+ quota_scores = union_pair_score.new_zeros((batch, 0))
68
+ quota_valid = pair_valid.new_zeros((batch, 0))
69
+ if quota:
70
+ start_idx = pair_start.clamp(0, start_logits.shape[2] - 1).unsqueeze(1).expand(batch, n_queries, -1)
71
+ end_idx = pair_end.clamp(0, end_logits.shape[2] - 1).unsqueeze(1).expand(batch, n_queries, -1)
72
+ per_query = start_logits.gather(2, start_idx) + end_logits.gather(2, end_idx) + compatibility.unsqueeze(1)
73
+ per_query_valid = pair_valid.unsqueeze(1) & query_mask.unsqueeze(-1)
74
+ ranked = torch.argsort(
75
+ per_query.masked_fill(~per_query_valid, MASK_LOGIT),
76
+ dim=-1,
77
+ descending=True,
78
+ stable=True,
79
+ )[..., :quota]
80
+ quota_start = start_idx.gather(-1, ranked)
81
+ quota_end = end_idx.gather(-1, ranked)
82
+ quota_valid = per_query_valid.gather(-1, ranked).reshape(batch, -1)
83
+ quota_keys = (quota_start * n_boundaries + quota_end).reshape(batch, -1)
84
+ rank_bonus = torch.arange(quota, 0, -1, device=start_projection.device, dtype=union_pair_score.dtype)
85
+ quota_scores = (
86
+ union_pair_score.new_full((batch, n_queries, quota), -MASK_LOGIT * 0.5) + rank_bonus.view(1, 1, quota)
87
+ ).reshape(batch, -1)
88
+
89
+ global_keys = pair_start * n_boundaries + pair_end
90
+ all_keys = torch.cat((quota_keys, global_keys), -1)
91
+ all_scores = torch.cat((quota_scores, union_pair_score.detach()), -1)
92
+ all_valid = torch.cat((quota_valid, pair_valid), -1)
93
+ selected_keys, selected_valid = _deduplicate_pool(all_keys, all_scores, all_valid, pool_size, n_boundaries)
94
+ selected_keys = torch.where(selected_valid, selected_keys, torch.zeros_like(selected_keys))
95
+ selected_s = torch.div(selected_keys, n_boundaries, rounding_mode="floor")
96
+ selected_e = selected_keys - selected_s * n_boundaries
97
+ indices = torch.stack((selected_s, selected_e), -1)
98
+ indices = torch.where(selected_valid.unsqueeze(-1), indices, torch.zeros_like(indices))
99
+
100
+ gathered_start = gather_rows(start_projection, selected_s)
101
+ gathered_end = gather_rows(end_projection, selected_e)
102
+ selected_compat = (gathered_start * gathered_end).sum(-1) / math.sqrt(dim)
103
+ selected_score = (
104
+ selected_compat
105
+ + union_start.gather(1, selected_s.clamp(0, n_boundaries - 1))
106
+ + union_end.gather(1, selected_e.clamp(0, n_boundaries - 1))
107
+ )
108
+ selected_score = selected_score.masked_fill(~selected_valid, MASK_LOGIT)
109
+ selected_compat = torch.where(selected_valid, selected_compat, torch.zeros_like(selected_compat))
110
+ return PooledCandidates(
111
+ indices=indices,
112
+ mask=selected_valid,
113
+ proposal_logits=selected_score,
114
+ gold_mask=None,
115
+ compat_logits=selected_compat,
116
+ stats=None,
117
+ )
extraction_runtime.py ADDED
@@ -0,0 +1,525 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """GLiNER2.5 multilingual extraction runtime using only Core ML trained weights."""
2
+
3
+ import json
4
+ from contextvars import ContextVar
5
+ from pathlib import Path
6
+
7
+ import coremltools as ct
8
+ import numpy as np
9
+ import torch
10
+ from gliner2.configuration import BoundaryHeadSettings
11
+ from gliner2.models.boundary.engine import BoundaryExtractor
12
+ from gliner2.models.boundary.records import RecordGroupOutput
13
+ from gliner2.models.boundary.relations import (
14
+ RelationProposalSettings,
15
+ RelationTypeSpec,
16
+ TypedRelationPairGenerator,
17
+ )
18
+ from gliner2.models.outputs import CandidateTensorBatch, ExtractorOutput
19
+
20
+ from convert_extraction_names import FEATURE_NAMES, SCORE_INPUT_NAMES
21
+ from extraction_pool import select_candidates
22
+ from preprocessing import load_processor, prepare_extraction
23
+
24
+ MODEL_PREFIX = "gliner2_multi"
25
+
26
+
27
+ def to_tensor(value):
28
+ return torch.from_numpy(np.asarray(value).copy())
29
+
30
+
31
+ def padded(value, size):
32
+ if value.shape[0] > size:
33
+ raise ValueError(f"Request exceeds Core ML bucket capacity {size}")
34
+ result = value.new_zeros((size, *value.shape[1:]))
35
+ result[: value.shape[0]] = value
36
+ return result
37
+
38
+
39
+ class CoreMLBoundaryHead(torch.nn.Module):
40
+ """Provide native decoder tensors from the converted extraction stages."""
41
+
42
+ def __init__(self, context: ContextVar, explicit_model, max_queries: int, max_spans: int):
43
+ super().__init__()
44
+ self.context = context
45
+ self.explicit_model = explicit_model
46
+ self.max_queries = max_queries
47
+ self.max_spans = max_spans
48
+
49
+ def forward(self, text, text_mask, query, query_mask, return_candidates=True):
50
+ state = self.context.get()
51
+ features = state["features"]
52
+ full = state["candidates"]
53
+ count = query.shape[1]
54
+ selected = CandidateTensorBatch(
55
+ indices=full.indices[:, :count],
56
+ proposal_logits=full.proposal_logits[:, :count],
57
+ pair_logits=full.pair_logits[:, :count],
58
+ valid_mask=full.valid_mask[:, :count],
59
+ query_mask=full.query_mask[:, :count],
60
+ candidate_states=full.candidate_states[:, :count],
61
+ )
62
+ return ExtractorOutput(
63
+ candidates=selected if return_candidates else None,
64
+ start_logits=features["start_logits"][:, :count],
65
+ end_logits=features["end_logits"][:, :count],
66
+ null_logits=features["null_logits"][:, :count],
67
+ count_log_rates=features["count_log_rates"][:, :count],
68
+ batch_size=1,
69
+ )
70
+
71
+ def score_explicit_spans(self, text, text_mask, query, query_mask, indices, valid_mask=None):
72
+ state = self.context.get()
73
+ features = state["features"]
74
+ full_queries = features["query_states"][0]
75
+ active_queries = int(state["arrays"]["query_mask"].sum())
76
+ query_count = query.shape[1]
77
+ span_count = indices.shape[2]
78
+ if query_count > self.max_queries or span_count > self.max_spans:
79
+ raise ValueError("Explicit-span request exceeds Core ML bucket capacity")
80
+ selected = []
81
+ for row in query[0]:
82
+ equal = torch.isclose(full_queries[:active_queries], row, atol=1e-6, rtol=0).all(-1)
83
+ matches = equal.nonzero(as_tuple=False).flatten()
84
+ if matches.numel() != 1:
85
+ raise ValueError("Explicit-span query cannot be mapped to the encoded schema")
86
+ selected.append(int(matches[0]))
87
+ selection = torch.tensor(selected, dtype=torch.long)
88
+ query_states = padded(query[0], self.max_queries).unsqueeze(0)
89
+ query_mask_padded = padded(query_mask[0].float(), self.max_queries).unsqueeze(0)
90
+
91
+ def selected_feature(name):
92
+ source = features[name][0].index_select(0, selection)
93
+ return padded(source, self.max_queries).unsqueeze(0)
94
+
95
+ span_indices = torch.zeros(1, self.max_queries, self.max_spans, 2, dtype=torch.int32)
96
+ span_mask = torch.zeros(1, self.max_queries, self.max_spans, dtype=torch.float32)
97
+ span_indices[:, :query_count, :span_count] = indices.int()
98
+ span_mask[:, :query_count, :span_count] = valid_mask.float() if valid_mask is not None else 1.0
99
+ values = (
100
+ text,
101
+ text_mask.float(),
102
+ query_states,
103
+ query_mask_padded,
104
+ features["boundary_states"],
105
+ selected_feature("start_logits"),
106
+ selected_feature("end_logits"),
107
+ selected_feature("inside_prefix"),
108
+ selected_feature("inside_prefix_mean"),
109
+ span_indices,
110
+ span_mask,
111
+ )
112
+ names = (
113
+ "text_states",
114
+ "text_mask",
115
+ "query_states",
116
+ "query_mask",
117
+ "boundary_states",
118
+ "start_logits",
119
+ "end_logits",
120
+ "inside_prefix",
121
+ "inside_prefix_mean",
122
+ "span_indices",
123
+ "span_mask",
124
+ )
125
+ output = self.explicit_model.predict(
126
+ {
127
+ name: value.numpy().astype(np.int32 if name == "span_indices" else np.float32)
128
+ for name, value in zip(names, values)
129
+ }
130
+ )["span_logits"]
131
+ return to_tensor(output)[:, :query_count, :span_count]
132
+
133
+
134
+ class CoreMLRelationScorer(torch.nn.Module):
135
+ """Call the trained Core ML relation graph after native pair selection."""
136
+
137
+ def __init__(self, context: ContextVar, model, max_words: int, max_relations: int, pair_cap: int):
138
+ super().__init__()
139
+ self.context = context
140
+ self.model = model
141
+ self.max_words = max_words
142
+ self.max_relations = max_relations
143
+ self.pair_cap = pair_cap
144
+
145
+ def forward(self, text, relation, candidates, pairs):
146
+ count = len(pairs)
147
+ if text.shape[1] != self.max_words or relation.shape[1] > self.max_relations or count > self.pair_cap:
148
+ raise ValueError("Relation request exceeds Core ML bucket capacity")
149
+ relation_states = torch.zeros(1, self.max_relations, relation.shape[-1], dtype=relation.dtype)
150
+ relation_states[:, : relation.shape[1]] = relation
151
+ state = self.context.get()
152
+ text_length = int(state["arrays"]["text_mask"].sum())
153
+ fields = (
154
+ text,
155
+ torch.tensor([text_length], dtype=torch.int32),
156
+ relation_states,
157
+ padded(pairs.batch_index.int(), self.pair_cap),
158
+ padded(pairs.relation_index.int(), self.pair_cap),
159
+ padded(pairs.head_start.int(), self.pair_cap),
160
+ padded(pairs.head_end.int(), self.pair_cap),
161
+ padded(pairs.tail_start.int(), self.pair_cap),
162
+ padded(pairs.tail_end.int(), self.pair_cap),
163
+ padded(pairs.pair_mask.float(), self.pair_cap),
164
+ )
165
+ names = (
166
+ "text_states",
167
+ "text_length",
168
+ "relation_states",
169
+ "batch_index",
170
+ "relation_index",
171
+ "head_start",
172
+ "head_end",
173
+ "tail_start",
174
+ "tail_end",
175
+ "pair_mask",
176
+ )
177
+ output = self.model.predict(
178
+ {
179
+ name: value.numpy().astype(
180
+ np.float32 if name in ("text_states", "relation_states", "pair_mask") else np.int32
181
+ )
182
+ for name, value in zip(names, fields)
183
+ }
184
+ )["relation_logits"]
185
+ return to_tensor(output)[:count]
186
+
187
+
188
+ class CoreMLRecordHead(torch.nn.Module):
189
+ """Keep GLiNER2's instance ordering while running learned layers in Core ML."""
190
+
191
+ def __init__(self, assignment_model, anchorless_model, max_fields=8, max_candidates=192, max_instances=1536):
192
+ super().__init__()
193
+ self.assignment_model = assignment_model
194
+ self.anchorless_model = anchorless_model
195
+ self.max_fields = max_fields
196
+ self.max_candidates = max_candidates
197
+ self.max_instances = max_instances
198
+
199
+ def forward_group(self, spec, query_states, candidates, sample_index):
200
+ field_specs = list(spec.fields)
201
+ field_query_ids = [field.query_id for field in field_specs]
202
+ if len(field_specs) > self.max_fields:
203
+ raise ValueError("Record has more fields than the Core ML bucket")
204
+ query_count = query_states.shape[0]
205
+ if any(query_id < 0 or query_id >= query_count for query_id in field_query_ids):
206
+ raise ValueError("Record field query is outside the encoded schema")
207
+ field_states, field_spans, field_logits, field_masks = [], [], [], []
208
+ for query_id in field_query_ids:
209
+ mask = candidates.valid_mask[sample_index, query_id]
210
+ if int(mask.sum()) > self.max_candidates:
211
+ raise ValueError("Record candidate count exceeds Core ML bucket")
212
+ field_states.append(candidates.candidate_states[sample_index, query_id][mask])
213
+ field_spans.append(candidates.indices[sample_index, query_id][mask])
214
+ field_logits.append(candidates.pair_logits[sample_index, query_id][mask])
215
+ field_masks.append(torch.ones(int(mask.sum()), dtype=torch.bool))
216
+
217
+ instance_seed = []
218
+ instance_spans = []
219
+ if spec.mode == "natural":
220
+ anchor = field_query_ids.index(spec.anchor_query_id)
221
+ instances = field_states[anchor]
222
+ for index, span in enumerate(field_spans[anchor]):
223
+ instance_seed.append((anchor, index))
224
+ instance_spans.append((int(span[0]), int(span[1])))
225
+ elif spec.mode == "latent":
226
+ instances = (
227
+ torch.cat(field_states, 0) if field_states else query_states.new_zeros((0, query_states.shape[-1]))
228
+ )
229
+ for field_index, spans in enumerate(field_spans):
230
+ for index, span in enumerate(spans):
231
+ instance_seed.append((field_index, index))
232
+ instance_spans.append((int(span[0]), int(span[1])))
233
+ else:
234
+ context_states = (
235
+ torch.cat(field_states, 0) if field_states else query_states.new_zeros((0, query_states.shape[-1]))
236
+ )
237
+ context_capacity = self.max_fields * self.max_candidates
238
+ context_mask = torch.zeros(context_capacity, dtype=torch.float32)
239
+ context_mask[: context_states.shape[0]] = 1.0
240
+ output = self.anchorless_model.predict(
241
+ {
242
+ "context_states": padded(context_states, context_capacity).numpy().astype(np.float32),
243
+ "context_mask": context_mask.numpy(),
244
+ }
245
+ )["instance_states"]
246
+ instances = to_tensor(output)
247
+ instance_seed = [None] * instances.shape[0]
248
+ instance_spans = [None] * instances.shape[0]
249
+
250
+ count = instances.shape[0]
251
+ if count > self.max_instances:
252
+ raise ValueError("Record instance count exceeds Core ML bucket")
253
+ hidden = query_states.shape[-1]
254
+ candidate_states = torch.zeros(self.max_fields, self.max_candidates, hidden)
255
+ for index, states in enumerate(field_states):
256
+ candidate_states[index, : states.shape[0]] = states
257
+ fields = (
258
+ padded(instances, self.max_instances),
259
+ padded(query_states[field_query_ids], self.max_fields),
260
+ candidate_states,
261
+ )
262
+ predicted = self.assignment_model.predict(
263
+ {
264
+ name: value.numpy().astype(np.float32)
265
+ for name, value in zip(("instance_states", "field_queries", "field_candidate_states"), fields)
266
+ }
267
+ )
268
+ assignment = to_tensor(predicted["assignment_logits"])
269
+ assignment_by_field = [
270
+ assignment[:count, field_index, : 1 + states.shape[0]] for field_index, states in enumerate(field_states)
271
+ ]
272
+ if spec.mode == "natural":
273
+ object_logits = field_logits[field_query_ids.index(spec.anchor_query_id)]
274
+ elif spec.mode == "latent":
275
+ object_logits = to_tensor(predicted["latent_seed_logits"])[:count]
276
+ else:
277
+ object_logits = to_tensor(predicted["object_logits"])[:count]
278
+ return RecordGroupOutput(
279
+ spec=spec,
280
+ object_logits=object_logits,
281
+ assign_logits=assignment_by_field,
282
+ field_query_ids=field_query_ids,
283
+ field_specs=field_specs,
284
+ field_spans=field_spans,
285
+ field_cand_mask=field_masks,
286
+ field_cand_logits=field_logits,
287
+ instance_seed=instance_seed,
288
+ instance_spans=instance_spans,
289
+ )
290
+
291
+
292
+ class CoreMLBoundaryExtractor(BoundaryExtractor):
293
+ """Native GLiNER2 schema/decoder with all trained extraction heads in Core ML."""
294
+
295
+ def __init__(self, model_dir: str, precision: str = "fp32", compute_units=ct.ComputeUnit.CPU_ONLY):
296
+ if precision not in ("fp16", "fp32"):
297
+ raise ValueError("precision must be fp16 or fp32")
298
+ torch.nn.Module.__init__(self)
299
+ folder = Path(model_dir)
300
+ config = json.loads((folder / "config.json").read_text())
301
+ self.boundary_settings = BoundaryHeadSettings(**config["boundary_head"])
302
+ self.processor = load_processor(str(folder / "tokenizer"))
303
+ self.enable_records = self.boundary_settings.enable_records
304
+ self.enable_relations = self.boundary_settings.enable_relations
305
+ self.strict_extraction = True
306
+ self.length, self.max_words, self.max_queries = 128, 64, 8
307
+ self._context = ContextVar("gliner2_coreml_extraction_context")
308
+ suffix = f"{precision}_L128_W64_Q8"
309
+ self.features_model = ct.models.MLModel(
310
+ str(folder / f"{MODEL_PREFIX}_extraction_features_{suffix}.mlpackage"), compute_units=compute_units
311
+ )
312
+ self.scorer_model = ct.models.MLModel(
313
+ str(folder / f"{MODEL_PREFIX}_extraction_scorer_{suffix}.mlpackage"), compute_units=compute_units
314
+ )
315
+ explicit = ct.models.MLModel(
316
+ str(folder / f"{MODEL_PREFIX}_explicit_{precision}_W64_Q8_S64.mlpackage"), compute_units=compute_units
317
+ )
318
+ relation = ct.models.MLModel(
319
+ str(folder / f"{MODEL_PREFIX}_relation_{precision}_W64_R4_P256.mlpackage"), compute_units=compute_units
320
+ )
321
+ assignment = ct.models.MLModel(
322
+ str(folder / f"{MODEL_PREFIX}_record_assignment_{precision}_F8_C192_I1536.mlpackage"),
323
+ compute_units=compute_units,
324
+ )
325
+ anchorless = ct.models.MLModel(
326
+ str(folder / f"{MODEL_PREFIX}_record_anchorless_{precision}_F8_C192_I1536.mlpackage"),
327
+ compute_units=compute_units,
328
+ )
329
+ self.boundary_head = CoreMLBoundaryHead(self._context, explicit, self.max_queries, 64)
330
+ self.relation_scorer = CoreMLRelationScorer(self._context, relation, self.max_words, 4, 256)
331
+ self.record_decoder = CoreMLRecordHead(assignment, anchorless)
332
+ self.relation_pair_generator = TypedRelationPairGenerator(
333
+ RelationProposalSettings(
334
+ heads_per_relation=self.boundary_settings.relation_heads_per_type,
335
+ tails_per_relation=self.boundary_settings.relation_tails_per_type,
336
+ pair_cap=self.boundary_settings.relation_pair_cap,
337
+ argument_threshold=self.boundary_settings.relation_argument_proposal_threshold,
338
+ )
339
+ )
340
+
341
+ def _encode_core(self, batch):
342
+ features = self._context.get()["features"]
343
+ query_states = features["query_states"]
344
+ text_states = features["text_states"]
345
+ query_mask = to_tensor(self._context.get()["arrays"]["query_mask"]).bool()
346
+ query_count = int(query_mask.sum())
347
+ query_states = query_states[:, :query_count]
348
+ query_mask = query_mask[:, :query_count]
349
+ text_mask = to_tensor(self._context.get()["arrays"]["text_mask"]).bool()
350
+ ext_specs, cls_specs, rel_specs, word_offsets = [], [], [], []
351
+ for sample_index in range(len(batch)):
352
+ specs = [
353
+ {
354
+ "group_index": item.task_index,
355
+ "field_index": item.role_index,
356
+ "task_type": item.task_type,
357
+ "task_name": item.task_name,
358
+ "field_name": item.role_name,
359
+ }
360
+ for item in batch.query_layouts[sample_index].queries
361
+ ]
362
+ ext_specs.append(specs)
363
+ classifications = []
364
+ choice_offset = 0
365
+ for group_index in range(batch.schema_counts[sample_index]):
366
+ if batch.task_types[sample_index][group_index] != "classifications":
367
+ continue
368
+ count = max(len(batch.schema_special_indices[sample_index][group_index]) - 1, 0)
369
+ schema_tokens = batch.schema_tokens_list[sample_index][group_index]
370
+ if count:
371
+ classifications.append(
372
+ {
373
+ "group_index": group_index,
374
+ "task_name": schema_tokens[2],
375
+ "schema_tokens": schema_tokens,
376
+ "group_embs": features["classification_logits"][
377
+ sample_index, choice_offset : choice_offset + count
378
+ ],
379
+ }
380
+ )
381
+ choice_offset += count
382
+ cls_specs.append(classifications)
383
+ word_offsets.append(
384
+ max(int(batch.text_word_counts[sample_index]) - len(batch.start_mappings[sample_index]), 0)
385
+ )
386
+ groups = {}
387
+ for query_id, spec in enumerate(specs):
388
+ groups.setdefault(spec["group_index"], []).append(query_id)
389
+ relations = []
390
+ for group_index, role_ids in groups.items():
391
+ if batch.task_types[sample_index][group_index] != "relations" or len(role_ids) < 2:
392
+ continue
393
+ head_id, tail_id = role_ids[:2]
394
+ role_states = query_states[sample_index, [head_id, tail_id]]
395
+ state = (
396
+ torch.cat((role_states[0], role_states[1]), -1)
397
+ if self.boundary_settings.directional_relation_states
398
+ else role_states.mean(0)
399
+ )
400
+ relations.append(
401
+ {
402
+ "group_index": group_index,
403
+ "relation_type": specs[head_id]["task_name"],
404
+ "spec": RelationTypeSpec(
405
+ specs[head_id]["task_name"], head_query_ids=(head_id,), tail_query_ids=(tail_id,)
406
+ ),
407
+ "query_state": state,
408
+ }
409
+ )
410
+ rel_specs.append(relations)
411
+ return {
412
+ "text_states": text_states,
413
+ "text_mask": text_mask,
414
+ "text_lengths": text_mask.sum(-1).long(),
415
+ "query_states": query_states,
416
+ "query_mask": query_mask,
417
+ "ext_specs": ext_specs,
418
+ "cls_specs": cls_specs,
419
+ "rel_specs": rel_specs,
420
+ "word_offsets": word_offsets,
421
+ }
422
+
423
+ def _extract_classification_result(self, results, schema_name, schema, embs, schema_tokens, temperature=1.0):
424
+ cls_config = self._resolve_classification_config(schema_tokens[2], schema.get("classifications", []))
425
+ if cls_config is None:
426
+ return
427
+ if temperature <= 0:
428
+ raise ValueError("Classification temperature must be positive")
429
+ logits = embs / temperature
430
+ activation = cls_config.get("class_act", "auto")
431
+ multi_label = cls_config.get("multi_label", False)
432
+ if activation == "sigmoid" or (activation != "softmax" and multi_label):
433
+ probabilities = torch.sigmoid(logits)
434
+ else:
435
+ probabilities = torch.softmax(logits, dim=-1)
436
+ labels = cls_config["labels"]
437
+ if multi_label:
438
+ threshold = cls_config.get("cls_threshold", 0.5)
439
+ chosen = [
440
+ (labels[index], float(probabilities[index]))
441
+ for index in range(len(labels))
442
+ if float(probabilities[index]) >= threshold
443
+ ]
444
+ if not chosen:
445
+ best = int(probabilities.argmax())
446
+ chosen = [(labels[best], float(probabilities[best]))]
447
+ results[cls_config["task"]] = chosen
448
+ return
449
+ best = int(probabilities.argmax())
450
+ results[cls_config["task"]] = (labels[best], float(probabilities[best]))
451
+
452
+ def extract(
453
+ self,
454
+ text: str,
455
+ schema,
456
+ threshold: float = 0.5,
457
+ format_results: bool = True,
458
+ include_confidence: bool = False,
459
+ include_spans: bool = False,
460
+ overlap_policy=None,
461
+ ):
462
+ schema_dicts, metadata_list = self._build_schema_dicts_and_metadata([schema])
463
+ if overlap_policy is not None:
464
+ metadata_list[0]["_overlap_policy"] = self._resolved_overlap_policy(overlap_policy)
465
+ arrays, batch = prepare_extraction(
466
+ self.processor, text, schema_dicts[0], self.length, self.max_words, self.max_queries
467
+ )
468
+ predicted = self.features_model.predict(arrays)
469
+ features = {name: to_tensor(predicted[name]) for name in FEATURE_NAMES}
470
+ query_mask = to_tensor(arrays["query_mask"]).bool()
471
+ candidates = None
472
+ if bool(query_mask.any()):
473
+ head = self.boundary_settings
474
+ pool = select_candidates(
475
+ features["pool_start_projection"],
476
+ features["pool_end_projection"],
477
+ features["boundary_mask"].bool(),
478
+ query_mask,
479
+ features["start_logits"],
480
+ features["end_logits"],
481
+ boundary_top_k=head.pool_boundary_top_k,
482
+ pool_size=head.pool_size,
483
+ min_pool_per_query=head.min_pool_per_query,
484
+ )
485
+ values = (
486
+ features["text_states"],
487
+ to_tensor(arrays["text_mask"]),
488
+ features["query_states"],
489
+ to_tensor(arrays["query_mask"]),
490
+ features["boundary_states"],
491
+ features["start_logits"],
492
+ features["end_logits"],
493
+ features["inside_prefix"],
494
+ features["inside_prefix_mean"],
495
+ pool.indices.int(),
496
+ pool.mask.float(),
497
+ pool.compat_logits,
498
+ )
499
+ scored = self.scorer_model.predict(
500
+ {
501
+ name: value.numpy().astype(np.int32 if name == "candidate_indices" else np.float32)
502
+ for name, value in zip(SCORE_INPUT_NAMES, values)
503
+ }
504
+ )
505
+ candidates = CandidateTensorBatch(
506
+ indices=pool.indices.unsqueeze(1).expand(1, self.max_queries, -1, 2),
507
+ proposal_logits=pool.proposal_logits.unsqueeze(1).expand(1, self.max_queries, -1),
508
+ pair_logits=to_tensor(scored["pair_logits"]),
509
+ valid_mask=pool.mask.unsqueeze(1).expand(1, self.max_queries, -1),
510
+ query_mask=query_mask,
511
+ candidate_states=to_tensor(scored["candidate_states"]).unsqueeze(1).expand(1, self.max_queries, -1, -1),
512
+ )
513
+ token = self._context.set({"arrays": arrays, "features": features, "candidates": candidates})
514
+ try:
515
+ raw = self._extract_from_batch(batch, threshold, metadata_list, include_confidence, include_spans)[0]
516
+ if format_results:
517
+ return self.format_results(
518
+ raw,
519
+ include_confidence,
520
+ metadata_list[0].get("relation_order", []),
521
+ metadata_list[0].get("classification_tasks", []),
522
+ )
523
+ return raw
524
+ finally:
525
+ self._context.reset(token)
gliner2_multi_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/model.mlmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a0a9bfed957616b1d1f9c3ef4794d0cce69de82e364e36e52bb0f5ef078377c7
3
+ size 48820
gliner2_multi_explicit_fp32_W64_Q8_S64.mlpackage/Data/com.apple.CoreML/weights/weight.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c2c82c889b6e214f7614a40419cfd7b05d7eeef8ef10c4bef2f950942e4f651b
3
+ size 1103040
gliner2_multi_explicit_fp32_W64_Q8_S64.mlpackage/Manifest.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "fileFormatVersion": "1.0.0",
3
+ "itemInfoEntries": {
4
+ "4293656E-8B02-42C6-8270-1A15234A033C": {
5
+ "author": "com.apple.CoreML",
6
+ "description": "CoreML Model Weights",
7
+ "name": "weights",
8
+ "path": "com.apple.CoreML/weights"
9
+ },
10
+ "8100562A-D22B-4871-914C-AB9CB44CBFB9": {
11
+ "author": "com.apple.CoreML",
12
+ "description": "CoreML Model Specification",
13
+ "name": "model.mlmodel",
14
+ "path": "com.apple.CoreML/model.mlmodel"
15
+ }
16
+ },
17
+ "rootModelIdentifier": "8100562A-D22B-4871-914C-AB9CB44CBFB9"
18
+ }
gliner2_multi_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:232fe09e50b91fef67f31015a0e90124e95f1d7084c448d99550d18741d2a81b
3
+ size 1012405
gliner2_multi_extraction_features_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4a4d2f7aa1395ce3aa72103dbdd295c6d4c3d8f0276d831df7a48c47cce57048
3
+ size 1154980292
gliner2_multi_extraction_features_fp32_L128_W64_Q8.mlpackage/Manifest.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "fileFormatVersion": "1.0.0",
3
+ "itemInfoEntries": {
4
+ "4A2EC563-096B-49E6-B36D-53185DCD41FD": {
5
+ "author": "com.apple.CoreML",
6
+ "description": "CoreML Model Specification",
7
+ "name": "model.mlmodel",
8
+ "path": "com.apple.CoreML/model.mlmodel"
9
+ },
10
+ "EC5258F9-1492-4E6C-AECE-7DFFF57AB1AE": {
11
+ "author": "com.apple.CoreML",
12
+ "description": "CoreML Model Weights",
13
+ "name": "weights",
14
+ "path": "com.apple.CoreML/weights"
15
+ }
16
+ },
17
+ "rootModelIdentifier": "4A2EC563-096B-49E6-B36D-53185DCD41FD"
18
+ }
gliner2_multi_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/model.mlmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:70dd5aa5aee2634c4ceff411dc71d079143ff687679ac763503fb1950c1ff3e7
3
+ size 32583
gliner2_multi_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Data/com.apple.CoreML/weights/weight.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:10a9fa54e3dbf2550a4636ae2275074c0f68770e60c351df1e3dcfddec6b9dc5
3
+ size 1717440
gliner2_multi_extraction_scorer_fp32_L128_W64_Q8.mlpackage/Manifest.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "fileFormatVersion": "1.0.0",
3
+ "itemInfoEntries": {
4
+ "45CB94EC-D25E-4E55-97C1-096E300D4901": {
5
+ "author": "com.apple.CoreML",
6
+ "description": "CoreML Model Weights",
7
+ "name": "weights",
8
+ "path": "com.apple.CoreML/weights"
9
+ },
10
+ "E90BD18E-A0C9-4A3D-8189-4C5EF4B3C62F": {
11
+ "author": "com.apple.CoreML",
12
+ "description": "CoreML Model Specification",
13
+ "name": "model.mlmodel",
14
+ "path": "com.apple.CoreML/model.mlmodel"
15
+ }
16
+ },
17
+ "rootModelIdentifier": "E90BD18E-A0C9-4A3D-8189-4C5EF4B3C62F"
18
+ }
gliner2_multi_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5f32406aebd0d9a907aafd0a2f83863b95a4fdeeaeeedb4ad003aa4d02b7074e
3
+ size 5003
gliner2_multi_record_anchorless_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a2620c168f1638982cd87243c172682dd4fc9ddd9e32f52bceb252b74a6452e1
3
+ size 2871232
gliner2_multi_record_anchorless_fp32_F8_C192_I1536.mlpackage/Manifest.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "fileFormatVersion": "1.0.0",
3
+ "itemInfoEntries": {
4
+ "DA8B2869-1F3E-4A46-A039-9B32E4B51C92": {
5
+ "author": "com.apple.CoreML",
6
+ "description": "CoreML Model Specification",
7
+ "name": "model.mlmodel",
8
+ "path": "com.apple.CoreML/model.mlmodel"
9
+ },
10
+ "F9900BE2-507C-4D16-B36A-BB23ACF54578": {
11
+ "author": "com.apple.CoreML",
12
+ "description": "CoreML Model Weights",
13
+ "name": "weights",
14
+ "path": "com.apple.CoreML/weights"
15
+ }
16
+ },
17
+ "rootModelIdentifier": "DA8B2869-1F3E-4A46-A039-9B32E4B51C92"
18
+ }
gliner2_multi_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/model.mlmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9b4d565ca5529e183c51254a16fa2f78be829bc01cb816f5010d158faf09ca85
3
+ size 7471
gliner2_multi_record_assignment_fp32_F8_C192_I1536.mlpackage/Data/com.apple.CoreML/weights/weight.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6498691d20934d4d65b345bf6c55000e6f72e0bc124493248a822150452bfa6e
3
+ size 1188480
gliner2_multi_record_assignment_fp32_F8_C192_I1536.mlpackage/Manifest.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "fileFormatVersion": "1.0.0",
3
+ "itemInfoEntries": {
4
+ "69BCF198-3C66-45D4-A832-949DB697DBB0": {
5
+ "author": "com.apple.CoreML",
6
+ "description": "CoreML Model Weights",
7
+ "name": "weights",
8
+ "path": "com.apple.CoreML/weights"
9
+ },
10
+ "A56B337B-108A-4589-9B4E-4C11887A4025": {
11
+ "author": "com.apple.CoreML",
12
+ "description": "CoreML Model Specification",
13
+ "name": "model.mlmodel",
14
+ "path": "com.apple.CoreML/model.mlmodel"
15
+ }
16
+ },
17
+ "rootModelIdentifier": "A56B337B-108A-4589-9B4E-4C11887A4025"
18
+ }
gliner2_multi_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/model.mlmodel ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:18c143798c805f0b00162b069059a1e1af155c2de331d57d5300edb6db845ece
3
+ size 29832
gliner2_multi_relation_fp32_W64_R4_P256.mlpackage/Data/com.apple.CoreML/weights/weight.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d9cf4fce04c725db699f4ab54dc21335fa0388501f055ca454a5a34ecf0e8955
3
+ size 23630592
gliner2_multi_relation_fp32_W64_R4_P256.mlpackage/Manifest.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "fileFormatVersion": "1.0.0",
3
+ "itemInfoEntries": {
4
+ "74412939-A2C3-44AA-9A10-770490C1A877": {
5
+ "author": "com.apple.CoreML",
6
+ "description": "CoreML Model Specification",
7
+ "name": "model.mlmodel",
8
+ "path": "com.apple.CoreML/model.mlmodel"
9
+ },
10
+ "BF14F25B-AE04-45D8-9FCB-23E1F9EED4C4": {
11
+ "author": "com.apple.CoreML",
12
+ "description": "CoreML Model Weights",
13
+ "name": "weights",
14
+ "path": "com.apple.CoreML/weights"
15
+ }
16
+ },
17
+ "rootModelIdentifier": "74412939-A2C3-44AA-9A10-770490C1A877"
18
+ }
preprocessing.py CHANGED
@@ -1,4 +1,5 @@
1
  """Native GLiNER2 schema preprocessing for a fixed Core ML bucket."""
 
2
  import numpy as np
3
  from gliner2 import Schema
4
  from gliner2.models.base import load_extractor_tokenizer
@@ -10,11 +11,13 @@ def load_processor(tokenizer_dir: str):
10
  """Load only the tokenizer and schema formatter needed by the Core ML model."""
11
  return SchemaTransformer(tokenizer=load_extractor_tokenizer(tokenizer_dir), token_pooling="first")
12
 
 
13
  def native_batch(native, text: str, task: str, labels: list[str], length: int):
14
  schema = Schema().classification(task, labels)
15
  collator = ExtractorCollator(native.processor, is_training=False, max_len=length, architecture=native.architecture)
16
  return collator([(text, schema.build())])
17
 
 
18
  def prepare_classification(native, text: str, task: str, labels: list[str], length: int, max_options: int):
19
  return prepare_with_processor(native.processor, text, task, labels, length, max_options)
20
 
@@ -41,3 +44,46 @@ def prepare_with_processor(processor, text: str, task: str, labels: list[str], l
41
  "marker_indices": indices.astype(np.int32),
42
  "marker_mask": mask.astype(np.float32),
43
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  """Native GLiNER2 schema preprocessing for a fixed Core ML bucket."""
2
+
3
  import numpy as np
4
  from gliner2 import Schema
5
  from gliner2.models.base import load_extractor_tokenizer
 
11
  """Load only the tokenizer and schema formatter needed by the Core ML model."""
12
  return SchemaTransformer(tokenizer=load_extractor_tokenizer(tokenizer_dir), token_pooling="first")
13
 
14
+
15
  def native_batch(native, text: str, task: str, labels: list[str], length: int):
16
  schema = Schema().classification(task, labels)
17
  collator = ExtractorCollator(native.processor, is_training=False, max_len=length, architecture=native.architecture)
18
  return collator([(text, schema.build())])
19
 
20
+
21
  def prepare_classification(native, text: str, task: str, labels: list[str], length: int, max_options: int):
22
  return prepare_with_processor(native.processor, text, task, labels, length, max_options)
23
 
 
44
  "marker_indices": indices.astype(np.int32),
45
  "marker_mask": mask.astype(np.float32),
46
  }
47
+
48
+
49
+ def prepare_extraction(
50
+ processor,
51
+ text: str,
52
+ schema,
53
+ length: int,
54
+ max_words: int,
55
+ max_queries: int,
56
+ max_choices: int = 8,
57
+ ):
58
+ """Prepare an extractive schema without allowing upstream word truncation."""
59
+ if min(length, max_words, max_queries, max_choices) < 1:
60
+ raise ValueError("Extraction bucket dimensions must all be positive")
61
+ built_schema = schema.build() if hasattr(schema, "build") else schema
62
+ collator = ExtractorCollator(processor, is_training=False, max_len=None, architecture="boundary")
63
+ batch = collator([(text, built_schema)])
64
+ if batch.input_ids.shape[1] > length:
65
+ raise ValueError(f"Schema and text require {batch.input_ids.shape[1]} subwords; bucket holds {length}")
66
+ if batch.text_word_indices.shape[1] > max_words:
67
+ raise ValueError(f"Text requires {batch.text_word_indices.shape[1]} words; bucket holds {max_words}")
68
+ if batch.query_marker_indices.shape[1] > max_queries:
69
+ raise ValueError(f"Schema requires {batch.query_marker_indices.shape[1]} queries; bucket holds {max_queries}")
70
+ if batch.cls_marker_indices.shape[1] > max_choices:
71
+ raise ValueError(f"Schema requires {batch.cls_marker_indices.shape[1]} choices; bucket holds {max_choices}")
72
+ if batch.query_marker_indices.shape[1] == 0 and batch.cls_marker_indices.shape[1] == 0:
73
+ raise ValueError("Schema has no extraction or classification queries")
74
+
75
+ def padded(values, width, fill=0):
76
+ array = values.numpy()
77
+ return np.pad(array, ((0, 0), (0, width - array.shape[1])), constant_values=fill)
78
+
79
+ arrays = {
80
+ "input_ids": padded(batch.input_ids, length, processor.tokenizer.pad_token_id).astype(np.int32),
81
+ "attention_mask": padded(batch.attention_mask, length).astype(np.int32),
82
+ "text_indices": padded(batch.text_word_indices, max_words).astype(np.int32),
83
+ "text_mask": padded(batch.text_word_mask, max_words).astype(np.float32),
84
+ "query_indices": padded(batch.query_marker_indices, max_queries).astype(np.int32),
85
+ "query_mask": padded(batch.query_marker_mask, max_queries).astype(np.float32),
86
+ "cls_indices": padded(batch.cls_marker_indices, max_choices).astype(np.int32),
87
+ "cls_mask": padded(batch.cls_marker_mask, max_choices).astype(np.float32),
88
+ }
89
+ return arrays, batch
pyproject.toml CHANGED
@@ -1,7 +1,7 @@
1
  [project]
2
  name = "gliner2-multi-coreml"
3
  version = "0.1.0"
4
- description = "Pinned GLiNER 2.5 small decision classifier Core ML export"
5
  requires-python = ">=3.12,<3.13"
6
  dependencies = [
7
  "coremltools==9.0",
@@ -9,6 +9,7 @@ dependencies = [
9
  "huggingface-hub>=0.34,<1",
10
  "numpy<2.3",
11
  "protobuf>=5,<7",
 
12
  "sentencepiece>=0.2,<0.3",
13
  "torch==2.7.0",
14
  "transformers==4.57.6",
@@ -16,6 +17,7 @@ dependencies = [
16
 
17
  [dependency-groups]
18
  dev = ["pytest>=8.4", "ruff>=0.13"]
 
19
 
20
  [tool.pytest.ini_options]
21
  testpaths = ["tests"]
 
1
  [project]
2
  name = "gliner2-multi-coreml"
3
  version = "0.1.0"
4
+ description = "Pinned GLiNER 2.5 multi Core ML classification and extraction export"
5
  requires-python = ">=3.12,<3.13"
6
  dependencies = [
7
  "coremltools==9.0",
 
9
  "huggingface-hub>=0.34,<1",
10
  "numpy<2.3",
11
  "protobuf>=5,<7",
12
+ "psutil>=7,<8",
13
  "sentencepiece>=0.2,<0.3",
14
  "torch==2.7.0",
15
  "transformers==4.57.6",
 
17
 
18
  [dependency-groups]
19
  dev = ["pytest>=8.4", "ruff>=0.13"]
20
+ compression = ["scikit-learn==1.5.1"]
21
 
22
  [tool.pytest.ini_options]
23
  testpaths = ["tests"]
reports/extraction-validation.json ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "purpose": "selected local Core ML extraction parity and latency, not a Decision Index score",
3
+ "source_model": "fastino/gliner2.5-multi-v1",
4
+ "source_revision": "a221b77a8baf4a613b8f8652661d41fa10a5641e",
5
+ "runtime": {
6
+ "coremltools": "9.0",
7
+ "gliner2": "2.0.0",
8
+ "mac": "M5 Pro, 24 GB, macOS 27.0"
9
+ },
10
+ "bucket": "L128/W64/Q8/K8; candidate C192, explicit S64, relation R4/P256, records F8/C192/I1536",
11
+ "fixtures": {
12
+ "fp32": {
13
+ "matched_structures": 15,
14
+ "fixture_count": 15,
15
+ "failed_cases": [],
16
+ "maximum_confidence_error": 4.887580871582031e-06
17
+ },
18
+ "fp16": {
19
+ "matched_structures": 14,
20
+ "fixture_count": 15,
21
+ "failed_cases": [
22
+ "record_latent"
23
+ ],
24
+ "maximum_confidence_error": 0.037593305110931396
25
+ }
26
+ },
27
+ "latency": {
28
+ "fp32_all": {
29
+ "p50_ms": 10.563604475464672,
30
+ "p95_ms": 12.338125030510128,
31
+ "warmup": 20,
32
+ "iterations": 200,
33
+ "shape": "L128/W64/Q8/C192"
34
+ },
35
+ "fp32_cpu_and_gpu": {
36
+ "p50_ms": 10.022375005064532,
37
+ "p95_ms": 10.64095797482878,
38
+ "warmup": 20,
39
+ "iterations": 200,
40
+ "shape": "L128/W64/Q8/C192"
41
+ },
42
+ "fp32_cpu_and_neural_engine": {
43
+ "p50_ms": 36.07562501565553,
44
+ "p95_ms": 52.11695801699534,
45
+ "warmup": 20,
46
+ "iterations": 200,
47
+ "shape": "L128/W64/Q8/C192"
48
+ },
49
+ "fp32_cpu_only": {
50
+ "p50_ms": 36.11889548483305,
51
+ "p95_ms": 62.56379099795595,
52
+ "warmup": 20,
53
+ "iterations": 200,
54
+ "shape": "L128/W64/Q8/C192"
55
+ }
56
+ },
57
+ "feature_compute_plan": {
58
+ "fp32-all": {
59
+ "cpu_percent": 0.0,
60
+ "gpu_percent": 100.0,
61
+ "ane_percent": 0.0
62
+ },
63
+ "fp32-cpu_and_neural_engine": {
64
+ "cpu_percent": 100.0,
65
+ "gpu_percent": 0.0,
66
+ "ane_percent": 0.0
67
+ }
68
+ },
69
+ "compression": {
70
+ "attempt": "LUT8 per-tensor k-means on FP32 feature package using scikit-learn 1.5.1",
71
+ "result": "stopped after two minutes with zero graph operations processed and repeated k-means divide-by-zero/overflow warnings",
72
+ "release": false
73
+ },
74
+ "published_extraction_precisions": [
75
+ "fp32"
76
+ ],
77
+ "limitations": [
78
+ "fixed shape; requests over capacity fail",
79
+ "small selected real-text fixtures only; no full Decision Index score",
80
+ "current pinned source revision is not established as the historical evaluation checkpoint"
81
+ ]
82
+ }
reports/extraction-verify-fp16.json ADDED
@@ -0,0 +1,849 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "source_model": "fastino/gliner2.5-multi-v1",
3
+ "source_revision": "a221b77a8baf4a613b8f8652661d41fa10a5641e",
4
+ "precision": "fp16",
5
+ "selected_manifest": "fifteen fixed real-text schema fixtures, not a Decision Index score",
6
+ "matched": 14,
7
+ "total": 15,
8
+ "cases": [
9
+ {
10
+ "name": "entities",
11
+ "text": "Alice founded Acme in Toronto in 2020.",
12
+ "native": {
13
+ "entities": {
14
+ "person": [
15
+ {
16
+ "text": "Alice",
17
+ "confidence": 0.9971066117286682,
18
+ "start": 0,
19
+ "end": 5
20
+ }
21
+ ],
22
+ "organization": [
23
+ {
24
+ "text": "Acme",
25
+ "confidence": 0.993619978427887,
26
+ "start": 14,
27
+ "end": 18
28
+ }
29
+ ],
30
+ "location": [
31
+ {
32
+ "text": "Toronto",
33
+ "confidence": 0.9991662502288818,
34
+ "start": 22,
35
+ "end": 29
36
+ }
37
+ ]
38
+ }
39
+ },
40
+ "coreml": {
41
+ "entities": {
42
+ "person": [
43
+ {
44
+ "text": "Alice",
45
+ "confidence": 0.9968024492263794,
46
+ "start": 0,
47
+ "end": 5
48
+ }
49
+ ],
50
+ "organization": [
51
+ {
52
+ "text": "Acme",
53
+ "confidence": 0.9933589100837708,
54
+ "start": 14,
55
+ "end": 18
56
+ }
57
+ ],
58
+ "location": [
59
+ {
60
+ "text": "Toronto",
61
+ "confidence": 0.9990890026092529,
62
+ "start": 22,
63
+ "end": 29
64
+ }
65
+ ]
66
+ }
67
+ },
68
+ "structure_match": true,
69
+ "maximum_confidence_error": 0.00030416250228881836
70
+ },
71
+ {
72
+ "name": "entities_multi",
73
+ "text": "Marie Curie was born in Warsaw and worked in Paris.",
74
+ "native": {
75
+ "entities": {
76
+ "person": [
77
+ {
78
+ "text": "Marie Curie",
79
+ "confidence": 0.9977118968963623,
80
+ "start": 0,
81
+ "end": 11
82
+ }
83
+ ],
84
+ "city": [
85
+ {
86
+ "text": "Paris",
87
+ "confidence": 0.9992535710334778,
88
+ "start": 45,
89
+ "end": 50
90
+ },
91
+ {
92
+ "text": "Warsaw",
93
+ "confidence": 0.9987508058547974,
94
+ "start": 24,
95
+ "end": 30
96
+ }
97
+ ]
98
+ }
99
+ },
100
+ "coreml": {
101
+ "entities": {
102
+ "person": [
103
+ {
104
+ "text": "Marie Curie",
105
+ "confidence": 0.9976586103439331,
106
+ "start": 0,
107
+ "end": 11
108
+ }
109
+ ],
110
+ "city": [
111
+ {
112
+ "text": "Paris",
113
+ "confidence": 0.9993201494216919,
114
+ "start": 45,
115
+ "end": 50
116
+ },
117
+ {
118
+ "text": "Warsaw",
119
+ "confidence": 0.9988213181495667,
120
+ "start": 24,
121
+ "end": 30
122
+ }
123
+ ]
124
+ }
125
+ },
126
+ "structure_match": true,
127
+ "maximum_confidence_error": 7.051229476928711e-05
128
+ },
129
+ {
130
+ "name": "relations",
131
+ "text": "Alice founded Acme in Toronto.",
132
+ "native": {
133
+ "relation_extraction": {
134
+ "founded": [
135
+ {
136
+ "head": {
137
+ "text": "Alice",
138
+ "start": 0,
139
+ "end": 5,
140
+ "confidence": 0.6135826110839844
141
+ },
142
+ "tail": {
143
+ "text": "Acme",
144
+ "start": 14,
145
+ "end": 18,
146
+ "confidence": 0.6135826110839844
147
+ }
148
+ }
149
+ ]
150
+ }
151
+ },
152
+ "coreml": {
153
+ "relation_extraction": {
154
+ "founded": [
155
+ {
156
+ "head": {
157
+ "text": "Alice",
158
+ "start": 0,
159
+ "end": 5,
160
+ "confidence": 0.6124835014343262
161
+ },
162
+ "tail": {
163
+ "text": "Acme",
164
+ "start": 14,
165
+ "end": 18,
166
+ "confidence": 0.6124835014343262
167
+ }
168
+ }
169
+ ]
170
+ }
171
+ },
172
+ "structure_match": true,
173
+ "maximum_confidence_error": 0.0010991096496582031
174
+ },
175
+ {
176
+ "name": "relations_two",
177
+ "text": "Steve Jobs founded Apple.",
178
+ "native": {
179
+ "relation_extraction": {
180
+ "founded": [
181
+ {
182
+ "head": {
183
+ "text": "Steve Jobs",
184
+ "start": 0,
185
+ "end": 10,
186
+ "confidence": 0.6887430548667908
187
+ },
188
+ "tail": {
189
+ "text": "Apple",
190
+ "start": 19,
191
+ "end": 24,
192
+ "confidence": 0.6887430548667908
193
+ }
194
+ }
195
+ ]
196
+ }
197
+ },
198
+ "coreml": {
199
+ "relation_extraction": {
200
+ "founded": [
201
+ {
202
+ "head": {
203
+ "text": "Steve Jobs",
204
+ "start": 0,
205
+ "end": 10,
206
+ "confidence": 0.7065955400466919
207
+ },
208
+ "tail": {
209
+ "text": "Apple",
210
+ "start": 19,
211
+ "end": 24,
212
+ "confidence": 0.7065955400466919
213
+ }
214
+ }
215
+ ]
216
+ }
217
+ },
218
+ "structure_match": true,
219
+ "maximum_confidence_error": 0.017852485179901123
220
+ },
221
+ {
222
+ "name": "record_natural",
223
+ "text": "Alice works at Acme. Bob works at Beta.",
224
+ "native": {
225
+ "employment": [
226
+ {
227
+ "person": {
228
+ "text": "Alice",
229
+ "confidence": 0.9879767298698425,
230
+ "start": 0,
231
+ "end": 5
232
+ },
233
+ "company": {
234
+ "text": "Acme",
235
+ "confidence": 0.8241090774536133,
236
+ "start": 15,
237
+ "end": 19
238
+ }
239
+ },
240
+ {
241
+ "person": {
242
+ "text": "Bob",
243
+ "confidence": 0.9651215672492981,
244
+ "start": 21,
245
+ "end": 24
246
+ },
247
+ "company": {
248
+ "text": "Beta",
249
+ "confidence": 0.6678691506385803,
250
+ "start": 34,
251
+ "end": 38
252
+ }
253
+ }
254
+ ]
255
+ },
256
+ "coreml": {
257
+ "employment": [
258
+ {
259
+ "person": {
260
+ "text": "Alice",
261
+ "confidence": 0.987616240978241,
262
+ "start": 0,
263
+ "end": 5
264
+ },
265
+ "company": {
266
+ "text": "Acme",
267
+ "confidence": 0.7892336845397949,
268
+ "start": 15,
269
+ "end": 19
270
+ }
271
+ },
272
+ {
273
+ "person": {
274
+ "text": "Bob",
275
+ "confidence": 0.9630224108695984,
276
+ "start": 21,
277
+ "end": 24
278
+ },
279
+ "company": {
280
+ "text": "Beta",
281
+ "confidence": 0.653237521648407,
282
+ "start": 34,
283
+ "end": 38
284
+ }
285
+ }
286
+ ]
287
+ },
288
+ "structure_match": true,
289
+ "maximum_confidence_error": 0.03487539291381836
290
+ },
291
+ {
292
+ "name": "record_latent",
293
+ "text": "Alice works at Acme. Bob works at Beta.",
294
+ "native": {
295
+ "employment": [
296
+ {
297
+ "person": null,
298
+ "company": {
299
+ "text": "Beta",
300
+ "confidence": 0.5839666128158569,
301
+ "start": 34,
302
+ "end": 38
303
+ }
304
+ },
305
+ {
306
+ "person": null,
307
+ "company": {
308
+ "text": "Acme",
309
+ "confidence": 0.6181439161300659,
310
+ "start": 15,
311
+ "end": 19
312
+ }
313
+ },
314
+ {
315
+ "person": {
316
+ "text": "Alice",
317
+ "confidence": 0.9672633409500122,
318
+ "start": 0,
319
+ "end": 5
320
+ },
321
+ "company": null
322
+ },
323
+ {
324
+ "person": null,
325
+ "company": {
326
+ "text": "Beta",
327
+ "confidence": 0.8805453777313232,
328
+ "start": 34,
329
+ "end": 38
330
+ }
331
+ },
332
+ {
333
+ "person": null,
334
+ "company": {
335
+ "text": "Beta",
336
+ "confidence": 0.8966390490531921,
337
+ "start": 34,
338
+ "end": 38
339
+ }
340
+ },
341
+ {
342
+ "person": null,
343
+ "company": {
344
+ "text": "Beta",
345
+ "confidence": 0.7147471904754639,
346
+ "start": 34,
347
+ "end": 38
348
+ }
349
+ },
350
+ {
351
+ "person": null,
352
+ "company": {
353
+ "text": "Acme",
354
+ "confidence": 0.6521790027618408,
355
+ "start": 15,
356
+ "end": 19
357
+ }
358
+ },
359
+ {
360
+ "person": {
361
+ "text": "Bob",
362
+ "confidence": 0.5080211162567139,
363
+ "start": 21,
364
+ "end": 24
365
+ },
366
+ "company": null
367
+ },
368
+ {
369
+ "person": null,
370
+ "company": {
371
+ "text": "Beta",
372
+ "confidence": 0.5336229205131531,
373
+ "start": 34,
374
+ "end": 38
375
+ }
376
+ },
377
+ {
378
+ "person": null,
379
+ "company": {
380
+ "text": "Acme",
381
+ "confidence": 0.6106622815132141,
382
+ "start": 15,
383
+ "end": 19
384
+ }
385
+ }
386
+ ]
387
+ },
388
+ "coreml": {
389
+ "employment": [
390
+ {
391
+ "person": null,
392
+ "company": {
393
+ "text": "Beta",
394
+ "confidence": 0.5487339496612549,
395
+ "start": 34,
396
+ "end": 38
397
+ }
398
+ },
399
+ {
400
+ "person": null,
401
+ "company": {
402
+ "text": "Beta",
403
+ "confidence": 0.5954337120056152,
404
+ "start": 34,
405
+ "end": 38
406
+ }
407
+ },
408
+ {
409
+ "person": {
410
+ "text": "Alice",
411
+ "confidence": 0.9568765759468079,
412
+ "start": 0,
413
+ "end": 5
414
+ },
415
+ "company": null
416
+ },
417
+ {
418
+ "person": null,
419
+ "company": {
420
+ "text": "Beta",
421
+ "confidence": 0.8806945085525513,
422
+ "start": 34,
423
+ "end": 38
424
+ }
425
+ },
426
+ {
427
+ "person": null,
428
+ "company": {
429
+ "text": "Beta",
430
+ "confidence": 0.8680456876754761,
431
+ "start": 34,
432
+ "end": 38
433
+ }
434
+ },
435
+ {
436
+ "person": null,
437
+ "company": {
438
+ "text": "Beta",
439
+ "confidence": 0.6771538853645325,
440
+ "start": 34,
441
+ "end": 38
442
+ }
443
+ },
444
+ {
445
+ "person": null,
446
+ "company": {
447
+ "text": "Acme",
448
+ "confidence": 0.6240644454956055,
449
+ "start": 15,
450
+ "end": 19
451
+ }
452
+ },
453
+ {
454
+ "person": null,
455
+ "company": {
456
+ "text": "Beta",
457
+ "confidence": 0.5099550485610962,
458
+ "start": 34,
459
+ "end": 38
460
+ }
461
+ },
462
+ {
463
+ "person": null,
464
+ "company": {
465
+ "text": "Acme",
466
+ "confidence": 0.5437113642692566,
467
+ "start": 15,
468
+ "end": 19
469
+ }
470
+ }
471
+ ]
472
+ },
473
+ "structure_match": false,
474
+ "maximum_confidence_error": 0.037593305110931396
475
+ },
476
+ {
477
+ "name": "record_anchorless",
478
+ "text": "Alice works at Acme. Bob works at Beta.",
479
+ "native": {},
480
+ "coreml": {},
481
+ "structure_match": true,
482
+ "maximum_confidence_error": null
483
+ },
484
+ {
485
+ "name": "attributes",
486
+ "text": "Alice founded Acme.",
487
+ "native": {
488
+ "entities": {
489
+ "person": [
490
+ {
491
+ "text": "Alice",
492
+ "confidence": 0.9935710430145264,
493
+ "start": 0,
494
+ "end": 5,
495
+ "role": {
496
+ "label": "founder",
497
+ "confidence": 0.999847412109375
498
+ }
499
+ }
500
+ ],
501
+ "organization": [
502
+ {
503
+ "text": "Acme",
504
+ "confidence": 0.9912181496620178,
505
+ "start": 14,
506
+ "end": 18,
507
+ "role": {
508
+ "label": "founder",
509
+ "confidence": 0.9899052977561951
510
+ }
511
+ }
512
+ ]
513
+ }
514
+ },
515
+ "coreml": {
516
+ "entities": {
517
+ "person": [
518
+ {
519
+ "text": "Alice",
520
+ "confidence": 0.993854820728302,
521
+ "start": 0,
522
+ "end": 5,
523
+ "role": {
524
+ "label": "founder",
525
+ "confidence": 0.9998377561569214
526
+ }
527
+ }
528
+ ],
529
+ "organization": [
530
+ {
531
+ "text": "Acme",
532
+ "confidence": 0.9906960725784302,
533
+ "start": 14,
534
+ "end": 18,
535
+ "role": {
536
+ "label": "founder",
537
+ "confidence": 0.9916600584983826
538
+ }
539
+ }
540
+ ]
541
+ }
542
+ },
543
+ "structure_match": true,
544
+ "maximum_confidence_error": 0.0017547607421875
545
+ },
546
+ {
547
+ "name": "mixed_classification",
548
+ "text": "Alice founded Acme.",
549
+ "native": {
550
+ "entities": {
551
+ "person": [
552
+ {
553
+ "text": "Alice",
554
+ "confidence": 0.9987186193466187,
555
+ "start": 0,
556
+ "end": 5
557
+ }
558
+ ],
559
+ "organization": [
560
+ {
561
+ "text": "Acme",
562
+ "confidence": 0.9962791800498962,
563
+ "start": 14,
564
+ "end": 18
565
+ }
566
+ ]
567
+ },
568
+ "sentiment": {
569
+ "label": "positive",
570
+ "confidence": 0.9953342080116272
571
+ }
572
+ },
573
+ "coreml": {
574
+ "entities": {
575
+ "person": [
576
+ {
577
+ "text": "Alice",
578
+ "confidence": 0.9987550973892212,
579
+ "start": 0,
580
+ "end": 5
581
+ }
582
+ ],
583
+ "organization": [
584
+ {
585
+ "text": "Acme",
586
+ "confidence": 0.9962051510810852,
587
+ "start": 14,
588
+ "end": 18
589
+ }
590
+ ]
591
+ },
592
+ "sentiment": {
593
+ "label": "positive",
594
+ "confidence": 0.9948803186416626
595
+ }
596
+ },
597
+ "structure_match": true,
598
+ "maximum_confidence_error": 0.0004538893699645996
599
+ },
600
+ {
601
+ "name": "classification_only",
602
+ "text": "The rocket launched successfully.",
603
+ "native": {
604
+ "topic": {
605
+ "label": "sports",
606
+ "confidence": 0.5060790777206421
607
+ }
608
+ },
609
+ "coreml": {
610
+ "topic": {
611
+ "label": "sports",
612
+ "confidence": 0.5021564364433289
613
+ }
614
+ },
615
+ "structure_match": true,
616
+ "maximum_confidence_error": 0.003922641277313232
617
+ },
618
+ {
619
+ "name": "enum_choice",
620
+ "text": "The item is electronics.",
621
+ "native": {
622
+ "product": [
623
+ {
624
+ "category": {
625
+ "text": "electronics",
626
+ "confidence": 0.9997705817222595
627
+ }
628
+ }
629
+ ]
630
+ },
631
+ "coreml": {
632
+ "product": [
633
+ {
634
+ "category": {
635
+ "text": "electronics",
636
+ "confidence": 0.9997565150260925
637
+ }
638
+ }
639
+ ]
640
+ },
641
+ "structure_match": true,
642
+ "maximum_confidence_error": 1.4066696166992188e-05
643
+ },
644
+ {
645
+ "name": "es_entities",
646
+ "text": "María trabaja en Madrid para Telefónica.",
647
+ "native": {
648
+ "entities": {
649
+ "persona": [
650
+ {
651
+ "text": "María",
652
+ "confidence": 0.997144877910614,
653
+ "start": 0,
654
+ "end": 5
655
+ }
656
+ ],
657
+ "organización": [
658
+ {
659
+ "text": "Telefónica",
660
+ "confidence": 0.9903497695922852,
661
+ "start": 29,
662
+ "end": 39
663
+ }
664
+ ],
665
+ "ciudad": [
666
+ {
667
+ "text": "Madrid",
668
+ "confidence": 0.9995064735412598,
669
+ "start": 17,
670
+ "end": 23
671
+ }
672
+ ]
673
+ }
674
+ },
675
+ "coreml": {
676
+ "entities": {
677
+ "persona": [
678
+ {
679
+ "text": "María",
680
+ "confidence": 0.9972207546234131,
681
+ "start": 0,
682
+ "end": 5
683
+ }
684
+ ],
685
+ "organización": [
686
+ {
687
+ "text": "Telefónica",
688
+ "confidence": 0.9890975952148438,
689
+ "start": 29,
690
+ "end": 39
691
+ }
692
+ ],
693
+ "ciudad": [
694
+ {
695
+ "text": "Madrid",
696
+ "confidence": 0.9995102882385254,
697
+ "start": 17,
698
+ "end": 23
699
+ }
700
+ ]
701
+ }
702
+ },
703
+ "structure_match": true,
704
+ "maximum_confidence_error": 0.0012521743774414062
705
+ },
706
+ {
707
+ "name": "fr_entities",
708
+ "text": "Jean Dupont travaille à Paris pour Airbus.",
709
+ "native": {
710
+ "entities": {
711
+ "personne": [
712
+ {
713
+ "text": "Jean Dupont",
714
+ "confidence": 0.9973317384719849,
715
+ "start": 0,
716
+ "end": 11
717
+ }
718
+ ],
719
+ "entreprise": [
720
+ {
721
+ "text": "Airbus",
722
+ "confidence": 0.9904603958129883,
723
+ "start": 35,
724
+ "end": 41
725
+ }
726
+ ],
727
+ "ville": [
728
+ {
729
+ "text": "Paris",
730
+ "confidence": 0.9991673231124878,
731
+ "start": 24,
732
+ "end": 29
733
+ }
734
+ ]
735
+ }
736
+ },
737
+ "coreml": {
738
+ "entities": {
739
+ "personne": [
740
+ {
741
+ "text": "Jean Dupont",
742
+ "confidence": 0.997285008430481,
743
+ "start": 0,
744
+ "end": 11
745
+ }
746
+ ],
747
+ "entreprise": [
748
+ {
749
+ "text": "Airbus",
750
+ "confidence": 0.9899086952209473,
751
+ "start": 35,
752
+ "end": 41
753
+ }
754
+ ],
755
+ "ville": [
756
+ {
757
+ "text": "Paris",
758
+ "confidence": 0.9991958737373352,
759
+ "start": 24,
760
+ "end": 29
761
+ }
762
+ ]
763
+ }
764
+ },
765
+ "structure_match": true,
766
+ "maximum_confidence_error": 0.0005517005920410156
767
+ },
768
+ {
769
+ "name": "zh_entities",
770
+ "text": "张伟在北京大学工作。",
771
+ "native": {
772
+ "entities": {
773
+ "人物": [
774
+ {
775
+ "text": "张伟在北京大学工作",
776
+ "confidence": 0.9425729513168335,
777
+ "start": 0,
778
+ "end": 9
779
+ }
780
+ ],
781
+ "机构": [],
782
+ "地点": []
783
+ }
784
+ },
785
+ "coreml": {
786
+ "entities": {
787
+ "人物": [
788
+ {
789
+ "text": "张伟在北京大学工作",
790
+ "confidence": 0.9488224983215332,
791
+ "start": 0,
792
+ "end": 9
793
+ }
794
+ ],
795
+ "机构": [],
796
+ "地点": []
797
+ }
798
+ },
799
+ "structure_match": true,
800
+ "maximum_confidence_error": 0.006249547004699707
801
+ },
802
+ {
803
+ "name": "de_entities",
804
+ "text": "Angela Merkel besuchte Berlin.",
805
+ "native": {
806
+ "entities": {
807
+ "Person": [
808
+ {
809
+ "text": "Angela Merkel",
810
+ "confidence": 0.9966775178909302,
811
+ "start": 0,
812
+ "end": 13
813
+ }
814
+ ],
815
+ "Stadt": [
816
+ {
817
+ "text": "Berlin",
818
+ "confidence": 0.9987296462059021,
819
+ "start": 23,
820
+ "end": 29
821
+ }
822
+ ]
823
+ }
824
+ },
825
+ "coreml": {
826
+ "entities": {
827
+ "Person": [
828
+ {
829
+ "text": "Angela Merkel",
830
+ "confidence": 0.996434211730957,
831
+ "start": 0,
832
+ "end": 13
833
+ }
834
+ ],
835
+ "Stadt": [
836
+ {
837
+ "text": "Berlin",
838
+ "confidence": 0.998798131942749,
839
+ "start": 23,
840
+ "end": 29
841
+ }
842
+ ]
843
+ }
844
+ },
845
+ "structure_match": true,
846
+ "maximum_confidence_error": 0.00024330615997314453
847
+ }
848
+ ]
849
+ }
reports/extraction-verify-fp32.json ADDED
@@ -0,0 +1,858 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "source_model": "fastino/gliner2.5-multi-v1",
3
+ "source_revision": "a221b77a8baf4a613b8f8652661d41fa10a5641e",
4
+ "precision": "fp32",
5
+ "selected_manifest": "fifteen fixed real-text schema fixtures, not a Decision Index score",
6
+ "matched": 15,
7
+ "total": 15,
8
+ "cases": [
9
+ {
10
+ "name": "entities",
11
+ "text": "Alice founded Acme in Toronto in 2020.",
12
+ "native": {
13
+ "entities": {
14
+ "person": [
15
+ {
16
+ "text": "Alice",
17
+ "confidence": 0.9971066117286682,
18
+ "start": 0,
19
+ "end": 5
20
+ }
21
+ ],
22
+ "organization": [
23
+ {
24
+ "text": "Acme",
25
+ "confidence": 0.993619978427887,
26
+ "start": 14,
27
+ "end": 18
28
+ }
29
+ ],
30
+ "location": [
31
+ {
32
+ "text": "Toronto",
33
+ "confidence": 0.9991662502288818,
34
+ "start": 22,
35
+ "end": 29
36
+ }
37
+ ]
38
+ }
39
+ },
40
+ "coreml": {
41
+ "entities": {
42
+ "person": [
43
+ {
44
+ "text": "Alice",
45
+ "confidence": 0.9971067309379578,
46
+ "start": 0,
47
+ "end": 5
48
+ }
49
+ ],
50
+ "organization": [
51
+ {
52
+ "text": "Acme",
53
+ "confidence": 0.9936198592185974,
54
+ "start": 14,
55
+ "end": 18
56
+ }
57
+ ],
58
+ "location": [
59
+ {
60
+ "text": "Toronto",
61
+ "confidence": 0.9991662502288818,
62
+ "start": 22,
63
+ "end": 29
64
+ }
65
+ ]
66
+ }
67
+ },
68
+ "structure_match": true,
69
+ "maximum_confidence_error": 1.1920928955078125e-07
70
+ },
71
+ {
72
+ "name": "entities_multi",
73
+ "text": "Marie Curie was born in Warsaw and worked in Paris.",
74
+ "native": {
75
+ "entities": {
76
+ "person": [
77
+ {
78
+ "text": "Marie Curie",
79
+ "confidence": 0.9977118968963623,
80
+ "start": 0,
81
+ "end": 11
82
+ }
83
+ ],
84
+ "city": [
85
+ {
86
+ "text": "Paris",
87
+ "confidence": 0.9992535710334778,
88
+ "start": 45,
89
+ "end": 50
90
+ },
91
+ {
92
+ "text": "Warsaw",
93
+ "confidence": 0.9987508058547974,
94
+ "start": 24,
95
+ "end": 30
96
+ }
97
+ ]
98
+ }
99
+ },
100
+ "coreml": {
101
+ "entities": {
102
+ "person": [
103
+ {
104
+ "text": "Marie Curie",
105
+ "confidence": 0.9977117776870728,
106
+ "start": 0,
107
+ "end": 11
108
+ }
109
+ ],
110
+ "city": [
111
+ {
112
+ "text": "Paris",
113
+ "confidence": 0.9992535710334778,
114
+ "start": 45,
115
+ "end": 50
116
+ },
117
+ {
118
+ "text": "Warsaw",
119
+ "confidence": 0.9987508058547974,
120
+ "start": 24,
121
+ "end": 30
122
+ }
123
+ ]
124
+ }
125
+ },
126
+ "structure_match": true,
127
+ "maximum_confidence_error": 1.1920928955078125e-07
128
+ },
129
+ {
130
+ "name": "relations",
131
+ "text": "Alice founded Acme in Toronto.",
132
+ "native": {
133
+ "relation_extraction": {
134
+ "founded": [
135
+ {
136
+ "head": {
137
+ "text": "Alice",
138
+ "start": 0,
139
+ "end": 5,
140
+ "confidence": 0.6135826110839844
141
+ },
142
+ "tail": {
143
+ "text": "Acme",
144
+ "start": 14,
145
+ "end": 18,
146
+ "confidence": 0.6135826110839844
147
+ }
148
+ }
149
+ ]
150
+ }
151
+ },
152
+ "coreml": {
153
+ "relation_extraction": {
154
+ "founded": [
155
+ {
156
+ "head": {
157
+ "text": "Alice",
158
+ "start": 0,
159
+ "end": 5,
160
+ "confidence": 0.613582193851471
161
+ },
162
+ "tail": {
163
+ "text": "Acme",
164
+ "start": 14,
165
+ "end": 18,
166
+ "confidence": 0.613582193851471
167
+ }
168
+ }
169
+ ]
170
+ }
171
+ },
172
+ "structure_match": true,
173
+ "maximum_confidence_error": 4.172325134277344e-07
174
+ },
175
+ {
176
+ "name": "relations_two",
177
+ "text": "Steve Jobs founded Apple.",
178
+ "native": {
179
+ "relation_extraction": {
180
+ "founded": [
181
+ {
182
+ "head": {
183
+ "text": "Steve Jobs",
184
+ "start": 0,
185
+ "end": 10,
186
+ "confidence": 0.6887430548667908
187
+ },
188
+ "tail": {
189
+ "text": "Apple",
190
+ "start": 19,
191
+ "end": 24,
192
+ "confidence": 0.6887430548667908
193
+ }
194
+ }
195
+ ]
196
+ }
197
+ },
198
+ "coreml": {
199
+ "relation_extraction": {
200
+ "founded": [
201
+ {
202
+ "head": {
203
+ "text": "Steve Jobs",
204
+ "start": 0,
205
+ "end": 10,
206
+ "confidence": 0.688742458820343
207
+ },
208
+ "tail": {
209
+ "text": "Apple",
210
+ "start": 19,
211
+ "end": 24,
212
+ "confidence": 0.688742458820343
213
+ }
214
+ }
215
+ ]
216
+ }
217
+ },
218
+ "structure_match": true,
219
+ "maximum_confidence_error": 5.960464477539062e-07
220
+ },
221
+ {
222
+ "name": "record_natural",
223
+ "text": "Alice works at Acme. Bob works at Beta.",
224
+ "native": {
225
+ "employment": [
226
+ {
227
+ "person": {
228
+ "text": "Alice",
229
+ "confidence": 0.9879767298698425,
230
+ "start": 0,
231
+ "end": 5
232
+ },
233
+ "company": {
234
+ "text": "Acme",
235
+ "confidence": 0.8241090774536133,
236
+ "start": 15,
237
+ "end": 19
238
+ }
239
+ },
240
+ {
241
+ "person": {
242
+ "text": "Bob",
243
+ "confidence": 0.9651215672492981,
244
+ "start": 21,
245
+ "end": 24
246
+ },
247
+ "company": {
248
+ "text": "Beta",
249
+ "confidence": 0.6678691506385803,
250
+ "start": 34,
251
+ "end": 38
252
+ }
253
+ }
254
+ ]
255
+ },
256
+ "coreml": {
257
+ "employment": [
258
+ {
259
+ "person": {
260
+ "text": "Alice",
261
+ "confidence": 0.9879767298698425,
262
+ "start": 0,
263
+ "end": 5
264
+ },
265
+ "company": {
266
+ "text": "Acme",
267
+ "confidence": 0.8241091370582581,
268
+ "start": 15,
269
+ "end": 19
270
+ }
271
+ },
272
+ {
273
+ "person": {
274
+ "text": "Bob",
275
+ "confidence": 0.9651216864585876,
276
+ "start": 21,
277
+ "end": 24
278
+ },
279
+ "company": {
280
+ "text": "Beta",
281
+ "confidence": 0.6678673028945923,
282
+ "start": 34,
283
+ "end": 38
284
+ }
285
+ }
286
+ ]
287
+ },
288
+ "structure_match": true,
289
+ "maximum_confidence_error": 1.8477439880371094e-06
290
+ },
291
+ {
292
+ "name": "record_latent",
293
+ "text": "Alice works at Acme. Bob works at Beta.",
294
+ "native": {
295
+ "employment": [
296
+ {
297
+ "person": null,
298
+ "company": {
299
+ "text": "Beta",
300
+ "confidence": 0.5839666128158569,
301
+ "start": 34,
302
+ "end": 38
303
+ }
304
+ },
305
+ {
306
+ "person": null,
307
+ "company": {
308
+ "text": "Acme",
309
+ "confidence": 0.6181439161300659,
310
+ "start": 15,
311
+ "end": 19
312
+ }
313
+ },
314
+ {
315
+ "person": {
316
+ "text": "Alice",
317
+ "confidence": 0.9672633409500122,
318
+ "start": 0,
319
+ "end": 5
320
+ },
321
+ "company": null
322
+ },
323
+ {
324
+ "person": null,
325
+ "company": {
326
+ "text": "Beta",
327
+ "confidence": 0.8805453777313232,
328
+ "start": 34,
329
+ "end": 38
330
+ }
331
+ },
332
+ {
333
+ "person": null,
334
+ "company": {
335
+ "text": "Beta",
336
+ "confidence": 0.8966390490531921,
337
+ "start": 34,
338
+ "end": 38
339
+ }
340
+ },
341
+ {
342
+ "person": null,
343
+ "company": {
344
+ "text": "Beta",
345
+ "confidence": 0.7147471904754639,
346
+ "start": 34,
347
+ "end": 38
348
+ }
349
+ },
350
+ {
351
+ "person": null,
352
+ "company": {
353
+ "text": "Acme",
354
+ "confidence": 0.6521790027618408,
355
+ "start": 15,
356
+ "end": 19
357
+ }
358
+ },
359
+ {
360
+ "person": {
361
+ "text": "Bob",
362
+ "confidence": 0.5080211162567139,
363
+ "start": 21,
364
+ "end": 24
365
+ },
366
+ "company": null
367
+ },
368
+ {
369
+ "person": null,
370
+ "company": {
371
+ "text": "Beta",
372
+ "confidence": 0.5336229205131531,
373
+ "start": 34,
374
+ "end": 38
375
+ }
376
+ },
377
+ {
378
+ "person": null,
379
+ "company": {
380
+ "text": "Acme",
381
+ "confidence": 0.6106622815132141,
382
+ "start": 15,
383
+ "end": 19
384
+ }
385
+ }
386
+ ]
387
+ },
388
+ "coreml": {
389
+ "employment": [
390
+ {
391
+ "person": null,
392
+ "company": {
393
+ "text": "Beta",
394
+ "confidence": 0.5839619040489197,
395
+ "start": 34,
396
+ "end": 38
397
+ }
398
+ },
399
+ {
400
+ "person": null,
401
+ "company": {
402
+ "text": "Acme",
403
+ "confidence": 0.6181442141532898,
404
+ "start": 15,
405
+ "end": 19
406
+ }
407
+ },
408
+ {
409
+ "person": {
410
+ "text": "Alice",
411
+ "confidence": 0.9672635793685913,
412
+ "start": 0,
413
+ "end": 5
414
+ },
415
+ "company": null
416
+ },
417
+ {
418
+ "person": null,
419
+ "company": {
420
+ "text": "Beta",
421
+ "confidence": 0.8805445432662964,
422
+ "start": 34,
423
+ "end": 38
424
+ }
425
+ },
426
+ {
427
+ "person": null,
428
+ "company": {
429
+ "text": "Beta",
430
+ "confidence": 0.8966377973556519,
431
+ "start": 34,
432
+ "end": 38
433
+ }
434
+ },
435
+ {
436
+ "person": null,
437
+ "company": {
438
+ "text": "Beta",
439
+ "confidence": 0.7147423028945923,
440
+ "start": 34,
441
+ "end": 38
442
+ }
443
+ },
444
+ {
445
+ "person": null,
446
+ "company": {
447
+ "text": "Acme",
448
+ "confidence": 0.6521808505058289,
449
+ "start": 15,
450
+ "end": 19
451
+ }
452
+ },
453
+ {
454
+ "person": {
455
+ "text": "Bob",
456
+ "confidence": 0.508021354675293,
457
+ "start": 21,
458
+ "end": 24
459
+ },
460
+ "company": null
461
+ },
462
+ {
463
+ "person": null,
464
+ "company": {
465
+ "text": "Beta",
466
+ "confidence": 0.5336205363273621,
467
+ "start": 34,
468
+ "end": 38
469
+ }
470
+ },
471
+ {
472
+ "person": null,
473
+ "company": {
474
+ "text": "Acme",
475
+ "confidence": 0.6106586456298828,
476
+ "start": 15,
477
+ "end": 19
478
+ }
479
+ }
480
+ ]
481
+ },
482
+ "structure_match": true,
483
+ "maximum_confidence_error": 4.887580871582031e-06
484
+ },
485
+ {
486
+ "name": "record_anchorless",
487
+ "text": "Alice works at Acme. Bob works at Beta.",
488
+ "native": {},
489
+ "coreml": {},
490
+ "structure_match": true,
491
+ "maximum_confidence_error": null
492
+ },
493
+ {
494
+ "name": "attributes",
495
+ "text": "Alice founded Acme.",
496
+ "native": {
497
+ "entities": {
498
+ "person": [
499
+ {
500
+ "text": "Alice",
501
+ "confidence": 0.9935710430145264,
502
+ "start": 0,
503
+ "end": 5,
504
+ "role": {
505
+ "label": "founder",
506
+ "confidence": 0.999847412109375
507
+ }
508
+ }
509
+ ],
510
+ "organization": [
511
+ {
512
+ "text": "Acme",
513
+ "confidence": 0.9912181496620178,
514
+ "start": 14,
515
+ "end": 18,
516
+ "role": {
517
+ "label": "founder",
518
+ "confidence": 0.9899052977561951
519
+ }
520
+ }
521
+ ]
522
+ }
523
+ },
524
+ "coreml": {
525
+ "entities": {
526
+ "person": [
527
+ {
528
+ "text": "Alice",
529
+ "confidence": 0.9935709238052368,
530
+ "start": 0,
531
+ "end": 5,
532
+ "role": {
533
+ "label": "founder",
534
+ "confidence": 0.999847412109375
535
+ }
536
+ }
537
+ ],
538
+ "organization": [
539
+ {
540
+ "text": "Acme",
541
+ "confidence": 0.9912181496620178,
542
+ "start": 14,
543
+ "end": 18,
544
+ "role": {
545
+ "label": "founder",
546
+ "confidence": 0.9899055361747742
547
+ }
548
+ }
549
+ ]
550
+ }
551
+ },
552
+ "structure_match": true,
553
+ "maximum_confidence_error": 2.384185791015625e-07
554
+ },
555
+ {
556
+ "name": "mixed_classification",
557
+ "text": "Alice founded Acme.",
558
+ "native": {
559
+ "entities": {
560
+ "person": [
561
+ {
562
+ "text": "Alice",
563
+ "confidence": 0.9987186193466187,
564
+ "start": 0,
565
+ "end": 5
566
+ }
567
+ ],
568
+ "organization": [
569
+ {
570
+ "text": "Acme",
571
+ "confidence": 0.9962791800498962,
572
+ "start": 14,
573
+ "end": 18
574
+ }
575
+ ]
576
+ },
577
+ "sentiment": {
578
+ "label": "positive",
579
+ "confidence": 0.9953342080116272
580
+ }
581
+ },
582
+ "coreml": {
583
+ "entities": {
584
+ "person": [
585
+ {
586
+ "text": "Alice",
587
+ "confidence": 0.9987186193466187,
588
+ "start": 0,
589
+ "end": 5
590
+ }
591
+ ],
592
+ "organization": [
593
+ {
594
+ "text": "Acme",
595
+ "confidence": 0.9962791800498962,
596
+ "start": 14,
597
+ "end": 18
598
+ }
599
+ ]
600
+ },
601
+ "sentiment": {
602
+ "label": "positive",
603
+ "confidence": 0.9953340888023376
604
+ }
605
+ },
606
+ "structure_match": true,
607
+ "maximum_confidence_error": 1.1920928955078125e-07
608
+ },
609
+ {
610
+ "name": "classification_only",
611
+ "text": "The rocket launched successfully.",
612
+ "native": {
613
+ "topic": {
614
+ "label": "sports",
615
+ "confidence": 0.5060790777206421
616
+ }
617
+ },
618
+ "coreml": {
619
+ "topic": {
620
+ "label": "sports",
621
+ "confidence": 0.50607830286026
622
+ }
623
+ },
624
+ "structure_match": true,
625
+ "maximum_confidence_error": 7.748603820800781e-07
626
+ },
627
+ {
628
+ "name": "enum_choice",
629
+ "text": "The item is electronics.",
630
+ "native": {
631
+ "product": [
632
+ {
633
+ "category": {
634
+ "text": "electronics",
635
+ "confidence": 0.9997705817222595
636
+ }
637
+ }
638
+ ]
639
+ },
640
+ "coreml": {
641
+ "product": [
642
+ {
643
+ "category": {
644
+ "text": "electronics",
645
+ "confidence": 0.9997705817222595
646
+ }
647
+ }
648
+ ]
649
+ },
650
+ "structure_match": true,
651
+ "maximum_confidence_error": 0.0
652
+ },
653
+ {
654
+ "name": "es_entities",
655
+ "text": "María trabaja en Madrid para Telefónica.",
656
+ "native": {
657
+ "entities": {
658
+ "persona": [
659
+ {
660
+ "text": "María",
661
+ "confidence": 0.997144877910614,
662
+ "start": 0,
663
+ "end": 5
664
+ }
665
+ ],
666
+ "organización": [
667
+ {
668
+ "text": "Telefónica",
669
+ "confidence": 0.9903497695922852,
670
+ "start": 29,
671
+ "end": 39
672
+ }
673
+ ],
674
+ "ciudad": [
675
+ {
676
+ "text": "Madrid",
677
+ "confidence": 0.9995064735412598,
678
+ "start": 17,
679
+ "end": 23
680
+ }
681
+ ]
682
+ }
683
+ },
684
+ "coreml": {
685
+ "entities": {
686
+ "persona": [
687
+ {
688
+ "text": "María",
689
+ "confidence": 0.9971449971199036,
690
+ "start": 0,
691
+ "end": 5
692
+ }
693
+ ],
694
+ "organización": [
695
+ {
696
+ "text": "Telefónica",
697
+ "confidence": 0.9903498888015747,
698
+ "start": 29,
699
+ "end": 39
700
+ }
701
+ ],
702
+ "ciudad": [
703
+ {
704
+ "text": "Madrid",
705
+ "confidence": 0.9995064735412598,
706
+ "start": 17,
707
+ "end": 23
708
+ }
709
+ ]
710
+ }
711
+ },
712
+ "structure_match": true,
713
+ "maximum_confidence_error": 1.1920928955078125e-07
714
+ },
715
+ {
716
+ "name": "fr_entities",
717
+ "text": "Jean Dupont travaille à Paris pour Airbus.",
718
+ "native": {
719
+ "entities": {
720
+ "personne": [
721
+ {
722
+ "text": "Jean Dupont",
723
+ "confidence": 0.9973317384719849,
724
+ "start": 0,
725
+ "end": 11
726
+ }
727
+ ],
728
+ "entreprise": [
729
+ {
730
+ "text": "Airbus",
731
+ "confidence": 0.9904603958129883,
732
+ "start": 35,
733
+ "end": 41
734
+ }
735
+ ],
736
+ "ville": [
737
+ {
738
+ "text": "Paris",
739
+ "confidence": 0.9991673231124878,
740
+ "start": 24,
741
+ "end": 29
742
+ }
743
+ ]
744
+ }
745
+ },
746
+ "coreml": {
747
+ "entities": {
748
+ "personne": [
749
+ {
750
+ "text": "Jean Dupont",
751
+ "confidence": 0.9973318576812744,
752
+ "start": 0,
753
+ "end": 11
754
+ }
755
+ ],
756
+ "entreprise": [
757
+ {
758
+ "text": "Airbus",
759
+ "confidence": 0.9904603958129883,
760
+ "start": 35,
761
+ "end": 41
762
+ }
763
+ ],
764
+ "ville": [
765
+ {
766
+ "text": "Paris",
767
+ "confidence": 0.9991673231124878,
768
+ "start": 24,
769
+ "end": 29
770
+ }
771
+ ]
772
+ }
773
+ },
774
+ "structure_match": true,
775
+ "maximum_confidence_error": 1.1920928955078125e-07
776
+ },
777
+ {
778
+ "name": "zh_entities",
779
+ "text": "张伟在北京大学工作。",
780
+ "native": {
781
+ "entities": {
782
+ "人物": [
783
+ {
784
+ "text": "张伟在北京大学工作",
785
+ "confidence": 0.9425729513168335,
786
+ "start": 0,
787
+ "end": 9
788
+ }
789
+ ],
790
+ "机构": [],
791
+ "地点": []
792
+ }
793
+ },
794
+ "coreml": {
795
+ "entities": {
796
+ "人物": [
797
+ {
798
+ "text": "张伟在北京大学工作",
799
+ "confidence": 0.9425737857818604,
800
+ "start": 0,
801
+ "end": 9
802
+ }
803
+ ],
804
+ "��构": [],
805
+ "地点": []
806
+ }
807
+ },
808
+ "structure_match": true,
809
+ "maximum_confidence_error": 8.344650268554688e-07
810
+ },
811
+ {
812
+ "name": "de_entities",
813
+ "text": "Angela Merkel besuchte Berlin.",
814
+ "native": {
815
+ "entities": {
816
+ "Person": [
817
+ {
818
+ "text": "Angela Merkel",
819
+ "confidence": 0.9966775178909302,
820
+ "start": 0,
821
+ "end": 13
822
+ }
823
+ ],
824
+ "Stadt": [
825
+ {
826
+ "text": "Berlin",
827
+ "confidence": 0.9987296462059021,
828
+ "start": 23,
829
+ "end": 29
830
+ }
831
+ ]
832
+ }
833
+ },
834
+ "coreml": {
835
+ "entities": {
836
+ "Person": [
837
+ {
838
+ "text": "Angela Merkel",
839
+ "confidence": 0.9966775178909302,
840
+ "start": 0,
841
+ "end": 13
842
+ }
843
+ ],
844
+ "Stadt": [
845
+ {
846
+ "text": "Berlin",
847
+ "confidence": 0.9987296462059021,
848
+ "start": 23,
849
+ "end": 29
850
+ }
851
+ ]
852
+ }
853
+ },
854
+ "structure_match": true,
855
+ "maximum_confidence_error": 0.0
856
+ }
857
+ ]
858
+ }
tests/test_extraction_export.py ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Real-checkpoint parity for trained extraction graph wrappers."""
2
+
3
+ from pathlib import Path
4
+
5
+ import torch
6
+ from gliner2 import AutoExtractor, Schema
7
+ from gliner2.training.trainer import ExtractorCollator
8
+
9
+ from extraction_export import ExtractionFeaturesExport, ExtractionScoreExport, coreml_trace_patches
10
+ from extraction_pool import select_candidates
11
+ from preprocessing import prepare_extraction
12
+
13
+ SOURCE = (
14
+ Path.home()
15
+ / ".cache/huggingface/hub/models--fastino--gliner2.5-multi-v1/snapshots/a221b77a8baf4a613b8f8652661d41fa10a5641e"
16
+ )
17
+
18
+
19
+ def test_export_wrappers_match_native_entity_scores():
20
+ torch.set_num_threads(4)
21
+ native = AutoExtractor.from_pretrained(str(SOURCE), map_location="cpu").eval()
22
+ text = "Alice founded Acme in Toronto in 2020."
23
+ schema = Schema().entities(["person", "organization", "location"])
24
+ batch = ExtractorCollator(native.processor, is_training=False, max_len=128, architecture=native.architecture)(
25
+ [(text, schema.build())]
26
+ )
27
+ arrays, _ = prepare_extraction(native.processor, text, schema, 128, 64, 8)
28
+ arguments = tuple(torch.from_numpy(value) for value in arrays.values())
29
+ with torch.no_grad():
30
+ core = native._encode_core(batch)
31
+ expected = native.boundary_head(
32
+ core["text_states"], core["text_mask"], core["query_states"], core["query_mask"]
33
+ )
34
+ with coreml_trace_patches():
35
+ features = ExtractionFeaturesExport(native).eval()(*arguments)
36
+ assert torch.allclose(features[0][:, : core["text_states"].shape[1]], core["text_states"], atol=1e-5)
37
+ assert torch.allclose(features[1][:, : core["query_states"].shape[1]], core["query_states"], atol=1e-5)
38
+ pool = select_candidates(
39
+ features[8],
40
+ features[9],
41
+ features[3].bool(),
42
+ arguments[5].bool(),
43
+ features[4],
44
+ features[5],
45
+ boundary_top_k=native.boundary_head.shared_pool_builder.pool_boundary_top_k,
46
+ pool_size=native.boundary_head.shared_pool_builder.pool_size,
47
+ min_pool_per_query=native.boundary_head.shared_pool_builder.min_pool_per_query,
48
+ )
49
+ assert torch.equal(
50
+ pool.indices.unsqueeze(1).expand_as(expected.candidates.indices), expected.candidates.indices
51
+ )
52
+ scores, candidate_states = ExtractionScoreExport(native).eval()(
53
+ features[0],
54
+ arguments[3],
55
+ features[1],
56
+ arguments[5],
57
+ features[2],
58
+ features[4],
59
+ features[5],
60
+ features[6],
61
+ features[7],
62
+ pool.indices.int(),
63
+ pool.mask.float(),
64
+ pool.compat_logits,
65
+ )
66
+ assert torch.allclose(scores[:, : core["query_states"].shape[1]], expected.candidates.pair_logits, atol=1e-4)
67
+ assert torch.allclose(candidate_states.unsqueeze(1), expected.candidates.candidate_states[:, :1], atol=1e-4)
tests/test_extraction_pool.py ADDED
@@ -0,0 +1,62 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Real-checkpoint parity for the weight-free candidate selection stage."""
2
+
3
+ from pathlib import Path
4
+
5
+ import torch
6
+ from gliner2 import AutoExtractor, Schema
7
+ from gliner2.training.trainer import ExtractorCollator
8
+
9
+ from extraction_pool import select_candidates
10
+
11
+ SOURCE = (
12
+ Path.home()
13
+ / ".cache/huggingface/hub/models--fastino--gliner2.5-multi-v1/snapshots/a221b77a8baf4a613b8f8652661d41fa10a5641e"
14
+ )
15
+
16
+
17
+ def test_candidate_pool_matches_native_with_real_weights():
18
+ torch.set_num_threads(4)
19
+ native = AutoExtractor.from_pretrained(str(SOURCE), map_location="cpu").eval()
20
+ fixtures = [
21
+ ("Alice founded Acme in Toronto in 2020.", ["person", "organization", "location"]),
22
+ ("Apple acquired Beats for three billion dollars.", ["company", "product", "money"]),
23
+ ]
24
+ for text, labels in fixtures:
25
+ schema = Schema().entities(labels)
26
+ batch = ExtractorCollator(native.processor, is_training=False, max_len=128, architecture=native.architecture)(
27
+ [(text, schema.build())]
28
+ )
29
+ with torch.no_grad():
30
+ core = native._encode_core(batch)
31
+ head = native.boundary_head
32
+ encoded = head.boundary_encoder(core["text_states"], core["text_mask"])
33
+ marginals = head.boundary_query_head(
34
+ encoded.states,
35
+ encoded.mask,
36
+ core["text_states"],
37
+ core["text_mask"],
38
+ core["query_states"],
39
+ core["query_mask"],
40
+ )
41
+ expected = head.shared_pool_builder(
42
+ encoded.states,
43
+ encoded.mask,
44
+ core["query_mask"],
45
+ marginals.start_logits,
46
+ marginals.end_logits,
47
+ )
48
+ actual = select_candidates(
49
+ head.shared_pool_builder.start_projection(encoded.states),
50
+ head.shared_pool_builder.end_projection(encoded.states),
51
+ encoded.mask,
52
+ core["query_mask"],
53
+ marginals.start_logits,
54
+ marginals.end_logits,
55
+ boundary_top_k=head.shared_pool_builder.pool_boundary_top_k,
56
+ pool_size=head.shared_pool_builder.pool_size,
57
+ min_pool_per_query=head.shared_pool_builder.min_pool_per_query,
58
+ )
59
+ assert torch.equal(actual.indices, expected.indices)
60
+ assert torch.equal(actual.mask, expected.mask)
61
+ assert torch.allclose(actual.compat_logits, expected.compat_logits, atol=1e-5)
62
+ assert torch.allclose(actual.proposal_logits, expected.proposal_logits, atol=1e-5)
tests/test_extraction_preprocessing.py ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Capacity checks for the real pinned GLiNER2 tokenizer and schema."""
2
+
3
+ from pathlib import Path
4
+
5
+ import pytest
6
+ from gliner2 import Schema
7
+
8
+ from preprocessing import load_processor, prepare_extraction
9
+
10
+ SOURCE = (
11
+ Path.home()
12
+ / ".cache/huggingface/hub/models--fastino--gliner2.5-multi-v1/snapshots/a221b77a8baf4a613b8f8652661d41fa10a5641e"
13
+ )
14
+
15
+
16
+ def test_extraction_keeps_all_queries_and_offsets():
17
+ processor = load_processor(str(SOURCE))
18
+ text = "Alice founded Acme in Toronto in 2020."
19
+ arrays, batch = prepare_extraction(
20
+ processor, text, Schema().entities(["person", "organization", "location"]), 128, 64, 8
21
+ )
22
+ assert int(arrays["query_mask"].sum()) == 3
23
+ assert int(arrays["text_mask"].sum()) == len(batch.start_mappings[0])
24
+ assert text[batch.start_mappings[0][0] : batch.end_mappings[0][0]] == "Alice"
25
+
26
+
27
+ @pytest.mark.parametrize("bucket", [(10, 64, 8), (128, 2, 8), (128, 64, 2)])
28
+ def test_extraction_rejects_capacity_exceeded(bucket):
29
+ processor = load_processor(str(SOURCE))
30
+ with pytest.raises(ValueError, match="bucket holds"):
31
+ prepare_extraction(
32
+ processor,
33
+ "Alice founded Acme in Toronto in 2020.",
34
+ Schema().entities(["person", "organization", "location"]),
35
+ *bucket,
36
+ )
37
+
38
+
39
+ def test_extraction_rejects_classification_choice_capacity():
40
+ processor = load_processor(str(SOURCE))
41
+ choices = [chr(ord("a") + index) for index in range(9)]
42
+ with pytest.raises(ValueError, match="choices; bucket holds 8"):
43
+ prepare_extraction(processor, "A short report.", Schema().classification("topic", choices), 128, 64, 8)
tests/test_extraction_record.py ADDED
@@ -0,0 +1,66 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Real-checkpoint parity of all three native record formation modes."""
2
+
3
+ from pathlib import Path
4
+
5
+ import torch
6
+ from gliner2 import AutoExtractor, Schema
7
+ from gliner2.training.trainer import ExtractorCollator
8
+
9
+ from extraction_export import ExtractionRecordAnchorlessExport, ExtractionRecordAssignmentExport
10
+
11
+ SOURCE = (
12
+ Path.home()
13
+ / ".cache/huggingface/hub/models--fastino--gliner2.5-multi-v1/snapshots/a221b77a8baf4a613b8f8652661d41fa10a5641e"
14
+ )
15
+
16
+
17
+ def test_record_heads_match_native_natural_latent_and_anchorless():
18
+ torch.set_num_threads(4)
19
+ native = AutoExtractor.from_pretrained(str(SOURCE), map_location="cpu").eval()
20
+ text = "Alice works at Acme. Bob works at Beta."
21
+ for mode in ("natural", "latent", "anchorless"):
22
+ schema = Schema()
23
+ builder = schema.structure("employment", mode=mode, anchor="person" if mode == "natural" else None)
24
+ builder.field("person", dtype="str")
25
+ builder.field("company", dtype="str")
26
+ batch = ExtractorCollator(native.processor, is_training=False, max_len=None, architecture="boundary")(
27
+ [(text, schema.build())]
28
+ )
29
+ with torch.no_grad():
30
+ core = native._encode_core(batch)
31
+ candidates = native.boundary_head(
32
+ core["text_states"], core["text_mask"], core["query_states"], core["query_mask"]
33
+ ).candidates
34
+ spec = next(iter(batch.record_specs[0].values()))
35
+ group = native.record_decoder.forward_group(spec, core["query_states"][0], candidates, 0)
36
+ field_states = []
37
+ for query_id in group.field_query_ids:
38
+ keep = candidates.valid_mask[0, query_id]
39
+ field_states.append(candidates.candidate_states[0, query_id][keep])
40
+ field_queries = core["query_states"][0][group.field_query_ids]
41
+ max_candidates = max(states.shape[0] for states in field_states)
42
+ padded_fields = torch.stack(
43
+ [
44
+ torch.nn.functional.pad(states, (0, 0, 0, max_candidates - states.shape[0]))
45
+ for states in field_states
46
+ ]
47
+ )
48
+ if mode == "natural":
49
+ anchor_index = group.field_query_ids.index(spec.anchor_query_id)
50
+ instances = field_states[anchor_index]
51
+ elif mode == "latent":
52
+ instances = torch.cat(field_states, 0)
53
+ else:
54
+ instances = native.record_decoder._anchorless_states(field_states)
55
+ context = torch.cat(field_states, 0)
56
+ predicted = ExtractionRecordAnchorlessExport(native).eval()(context, torch.ones(context.shape[0]))
57
+ assert torch.allclose(predicted, instances, atol=1e-5)
58
+ assignment, object_scores, latent_scores = ExtractionRecordAssignmentExport(native).eval()(
59
+ instances, field_queries, padded_fields
60
+ )
61
+ for field_index, expected in enumerate(group.assign_logits):
62
+ assert torch.allclose(assignment[:, field_index, : expected.shape[1]], expected, atol=1e-5)
63
+ if mode == "anchorless":
64
+ assert torch.allclose(object_scores, group.object_logits, atol=1e-5)
65
+ if mode == "latent":
66
+ assert torch.allclose(latent_scores, group.object_logits, atol=1e-5)
uv.lock CHANGED
@@ -101,6 +101,15 @@ wheels = [
101
  { url = "https://files.pythonhosted.org/packages/cc/61/d01fc49b8dea277640b55a9e15960dbca9fdc8c9fde18e572d39c59f4019/charset_normalizer-3.5.1-py3-none-any.whl", hash = "sha256:6df0ec430f9a831772c23ca5a224cba36517a58a84bb32c32bb59a9fa67c47f6", size = 68658, upload-time = "2026-08-15T08:20:43.306Z" },
102
  ]
103
 
 
 
 
 
 
 
 
 
 
104
  [[package]]
105
  name = "colorama"
106
  version = "0.4.6"
@@ -183,12 +192,16 @@ dependencies = [
183
  { name = "huggingface-hub" },
184
  { name = "numpy" },
185
  { name = "protobuf" },
 
186
  { name = "sentencepiece" },
187
  { name = "torch" },
188
  { name = "transformers" },
189
  ]
190
 
191
  [package.dev-dependencies]
 
 
 
192
  dev = [
193
  { name = "pytest" },
194
  { name = "ruff" },
@@ -201,12 +214,14 @@ requires-dist = [
201
  { name = "huggingface-hub", specifier = ">=0.34,<1" },
202
  { name = "numpy", specifier = "<2.3" },
203
  { name = "protobuf", specifier = ">=5,<7" },
 
204
  { name = "sentencepiece", specifier = ">=0.2,<0.3" },
205
  { name = "torch", specifier = "==2.7.0" },
206
  { name = "transformers", specifier = "==4.57.6" },
207
  ]
208
 
209
  [package.metadata.requires-dev]
 
210
  dev = [
211
  { name = "pytest", specifier = ">=8.4" },
212
  { name = "ruff", specifier = ">=0.13" },
@@ -277,6 +292,18 @@ wheels = [
277
  { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" },
278
  ]
279
 
 
 
 
 
 
 
 
 
 
 
 
 
280
  [[package]]
281
  name = "markupsafe"
282
  version = "3.0.3"
@@ -723,6 +750,46 @@ wheels = [
723
  { url = "https://files.pythonhosted.org/packages/8d/fc/7eedc3510d97878876e32774eebbeb61c43f148a96e915c84229a3e967aa/safetensors-0.8.0-cp310-abi3-win_arm64.whl", hash = "sha256:f7838e5135a406ad3e02efdcb8cf2e5397d368b0154537c4fec682dbc544d452", size = 340500, upload-time = "2026-06-09T07:52:26.745Z" },
724
  ]
725
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
726
  [[package]]
727
  name = "sentencepiece"
728
  version = "0.2.2"
@@ -759,6 +826,15 @@ wheels = [
759
  { url = "https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl", hash = "sha256:e091cc3e99d2141a0ba2847328f5479b05d94a6635cb96148ccb3f34671bd8f5", size = 6299353, upload-time = "2025-04-27T18:04:59.103Z" },
760
  ]
761
 
 
 
 
 
 
 
 
 
 
762
  [[package]]
763
  name = "tokenizers"
764
  version = "0.22.2"
 
101
  { url = "https://files.pythonhosted.org/packages/cc/61/d01fc49b8dea277640b55a9e15960dbca9fdc8c9fde18e572d39c59f4019/charset_normalizer-3.5.1-py3-none-any.whl", hash = "sha256:6df0ec430f9a831772c23ca5a224cba36517a58a84bb32c32bb59a9fa67c47f6", size = 68658, upload-time = "2026-08-15T08:20:43.306Z" },
102
  ]
103
 
104
+ [[package]]
105
+ name = "cloudpickle"
106
+ version = "3.1.2"
107
+ source = { registry = "https://pypi.org/simple" }
108
+ sdist = { url = "https://files.pythonhosted.org/packages/27/fb/576f067976d320f5f0114a8d9fa1215425441bb35627b1993e5afd8111e5/cloudpickle-3.1.2.tar.gz", hash = "sha256:7fda9eb655c9c230dab534f1983763de5835249750e85fbcef43aaa30a9a2414", size = 22330, upload-time = "2025-11-03T09:25:26.604Z" }
109
+ wheels = [
110
+ { url = "https://files.pythonhosted.org/packages/88/39/799be3f2f0f38cc727ee3b4f1445fe6d5e4133064ec2e4115069418a5bb6/cloudpickle-3.1.2-py3-none-any.whl", hash = "sha256:9acb47f6afd73f60dc1df93bb801b472f05ff42fa6c84167d25cb206be1fbf4a", size = 22228, upload-time = "2025-11-03T09:25:25.534Z" },
111
+ ]
112
+
113
  [[package]]
114
  name = "colorama"
115
  version = "0.4.6"
 
192
  { name = "huggingface-hub" },
193
  { name = "numpy" },
194
  { name = "protobuf" },
195
+ { name = "psutil" },
196
  { name = "sentencepiece" },
197
  { name = "torch" },
198
  { name = "transformers" },
199
  ]
200
 
201
  [package.dev-dependencies]
202
+ compression = [
203
+ { name = "scikit-learn" },
204
+ ]
205
  dev = [
206
  { name = "pytest" },
207
  { name = "ruff" },
 
214
  { name = "huggingface-hub", specifier = ">=0.34,<1" },
215
  { name = "numpy", specifier = "<2.3" },
216
  { name = "protobuf", specifier = ">=5,<7" },
217
+ { name = "psutil", specifier = ">=7,<8" },
218
  { name = "sentencepiece", specifier = ">=0.2,<0.3" },
219
  { name = "torch", specifier = "==2.7.0" },
220
  { name = "transformers", specifier = "==4.57.6" },
221
  ]
222
 
223
  [package.metadata.requires-dev]
224
+ compression = [{ name = "scikit-learn", specifier = "==1.5.1" }]
225
  dev = [
226
  { name = "pytest", specifier = ">=8.4" },
227
  { name = "ruff", specifier = ">=0.13" },
 
292
  { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" },
293
  ]
294
 
295
+ [[package]]
296
+ name = "joblib"
297
+ version = "1.6.0"
298
+ source = { registry = "https://pypi.org/simple" }
299
+ dependencies = [
300
+ { name = "cloudpickle" },
301
+ ]
302
+ sdist = { url = "https://files.pythonhosted.org/packages/d5/1d/537ab090f302b838943a1b56497dd53059b9a9b46a074936470173a2e207/joblib-1.6.0.tar.gz", hash = "sha256:2ccc96785b12046c08fd6d55839c12857831b54a3c1673ffadd2f04bfc4eda03", size = 327903, upload-time = "2026-08-31T09:39:04.122Z" }
303
+ wheels = [
304
+ { url = "https://files.pythonhosted.org/packages/18/53/84099323c2ec4be98d935f63c033ac4151ee83836ca1050ede3b3aadf155/joblib-1.6.0-py3-none-any.whl", hash = "sha256:3dbbf9f6e4b592a2357b854608e980fe6390d131d7a82f011a377ef2ebef7aba", size = 306115, upload-time = "2026-08-31T09:39:02.298Z" },
305
+ ]
306
+
307
  [[package]]
308
  name = "markupsafe"
309
  version = "3.0.3"
 
750
  { url = "https://files.pythonhosted.org/packages/8d/fc/7eedc3510d97878876e32774eebbeb61c43f148a96e915c84229a3e967aa/safetensors-0.8.0-cp310-abi3-win_arm64.whl", hash = "sha256:f7838e5135a406ad3e02efdcb8cf2e5397d368b0154537c4fec682dbc544d452", size = 340500, upload-time = "2026-06-09T07:52:26.745Z" },
751
  ]
752
 
753
+ [[package]]
754
+ name = "scikit-learn"
755
+ version = "1.5.1"
756
+ source = { registry = "https://pypi.org/simple" }
757
+ dependencies = [
758
+ { name = "joblib" },
759
+ { name = "numpy" },
760
+ { name = "scipy" },
761
+ { name = "threadpoolctl" },
762
+ ]
763
+ sdist = { url = "https://files.pythonhosted.org/packages/92/72/2961b9874a9ddf2b0f95f329d4e67f67c3301c1d88ba5e239ff25661bb85/scikit_learn-1.5.1.tar.gz", hash = "sha256:0ea5d40c0e3951df445721927448755d3fe1d80833b0b7308ebff5d2a45e6414", size = 6958368, upload-time = "2024-07-03T09:12:21.925Z" }
764
+ wheels = [
765
+ { url = "https://files.pythonhosted.org/packages/b1/8d/cf392a56e24627093a467642c8b9263052372131359b570df29aaf4811ab/scikit_learn-1.5.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:5944ce1faada31c55fb2ba20a5346b88e36811aab504ccafb9f0339e9f780395", size = 12102404, upload-time = "2024-07-03T09:11:46.261Z" },
766
+ { url = "https://files.pythonhosted.org/packages/d5/2c/734fc9269bdb6768905ac41b82d75264b26925b1e462f4ebf45fe4f17646/scikit_learn-1.5.1-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:0828673c5b520e879f2af6a9e99eee0eefea69a2188be1ca68a6121b809055c1", size = 11037398, upload-time = "2024-07-03T09:11:49.783Z" },
767
+ { url = "https://files.pythonhosted.org/packages/d3/a9/15774b178bcd1cde1c470adbdb554e1504dce7c302e02ff736c90d65e014/scikit_learn-1.5.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:508907e5f81390e16d754e8815f7497e52139162fd69c4fdbd2dfa5d6cc88915", size = 12089887, upload-time = "2024-07-03T09:11:53.134Z" },
768
+ { url = "https://files.pythonhosted.org/packages/8a/5d/047cde25131eef3a38d03317fa7d25d6f60ce6e8ccfd24ac88b3e309fc00/scikit_learn-1.5.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:97625f217c5c0c5d0505fa2af28ae424bd37949bb2f16ace3ff5f2f81fb4498b", size = 13079093, upload-time = "2024-07-03T09:11:55.93Z" },
769
+ { url = "https://files.pythonhosted.org/packages/cb/be/dec2a8d31d133034a8ec51ae68ac564ec9bde1c78a64551f1438c3690b9e/scikit_learn-1.5.1-cp312-cp312-win_amd64.whl", hash = "sha256:da3f404e9e284d2b0a157e1b56b6566a34eb2798205cba35a211df3296ab7a74", size = 10945350, upload-time = "2024-07-03T09:12:02.494Z" },
770
+ ]
771
+
772
+ [[package]]
773
+ name = "scipy"
774
+ version = "1.18.1"
775
+ source = { registry = "https://pypi.org/simple" }
776
+ dependencies = [
777
+ { name = "numpy" },
778
+ ]
779
+ sdist = { url = "https://files.pythonhosted.org/packages/7e/74/66de6258867beb2ef08f35f9f2ac017a52cacd5081714d239ff1a442d458/scipy-1.18.1.tar.gz", hash = "sha256:52c4b7422442aba924d03ad4019852b08a92e64ea187b933135687bfe2747307", size = 30781235, upload-time = "2026-08-21T23:28:50.599Z" }
780
+ wheels = [
781
+ { url = "https://files.pythonhosted.org/packages/18/f7/240c110c08693826b4513a52f5717d62ec7c7af72f2920821247c03b17b3/scipy-1.18.1-cp312-cp312-macosx_10_15_x86_64.whl", hash = "sha256:457fd7a2a8edeb044ab6ffbc0aa03ff6cd18491356e5e0c834d76ce621b916d1", size = 31111061, upload-time = "2026-08-21T23:23:44.522Z" },
782
+ { url = "https://files.pythonhosted.org/packages/05/4a/78c6285577c375e7cf27277ea8ee6961224327f1e1a0c44af5f17f23635c/scipy-1.18.1-cp312-cp312-macosx_12_0_arm64.whl", hash = "sha256:e708533e8b2ae2497d65346538a7dcc92814410b25b81432eac66de0f2af8265", size = 28733332, upload-time = "2026-08-21T23:23:50.015Z" },
783
+ { url = "https://files.pythonhosted.org/packages/a5/f6/a5b82f8abbe14d134691b8b903696f701d25a081353a29dc655c364d9e62/scipy-1.18.1-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:7bbf207c4453ce1ad2e00b17313852b33310b83090c2311bdaf97f93c0380d12", size = 20475078, upload-time = "2026-08-21T23:23:54.138Z" },
784
+ { url = "https://files.pythonhosted.org/packages/23/22/0858a0bbd6b3e825ceb8cd9baf9eaf3b2f2b1d77727eb6be40500bcdc92f/scipy-1.18.1-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:78c0665edead396b1abb4897c41a5c1d9bf090c8a637a4c20a61678e0a264e66", size = 23108904, upload-time = "2026-08-21T23:23:57.824Z" },
785
+ { url = "https://files.pythonhosted.org/packages/75/9a/2e71719f31eaefe0e3a1706c4a1ded94e664bfd95ffca2b219a671faee01/scipy-1.18.1-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3c085faa2cfa879c5141df483f836f4d691045a078224a670fa570fa01612d89", size = 34025113, upload-time = "2026-08-21T23:24:02.209Z" },
786
+ { url = "https://files.pythonhosted.org/packages/df/64/ff35eb9e54894cf471ff4716abd3c81eb0a0626869217ce3e6ba4ccf17d7/scipy-1.18.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f55fa87b6c612ecd6b058f167c53231b1d14e412efe361d3d6e38b3631c73218", size = 35344199, upload-time = "2026-08-21T23:24:07.844Z" },
787
+ { url = "https://files.pythonhosted.org/packages/d3/af/c5538be1792f7034c12c7db6ee67cace58253c7b87b122d68253eaf5de89/scipy-1.18.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:c35d74ce0e193ff740c2f2be2ac913ddc232fe6c1ff40b26cfecb9c670c63314", size = 35639587, upload-time = "2026-08-21T23:24:13.05Z" },
788
+ { url = "https://files.pythonhosted.org/packages/91/4c/075e4f66471bac101141ac739e9e135549be1bae584571bd03a530c056e1/scipy-1.18.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:d2924a03db38dc2e848bca2fe9f077dafb891480b91a00a0963a8cf86dfc31c1", size = 37480330, upload-time = "2026-08-21T23:24:19.608Z" },
789
+ { url = "https://files.pythonhosted.org/packages/39/e7/979fd14e75008623df31ba70d6bb144700f68feadcea042021c06a05bf82/scipy-1.18.1-cp312-cp312-win_amd64.whl", hash = "sha256:5e4d44984abc0020154ea81b247adeddcc3ac5527b975ff798bd1ba0adc513c2", size = 36658278, upload-time = "2026-08-21T23:24:25.463Z" },
790
+ { url = "https://files.pythonhosted.org/packages/c7/0b/e1525354ff9d7d5feb6d1b31af6d14072e5c91e9607b421fa1ec889660b3/scipy-1.18.1-cp312-cp312-win_arm64.whl", hash = "sha256:d65d448389b8436493abcf629cc94ad0cf32aecaf06e1acca1de53cc795f2f12", size = 24400588, upload-time = "2026-08-21T23:24:30.579Z" },
791
+ ]
792
+
793
  [[package]]
794
  name = "sentencepiece"
795
  version = "0.2.2"
 
826
  { url = "https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl", hash = "sha256:e091cc3e99d2141a0ba2847328f5479b05d94a6635cb96148ccb3f34671bd8f5", size = 6299353, upload-time = "2025-04-27T18:04:59.103Z" },
827
  ]
828
 
829
+ [[package]]
830
+ name = "threadpoolctl"
831
+ version = "3.7.0"
832
+ source = { registry = "https://pypi.org/simple" }
833
+ sdist = { url = "https://files.pythonhosted.org/packages/00/dc/6c58154c1c65f758ea979e7139cb76993a9cfc662d14e9be3c4a667cfb77/threadpoolctl-3.7.0.tar.gz", hash = "sha256:61348cfb77d53b9242e0017029244b559b810c142ced65b4e21eeca1843959a7", size = 31961, upload-time = "2026-09-15T15:46:20.263Z" }
834
+ wheels = [
835
+ { url = "https://files.pythonhosted.org/packages/43/3f/f88a53f60a472b46f4023f56d204dd7de33d34c5d2acbfa0d70a674e639e/threadpoolctl-3.7.0-py3-none-any.whl", hash = "sha256:cd8b60b5641b45c67bbf73c64c843235fc2d8a480c87389f52f5dbee893b86be", size = 26362, upload-time = "2026-09-15T15:46:19.168Z" },
836
+ ]
837
+
838
  [[package]]
839
  name = "tokenizers"
840
  version = "0.22.2"
verify-extraction.py ADDED
@@ -0,0 +1,170 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Small real-fixture native/Core ML parity check for entity extraction."""
2
+
3
+ import argparse
4
+ import json
5
+ from pathlib import Path
6
+
7
+ import coremltools as ct
8
+ import numpy as np
9
+ import torch
10
+ from gliner2 import AutoExtractor, Schema
11
+ from gliner2.models.boundary.model import _group_scored_candidates
12
+ from gliner2.models.outputs import CandidateTensorBatch
13
+ from huggingface_hub import snapshot_download
14
+
15
+ from convert_extraction_names import FEATURE_NAMES, SCORE_INPUT_NAMES
16
+ from extraction_pool import select_candidates
17
+ from preprocessing import prepare_extraction
18
+
19
+ MODEL_ID = "fastino/gliner2.5-multi-v1"
20
+ MODEL_REVISION = "a221b77a8baf4a613b8f8652661d41fa10a5641e"
21
+ FIXTURES = [
22
+ ("Alice founded Acme in Toronto in 2020.", ["person", "organization", "location"]),
23
+ ("María trabaja en Madrid para Telefónica.", ["persona", "organización", "ciudad"]),
24
+ ("Jean Dupont travaille à Paris pour Airbus.", ["personne", "entreprise", "ville"]),
25
+ ("张伟在北京大学工作。", ["人物", "机构", "地点"]),
26
+ ("Angela Merkel besuchte Berlin.", ["Person", "Stadt"]),
27
+ ]
28
+
29
+
30
+ def coreml_entity_result(native, features_model, scorer_model, text, schema, length, words, queries):
31
+ arrays, batch = prepare_extraction(native.processor, text, schema, length, words, queries)
32
+ features = features_model.predict(arrays)
33
+ tensor = {name: torch.from_numpy(np.asarray(features[name]).copy()) for name in FEATURE_NAMES}
34
+ text_mask = torch.from_numpy(arrays["text_mask"]).bool()
35
+ query_mask = torch.from_numpy(arrays["query_mask"]).bool()
36
+ head = native.boundary_head
37
+ pool = select_candidates(
38
+ tensor["pool_start_projection"],
39
+ tensor["pool_end_projection"],
40
+ tensor["boundary_mask"].bool(),
41
+ query_mask,
42
+ tensor["start_logits"],
43
+ tensor["end_logits"],
44
+ boundary_top_k=head.shared_pool_builder.pool_boundary_top_k,
45
+ pool_size=head.shared_pool_builder.pool_size,
46
+ min_pool_per_query=head.shared_pool_builder.min_pool_per_query,
47
+ )
48
+ score_values = (
49
+ tensor["text_states"],
50
+ text_mask.float(),
51
+ tensor["query_states"],
52
+ query_mask.float(),
53
+ tensor["boundary_states"],
54
+ tensor["start_logits"],
55
+ tensor["end_logits"],
56
+ tensor["inside_prefix"],
57
+ tensor["inside_prefix_mean"],
58
+ pool.indices.int(),
59
+ pool.mask.float(),
60
+ pool.compat_logits,
61
+ )
62
+ score_arrays = {
63
+ name: value.numpy().astype(np.int32 if name == "candidate_indices" else np.float32)
64
+ for name, value in zip(SCORE_INPUT_NAMES, score_values)
65
+ }
66
+ scores = scorer_model.predict(score_arrays)
67
+ candidate_batch = CandidateTensorBatch(
68
+ indices=pool.indices.unsqueeze(1).expand(1, queries, -1, 2),
69
+ proposal_logits=pool.proposal_logits.unsqueeze(1).expand(1, queries, -1),
70
+ pair_logits=torch.from_numpy(np.asarray(scores["pair_logits"]).copy()),
71
+ valid_mask=pool.mask.unsqueeze(1).expand(1, queries, -1),
72
+ query_mask=query_mask,
73
+ )
74
+ _, metadata = native._build_schema_dicts_and_metadata([schema])
75
+ with torch.no_grad():
76
+ core = native._encode_core(batch)
77
+ native_output = native._extract_from_batch(batch, 0.5, metadata, True, True)[0]
78
+ probs = torch.sigmoid(candidate_batch.pair_logits / native.boundary_settings.pair_temperature)
79
+ grouped = _group_scored_candidates(
80
+ candidate_batch,
81
+ threshold=0.5,
82
+ probabilities=probs,
83
+ count_log_rates=tensor["count_log_rates"],
84
+ adaptive_threshold=native.boundary_settings.adaptive_threshold,
85
+ )
86
+ extracted = native._decode_entities(
87
+ 0,
88
+ core,
89
+ core["ext_specs"][0],
90
+ grouped[0],
91
+ metadata[0],
92
+ torch.sigmoid(tensor["null_logits"])[0],
93
+ "flat",
94
+ core["word_offsets"][0],
95
+ batch.start_mappings[0],
96
+ batch.end_mappings[0],
97
+ text,
98
+ len(batch.start_mappings[0]),
99
+ True,
100
+ True,
101
+ )
102
+ coreml_output = {"entities": [extracted]} if extracted else {}
103
+ return native_output, coreml_output, pool, candidate_batch
104
+
105
+
106
+ def stripped_spans(result):
107
+ output = {}
108
+ for name, entities in result.get("entities", [{}])[0].items():
109
+ output[name] = [(item["text"], item["start"], item["end"]) for item in entities]
110
+ return output
111
+
112
+
113
+ def main():
114
+ parser = argparse.ArgumentParser()
115
+ parser.add_argument("--model-dir", required=True)
116
+ parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32")
117
+ parser.add_argument("--length", type=int, default=128)
118
+ parser.add_argument("--max-words", type=int, default=64)
119
+ parser.add_argument("--max-queries", type=int, default=8)
120
+ args = parser.parse_args()
121
+ torch.set_num_threads(4)
122
+ source = snapshot_download(MODEL_ID, revision=MODEL_REVISION)
123
+ native = AutoExtractor.from_pretrained(str(source), map_location="cpu").eval()
124
+ suffix = f"{args.precision}_L{args.length}_W{args.max_words}_Q{args.max_queries}"
125
+ folder = Path(args.model_dir)
126
+ features = ct.models.MLModel(
127
+ str(folder / f"gliner2_multi_extraction_features_{suffix}.mlpackage"), compute_units=ct.ComputeUnit.CPU_ONLY
128
+ )
129
+ scorer = ct.models.MLModel(
130
+ str(folder / f"gliner2_multi_extraction_scorer_{suffix}.mlpackage"), compute_units=ct.ComputeUnit.CPU_ONLY
131
+ )
132
+ cases = []
133
+ for text, labels in FIXTURES:
134
+ reference, actual, _, _ = coreml_entity_result(
135
+ native,
136
+ features,
137
+ scorer,
138
+ text,
139
+ Schema().entities(labels),
140
+ args.length,
141
+ args.max_words,
142
+ args.max_queries,
143
+ )
144
+ expected_spans = stripped_spans(reference)
145
+ actual_spans = stripped_spans(actual)
146
+ cases.append(
147
+ {
148
+ "text": text,
149
+ "native_spans": expected_spans,
150
+ "coreml_spans": actual_spans,
151
+ "span_match": expected_spans == actual_spans,
152
+ }
153
+ )
154
+ report = {
155
+ "source_model": MODEL_ID,
156
+ "source_revision": MODEL_REVISION,
157
+ "precision": args.precision,
158
+ "selected_manifest": "five fixed English, Spanish, French, Chinese and German entity fixtures",
159
+ "cases": cases,
160
+ "span_matches": sum(case["span_match"] for case in cases),
161
+ }
162
+ path = folder / f"verify-entities-{suffix}.json"
163
+ path.write_text(json.dumps(report, indent=2) + "\n")
164
+ print(json.dumps(report, indent=2))
165
+ if report["span_matches"] != len(cases):
166
+ raise SystemExit(1)
167
+
168
+
169
+ if __name__ == "__main__":
170
+ main()
verify-full-extraction.py ADDED
@@ -0,0 +1,144 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Selected end-to-end native/Core ML checks across GLiNER2 schema paths."""
2
+
3
+ import argparse
4
+ import json
5
+ from pathlib import Path
6
+
7
+ import torch
8
+ from gliner2 import AttributeGroup, AutoExtractor, Schema
9
+ from huggingface_hub import snapshot_download
10
+
11
+ from extraction_runtime import CoreMLBoundaryExtractor
12
+
13
+ MODEL_ID = "fastino/gliner2.5-multi-v1"
14
+ MODEL_REVISION = "a221b77a8baf4a613b8f8652661d41fa10a5641e"
15
+
16
+
17
+ def structure(mode):
18
+ schema = Schema()
19
+ builder = schema.structure("employment", mode=mode, anchor="person" if mode == "natural" else None)
20
+ builder.field("person", dtype="str")
21
+ builder.field("company", dtype="str")
22
+ return schema
23
+
24
+
25
+ def fixtures():
26
+ attribute = Schema().entities(["person", "organization"])
27
+ attribute.entity_attributes({"role": AttributeGroup(labels=["founder", "employee"])})
28
+ mixed = Schema().entities(["person", "organization"])
29
+ mixed.classification("sentiment", ["positive", "negative"])
30
+ choice = Schema()
31
+ choice.structure("product").field("category", dtype="str", choices=["electronics", "clothing"])
32
+ return [
33
+ (
34
+ "entities",
35
+ "Alice founded Acme in Toronto in 2020.",
36
+ Schema().entities(["person", "organization", "location"]),
37
+ ),
38
+ (
39
+ "entities_multi",
40
+ "Marie Curie was born in Warsaw and worked in Paris.",
41
+ Schema().entities(["person", "city"]),
42
+ ),
43
+ ("relations", "Alice founded Acme in Toronto.", Schema().relations(["founded"])),
44
+ ("relations_two", "Steve Jobs founded Apple.", Schema().relations(["founded"])),
45
+ ("record_natural", "Alice works at Acme. Bob works at Beta.", structure("natural")),
46
+ ("record_latent", "Alice works at Acme. Bob works at Beta.", structure("latent")),
47
+ ("record_anchorless", "Alice works at Acme. Bob works at Beta.", structure("anchorless")),
48
+ ("attributes", "Alice founded Acme.", attribute),
49
+ ("mixed_classification", "Alice founded Acme.", mixed),
50
+ (
51
+ "classification_only",
52
+ "The rocket launched successfully.",
53
+ Schema().classification("topic", ["science", "sports", "politics"]),
54
+ ),
55
+ ("enum_choice", "The item is electronics.", choice),
56
+ (
57
+ "es_entities",
58
+ "María trabaja en Madrid para Telefónica.",
59
+ Schema().entities(["persona", "organización", "ciudad"]),
60
+ ),
61
+ (
62
+ "fr_entities",
63
+ "Jean Dupont travaille à Paris pour Airbus.",
64
+ Schema().entities(["personne", "entreprise", "ville"]),
65
+ ),
66
+ ("zh_entities", "张伟在北京大学工作。", Schema().entities(["人物", "机构", "地点"])),
67
+ ("de_entities", "Angela Merkel besuchte Berlin.", Schema().entities(["Person", "Stadt"])),
68
+ ]
69
+
70
+
71
+ def without_confidence(value):
72
+ if isinstance(value, dict):
73
+ return {key: without_confidence(item) for key, item in value.items() if key != "confidence"}
74
+ if isinstance(value, list):
75
+ return [without_confidence(item) for item in value]
76
+ return value
77
+
78
+
79
+ def confidence_errors(reference, actual):
80
+ if isinstance(reference, dict) and isinstance(actual, dict):
81
+ errors = []
82
+ for key, value in reference.items():
83
+ if key == "confidence" and key in actual:
84
+ errors.append(abs(float(value) - float(actual[key])))
85
+ elif key in actual:
86
+ errors.extend(confidence_errors(value, actual[key]))
87
+ return errors
88
+ if isinstance(reference, list) and isinstance(actual, list):
89
+ return [error for a, b in zip(reference, actual) for error in confidence_errors(a, b)]
90
+ return []
91
+
92
+
93
+ def main():
94
+ parser = argparse.ArgumentParser()
95
+ parser.add_argument("--model-dir", required=True)
96
+ parser.add_argument("--precision", choices=["fp16", "fp32"], default="fp32")
97
+ parser.add_argument("--allow-mismatch", action="store_true")
98
+ args = parser.parse_args()
99
+ torch.set_num_threads(4)
100
+ source = snapshot_download(MODEL_ID, revision=MODEL_REVISION)
101
+ native = AutoExtractor.from_pretrained(str(source), map_location="cpu").eval()
102
+ runtime = CoreMLBoundaryExtractor(args.model_dir, precision=args.precision)
103
+ cases = []
104
+ for name, text, schema in fixtures():
105
+ expected = native.extract(text, schema, include_confidence=True, include_spans=True)
106
+ actual = runtime.extract(text, schema, include_confidence=True, include_spans=True)
107
+ errors = confidence_errors(expected, actual)
108
+ cases.append(
109
+ {
110
+ "name": name,
111
+ "text": text,
112
+ "native": expected,
113
+ "coreml": actual,
114
+ "structure_match": without_confidence(expected) == without_confidence(actual),
115
+ "maximum_confidence_error": max(errors, default=None),
116
+ }
117
+ )
118
+ result = {
119
+ "source_model": MODEL_ID,
120
+ "source_revision": MODEL_REVISION,
121
+ "precision": args.precision,
122
+ "selected_manifest": "fifteen fixed real-text schema fixtures, not a Decision Index score",
123
+ "matched": sum(case["structure_match"] for case in cases),
124
+ "total": len(cases),
125
+ "cases": cases,
126
+ }
127
+ path = Path(args.model_dir) / f"verify-full-{args.precision}.json"
128
+ path.write_text(json.dumps(result, indent=2, ensure_ascii=False) + "\n")
129
+ print(
130
+ json.dumps(
131
+ {
132
+ "matched": result["matched"],
133
+ "total": result["total"],
134
+ "failed": [case["name"] for case in cases if not case["structure_match"]],
135
+ },
136
+ indent=2,
137
+ )
138
+ )
139
+ if result["matched"] != result["total"] and not args.allow_mismatch:
140
+ raise SystemExit(1)
141
+
142
+
143
+ if __name__ == "__main__":
144
+ main()