Instructions to use FluidInference/gliner2-5-multi-coreml with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- GLiNER2
How to use FluidInference/gliner2-5-multi-coreml with GLiNER2:
from gliner2 import AutoExtractor extractor = AutoExtractor.from_pretrained("FluidInference/gliner2-5-multi-coreml") # Extract entities text = "Apple CEO Tim Cook announced iPhone 15 in Cupertino yesterday." result = extractor.extract_entities(text, ["company", "person", "product", "location"]) print(result) - Notebooks
- Google Colab
- Kaggle
Document ANE placement across all FP16 extraction stages
Browse files- README.md +2 -0
- reports/extraction-w8-validation.json +106 -1
README.md
CHANGED
|
@@ -58,6 +58,8 @@ model = CoreMLAdaptiveBoundaryExtractor(".")
|
|
| 58 |
print(model.extract("Alice founded Acme in Toronto.", Schema().entities(["person", "organization", "location"])))
|
| 59 |
```
|
| 60 |
|
|
|
|
|
|
|
| 61 |
Both W8 feature packages and the five FP16 head packages are included. `extraction-w8-assets.lock.json` records exact file hashes. `quantize-extraction-coreml.py` regenerates the W8 packages from the pinned feature exports; `verify-adaptive-extraction.py` checks the selected native-output manifest. Weight-only W8 does not imply int8 activation or matrix arithmetic.
|
| 62 |
|
| 63 |
## Classification
|
|
|
|
| 58 |
print(model.extract("Alice founded Acme in Toronto.", Schema().entities(["person", "organization", "location"])))
|
| 59 |
```
|
| 60 |
|
| 61 |
+
All six FP16 stages were profiled. The anchorless and record-assignment heads were fully assigned to ANE; the relation head was mixed, while the small scorer and explicit heads ran on CPU under this scheduler. `reports/extraction-w8-validation.json` records both the public compute-plan percentages and private executable-operation counts, which use different counting methods.
|
| 62 |
+
|
| 63 |
Both W8 feature packages and the five FP16 head packages are included. `extraction-w8-assets.lock.json` records exact file hashes. `quantize-extraction-coreml.py` regenerates the W8 packages from the pinned feature exports; `verify-adaptive-extraction.py` checks the selected native-output manifest. Weight-only W8 does not imply int8 activation or matrix arithmetic.
|
| 64 |
|
| 65 |
## Classification
|
reports/extraction-w8-validation.json
CHANGED
|
@@ -171,5 +171,110 @@
|
|
| 171 |
"machine": "arm64",
|
| 172 |
"macos": "27.0"
|
| 173 |
}
|
| 174 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 175 |
}
|
|
|
|
| 171 |
"machine": "arm64",
|
| 172 |
"macos": "27.0"
|
| 173 |
}
|
| 174 |
+
},
|
| 175 |
+
"fp16_stage_profile": [
|
| 176 |
+
{
|
| 177 |
+
"package": "gliner2_multi_explicit_fp16_W64_Q8_S64.mlpackage",
|
| 178 |
+
"profiled_package": "uncompressed FP16 stage; W8 release changes only feature weights",
|
| 179 |
+
"private_executable_ops": {
|
| 180 |
+
"ane": 0,
|
| 181 |
+
"cpu": 204,
|
| 182 |
+
"total": 204
|
| 183 |
+
},
|
| 184 |
+
"public_compute_plan_ane_percent": 0.0,
|
| 185 |
+
"principal_cpu_fallbacks": [
|
| 186 |
+
{
|
| 187 |
+
"reason": "ANE supported but scheduler chose CPU",
|
| 188 |
+
"count": 159
|
| 189 |
+
},
|
| 190 |
+
{
|
| 191 |
+
"reason": "Unsupported tensor data type: int32",
|
| 192 |
+
"count": 32
|
| 193 |
+
}
|
| 194 |
+
]
|
| 195 |
+
},
|
| 196 |
+
{
|
| 197 |
+
"package": "gliner2_multi_extraction_features_fp16_L128_W64_Q8.mlpackage",
|
| 198 |
+
"profiled_package": "uncompressed FP16 stage; W8 release changes only feature weights",
|
| 199 |
+
"private_executable_ops": {
|
| 200 |
+
"ane": 620,
|
| 201 |
+
"cpu": 28,
|
| 202 |
+
"total": 648
|
| 203 |
+
},
|
| 204 |
+
"public_compute_plan_ane_percent": 36.86,
|
| 205 |
+
"principal_cpu_fallbacks": [
|
| 206 |
+
{
|
| 207 |
+
"reason": "Unsupported tensor data type: int32",
|
| 208 |
+
"count": 24
|
| 209 |
+
},
|
| 210 |
+
{
|
| 211 |
+
"reason": "Unable to resolve operation input \"x\".",
|
| 212 |
+
"count": 2
|
| 213 |
+
}
|
| 214 |
+
]
|
| 215 |
+
},
|
| 216 |
+
{
|
| 217 |
+
"package": "gliner2_multi_extraction_scorer_fp16_L128_W64_Q8.mlpackage",
|
| 218 |
+
"profiled_package": "uncompressed FP16 stage; W8 release changes only feature weights",
|
| 219 |
+
"private_executable_ops": {
|
| 220 |
+
"ane": 0,
|
| 221 |
+
"cpu": 137,
|
| 222 |
+
"total": 137
|
| 223 |
+
},
|
| 224 |
+
"public_compute_plan_ane_percent": 0.0,
|
| 225 |
+
"principal_cpu_fallbacks": [
|
| 226 |
+
{
|
| 227 |
+
"reason": "ANE supported but scheduler chose CPU",
|
| 228 |
+
"count": 90
|
| 229 |
+
},
|
| 230 |
+
{
|
| 231 |
+
"reason": "Unsupported tensor data type: int32",
|
| 232 |
+
"count": 36
|
| 233 |
+
}
|
| 234 |
+
]
|
| 235 |
+
},
|
| 236 |
+
{
|
| 237 |
+
"package": "gliner2_multi_record_anchorless_fp16_F8_C192_I1536.mlpackage",
|
| 238 |
+
"profiled_package": "uncompressed FP16 stage; W8 release changes only feature weights",
|
| 239 |
+
"private_executable_ops": {
|
| 240 |
+
"ane": 17,
|
| 241 |
+
"cpu": 0,
|
| 242 |
+
"total": 17
|
| 243 |
+
},
|
| 244 |
+
"public_compute_plan_ane_percent": 100.0,
|
| 245 |
+
"principal_cpu_fallbacks": []
|
| 246 |
+
},
|
| 247 |
+
{
|
| 248 |
+
"package": "gliner2_multi_record_assignment_fp16_F8_C192_I1536.mlpackage",
|
| 249 |
+
"profiled_package": "uncompressed FP16 stage; W8 release changes only feature weights",
|
| 250 |
+
"private_executable_ops": {
|
| 251 |
+
"ane": 24,
|
| 252 |
+
"cpu": 0,
|
| 253 |
+
"total": 24
|
| 254 |
+
},
|
| 255 |
+
"public_compute_plan_ane_percent": 100.0,
|
| 256 |
+
"principal_cpu_fallbacks": []
|
| 257 |
+
},
|
| 258 |
+
{
|
| 259 |
+
"package": "gliner2_multi_relation_fp16_W64_R4_P256.mlpackage",
|
| 260 |
+
"profiled_package": "uncompressed FP16 stage; W8 release changes only feature weights",
|
| 261 |
+
"private_executable_ops": {
|
| 262 |
+
"ane": 49,
|
| 263 |
+
"cpu": 66,
|
| 264 |
+
"total": 115
|
| 265 |
+
},
|
| 266 |
+
"public_compute_plan_ane_percent": 93.49,
|
| 267 |
+
"principal_cpu_fallbacks": [
|
| 268 |
+
{
|
| 269 |
+
"reason": "Unsupported tensor data type: int32",
|
| 270 |
+
"count": 33
|
| 271 |
+
},
|
| 272 |
+
{
|
| 273 |
+
"reason": "Unable to resolve operation input \"y\".",
|
| 274 |
+
"count": 30
|
| 275 |
+
}
|
| 276 |
+
]
|
| 277 |
+
}
|
| 278 |
+
],
|
| 279 |
+
"profile_count_note": "Private executable-op counts and public compute-plan percentages use different counting methods; compute placement is hardware- and OS-specific."
|
| 280 |
}
|