FluidInference
/

gliner2-5-multi-coreml

@@ -44,4 +44,6 @@ The original L128/K8 classification packages remain available, with a separate `
 An optional **embedding-only W8 with asymmetric scaling** classifier keeps the encoder and heads FP16. It reduces the package from **576,528,829 to 385,193,894 bytes** (33.2%). On an M5 Pro plugged into AC, it agreed with FP16 on 100/100 selected eligible application requests (largest probability difference 0.01941). It matched the native classifier on 99/100 requests; the one near-tie disagreement at `jev.ag_news` row 100 is also present in the FP16 export. In a paired 100-request check, automatic-device full-request p50 was 4.283 ms FP16 versus 4.265 ms W8, so there is no established speed gain. The W8 compute plan put 496/516 executable ops on ANE under forced CPU+ANE, with 20 CPU boundary/constant ops; forced CPU+ANE was slower than automatic device choice. Symmetric embedding W8 exceeded the 0.02 probability-difference gate on the first selected manifest and is omitted. Use `runtime.py --precision embedding_w8_linear` for the compressed classification package; extraction continues to use FP32 packages.
 The Core ML deployment target is iOS 17/macOS 14. Conversion scripts, pinned dependencies, asset hashes, and selected verification reports are included. The source revision is a current pinned snapshot; identity with the historical Decision Index evaluation checkpoint has not been established.

 An optional **embedding-only W8 with asymmetric scaling** classifier keeps the encoder and heads FP16. It reduces the package from **576,528,829 to 385,193,894 bytes** (33.2%). On an M5 Pro plugged into AC, it agreed with FP16 on 100/100 selected eligible application requests (largest probability difference 0.01941). It matched the native classifier on 99/100 requests; the one near-tie disagreement at `jev.ag_news` row 100 is also present in the FP16 export. In a paired 100-request check, automatic-device full-request p50 was 4.283 ms FP16 versus 4.265 ms W8, so there is no established speed gain. The W8 compute plan put 496/516 executable ops on ANE under forced CPU+ANE, with 20 CPU boundary/constant ops; forced CPU+ANE was slower than automatic device choice. Symmetric embedding W8 exceeded the 0.02 probability-difference gate on the first selected manifest and is omitted. Use `runtime.py --precision embedding_w8_linear` for the compressed classification package; extraction continues to use FP32 packages.
+When downloading with `huggingface_hub.snapshot_download`, pass `local_dir="./gliner2-multi-coreml"` and point the runtime there. Core ML compilation on the tested macOS release rejected the symlinked weight file in the default Hub cache snapshot; use a materialized local directory.
 The Core ML deployment target is iOS 17/macOS 14. Conversion scripts, pinned dependencies, asset hashes, and selected verification reports are included. The source revision is a current pinned snapshot; identity with the historical Decision Index evaluation checkpoint has not been established.