aac6fef/laya-coreml-adaptive-fp32

laya Core ML adaptive FP32

Native Core ML conversion of convaiinnovations/laya, pinned source revision c5d78730f3493e4fe16d61507ef4b78eef7318cf. Requires laya-coreml >= 0.5.1, Apple silicon and macOS 15+. Tested on M3 Max. This is an independent conversion.

import laya_coreml as laya
with laya.load("aac6fef/laya-coreml-adaptive-fp32") as agent:
    result = agent.predict("The customer requests a refund.", {
        "refund": {"type": "noul", "instructions": "Does the customer request a refund?"}
    })

Install with pip install laya-coreml==0.5.1. Pin this repository's commit SHA with revision= for reproducibility. Inference requires no PyTorch, MLX or Transformers.

The bundle contains B=1/2/4/16 graphs, dense short sequence buckets, 512-token capacity and up to 32 options per graph. Logical batch/window behavior follows upstream Laya v0.4.12. Compute precision is float32, running on CPU/GPU; this is not the fixed-shape ANE model.

Validation: 63/63 selected answers match upstream FP32, maximum calibrated probability error 6.43223004e-06, 100 stable repeated calls, and full public-output checks. See validation.json. The model files are byte-identical to the validated dense export; packaging changes provenance metadata and adds this card/license. These finite checks do not prove universal model accuracy.

Adaptive graphs load lazily. The runtime's default cache retains at most four extra execution instances, never answers. This improves repeated-shape latency at a substantial resident-memory cost; shape_cache_size=0 disables extra instances. Cold/new shapes can be slower. FP16 is not numerically identical to FP32. The 0.5.1 FP32 benchmark recorded Core ML process peak RSS of 5.10–8.77 GiB; that is not a full unified/device memory measure or an FP16 memory estimate.

Runtime and qualification evidence. Original single-graph Hub repositories remain unchanged.