Exported and not published. On an iPhone 16 the best Core ML setup (fp16, Neural
Engine encode and decode) takes ~84 ms per sentence against ~46 ms for XNNPACK int8:
a one-token decode step costs 3.8 ms vs 2.0 ms. Use xnnpack/.
Exported and not published. On an iPhone 16 the best Core ML setup (fp16, Neural
Engine encode and decode) takes ~84 ms per sentence against ~46 ms for XNNPACK int8:
a one-token decode step costs 3.8 ms vs 2.0 ms. Use xnnpack/.