File size: 230 Bytes
9ae5659 | 1 2 3 4 | Exported and not published. On an iPhone 16 the best Core ML setup (fp16, Neural
Engine encode and decode) takes ~84 ms per sentence against ~46 ms for XNNPACK int8:
a one-token decode step costs 3.8 ms vs 2.0 ms. Use `xnnpack/`.
|