card: CoreAIKit's LFM3 support is not in a release or on the kit's main yet; what runs without the kit
Browse files- README.md +8 -5
- SHA256SUMS +1 -1
README.md
CHANGED
|
@@ -71,8 +71,8 @@ questions below. What this port adds is a Neural Engine bundle for the iPhone an
|
|
| 71 |
**iPhone 18 Pro** (iOS 27.2), a 768-token prompt and 256 generated tokens: prefill 590.6 tok/s, decode 47.17
|
| 72 |
tok/s (medians of 6 trials), one turn 6.71 s (768 / prefill + 255 / decode, not timed), peak footprint 2.82 GB,
|
| 73 |
thermal state nominal at the start and end of every run. In the gate app the first load on the phone took 34.9 s
|
| 74 |
-
and later loads 4.4 s or less;
|
| 75 |
-
load, then 3.7 s and 0.4 s.
|
| 76 |
|
| 77 |
## What was checked
|
| 78 |
|
|
@@ -130,8 +130,9 @@ The full table, the transcripts and how the fp16 activations set a floor on this
|
|
| 130 |
|
| 131 |
## Use it
|
| 132 |
|
| 133 |
-
|
| 134 |
-
|
|
|
|
| 135 |
LFM3). The CoreAIKit checks above ran a kit build that has it; with such a build, on a Mac or an iPhone 18 Pro:
|
| 136 |
|
| 137 |
```swift
|
|
@@ -167,7 +168,9 @@ Without the kit: the Mac gates ran `llm-runner` built from the zoo's fork of
|
|
| 167 |
[apple/coreai-models](https://github.com/apple/coreai-models), whose pipelined engine carries the conv state as an
|
| 168 |
extra state and finds the `prefill` function by name, with `COREAI_CHUNK_THRESHOLD=1` and
|
| 169 |
`--inference-engine-variant coreai-pipelined --warmup off`; the iPhone checks ran the `StaticShapeEngine` of that
|
| 170 |
-
fork's tag `0.2.8-zoo`, the one CoreAIKit pins.
|
|
|
|
|
|
|
| 171 |
|
| 172 |
## Limits
|
| 173 |
|
|
|
|
| 71 |
**iPhone 18 Pro** (iOS 27.2), a 768-token prompt and 256 generated tokens: prefill 590.6 tok/s, decode 47.17
|
| 72 |
tok/s (medians of 6 trials), one turn 6.71 s (768 / prefill + 255 / decode, not timed), peak footprint 2.82 GB,
|
| 73 |
thermal state nominal at the start and end of every run. In the gate app the first load on the phone took 34.9 s
|
| 74 |
+
and later loads 4.4 s or less; a test app built on CoreAIKit (`ChatSession(catalog:)`), with the bundle at its own path, took 32.4 s on its
|
| 75 |
+
first load, then 3.7 s and 0.4 s.
|
| 76 |
|
| 77 |
## What was checked
|
| 78 |
|
|
|
|
| 130 |
|
| 131 |
## Use it
|
| 132 |
|
| 133 |
+
What runs today without CoreAIKit is at the end of this section. CoreAIKit's support for LFM3 — the XML tool
|
| 134 |
+
calls, reading a reply that starts inside the thinking block, the thinking switch (`enableThinking`) and the `lfm3-1.7b` catalog
|
| 135 |
+
entry — is not in a CoreAIKit release or on the kit's main branch (0.7.3 does not support
|
| 136 |
LFM3). The CoreAIKit checks above ran a kit build that has it; with such a build, on a Mac or an iPhone 18 Pro:
|
| 137 |
|
| 138 |
```swift
|
|
|
|
| 168 |
[apple/coreai-models](https://github.com/apple/coreai-models), whose pipelined engine carries the conv state as an
|
| 169 |
extra state and finds the `prefill` function by name, with `COREAI_CHUNK_THRESHOLD=1` and
|
| 170 |
`--inference-engine-variant coreai-pipelined --warmup off`; the iPhone checks ran the `StaticShapeEngine` of that
|
| 171 |
+
fork's tag `0.2.8-zoo`, the one CoreAIKit pins. The Mac tool-call checks also ran through the zoo's
|
| 172 |
+
`ZooLanguageModel` (`swift/Sources/ZooFMProvider` in the zoo; it depends on the coreai-models package, not on
|
| 173 |
+
CoreAIKit).
|
| 174 |
|
| 175 |
## Limits
|
| 176 |
|
SHA256SUMS
CHANGED
|
@@ -52,4 +52,4 @@ b7915955d140c6c01bed43459ebc92d33c44d701270936384c41b9e41523cc6f ios-h19p/lfm3_
|
|
| 52 |
7e226e34bface23ccf6e495d157bf77186c1c30a7659880e1a8994db528c3701 ios-h19p/tokenizer/tokenizer.json
|
| 53 |
bf028f813efc6137bc07ed045cafe71820df9a5e26a38e39954706ed1e30a457 ios-h19p/tokenizer/tokenizer_config.json
|
| 54 |
4d28ca14dedc0b3d0fcc2b3339f0e79931faa33874f3d24f522183a8fc70068c LICENSE
|
| 55 |
-
|
|
|
|
| 52 |
7e226e34bface23ccf6e495d157bf77186c1c30a7659880e1a8994db528c3701 ios-h19p/tokenizer/tokenizer.json
|
| 53 |
bf028f813efc6137bc07ed045cafe71820df9a5e26a38e39954706ed1e30a457 ios-h19p/tokenizer/tokenizer_config.json
|
| 54 |
4d28ca14dedc0b3d0fcc2b3339f0e79931faa33874f3d24f522183a8fc70068c LICENSE
|
| 55 |
+
089ef7119466ed5a26175f31502de64ebcabc4dc45a28fea93d621b287ddd582 README.md
|