mlboydaisuke commited on
Commit
8621dc4
·
verified ·
1 Parent(s): c8b5a0f

card: CoreAIKit's LFM3 support is not in a release or on the kit's main yet; what runs without the kit

Browse files
Files changed (2) hide show
  1. README.md +8 -5
  2. SHA256SUMS +1 -1
README.md CHANGED
@@ -71,8 +71,8 @@ questions below. What this port adds is a Neural Engine bundle for the iPhone an
71
  **iPhone 18 Pro** (iOS 27.2), a 768-token prompt and 256 generated tokens: prefill 590.6 tok/s, decode 47.17
72
  tok/s (medians of 6 trials), one turn 6.71 s (768 / prefill + 255 / decode, not timed), peak footprint 2.82 GB,
73
  thermal state nominal at the start and end of every run. In the gate app the first load on the phone took 34.9 s
74
- and later loads 4.4 s or less; CoreAIKit's test app, with the bundle at its own path, took 32.4 s on its first
75
- load, then 3.7 s and 0.4 s.
76
 
77
  ## What was checked
78
 
@@ -130,8 +130,9 @@ The full table, the transcripts and how the fp16 activations set a floor on this
130
 
131
  ## Use it
132
 
133
- CoreAIKit's support for LFM3 — the XML tool calls, the reasoning stream, the thinking switch and the
134
- `lfm3-1.7b` catalog entry — is not in a CoreAIKit release or on the kit's main branch (0.7.3 does not support
 
135
  LFM3). The CoreAIKit checks above ran a kit build that has it; with such a build, on a Mac or an iPhone 18 Pro:
136
 
137
  ```swift
@@ -167,7 +168,9 @@ Without the kit: the Mac gates ran `llm-runner` built from the zoo's fork of
167
  [apple/coreai-models](https://github.com/apple/coreai-models), whose pipelined engine carries the conv state as an
168
  extra state and finds the `prefill` function by name, with `COREAI_CHUNK_THRESHOLD=1` and
169
  `--inference-engine-variant coreai-pipelined --warmup off`; the iPhone checks ran the `StaticShapeEngine` of that
170
- fork's tag `0.2.8-zoo`, the one CoreAIKit pins.
 
 
171
 
172
  ## Limits
173
 
 
71
  **iPhone 18 Pro** (iOS 27.2), a 768-token prompt and 256 generated tokens: prefill 590.6 tok/s, decode 47.17
72
  tok/s (medians of 6 trials), one turn 6.71 s (768 / prefill + 255 / decode, not timed), peak footprint 2.82 GB,
73
  thermal state nominal at the start and end of every run. In the gate app the first load on the phone took 34.9 s
74
+ and later loads 4.4 s or less; a test app built on CoreAIKit (`ChatSession(catalog:)`), with the bundle at its own path, took 32.4 s on its
75
+ first load, then 3.7 s and 0.4 s.
76
 
77
  ## What was checked
78
 
 
130
 
131
  ## Use it
132
 
133
+ What runs today without CoreAIKit is at the end of this section. CoreAIKit's support for LFM3 — the XML tool
134
+ calls, reading a reply that starts inside the thinking block, the thinking switch (`enableThinking`) and the `lfm3-1.7b` catalog
135
+ entry — is not in a CoreAIKit release or on the kit's main branch (0.7.3 does not support
136
  LFM3). The CoreAIKit checks above ran a kit build that has it; with such a build, on a Mac or an iPhone 18 Pro:
137
 
138
  ```swift
 
168
  [apple/coreai-models](https://github.com/apple/coreai-models), whose pipelined engine carries the conv state as an
169
  extra state and finds the `prefill` function by name, with `COREAI_CHUNK_THRESHOLD=1` and
170
  `--inference-engine-variant coreai-pipelined --warmup off`; the iPhone checks ran the `StaticShapeEngine` of that
171
+ fork's tag `0.2.8-zoo`, the one CoreAIKit pins. The Mac tool-call checks also ran through the zoo's
172
+ `ZooLanguageModel` (`swift/Sources/ZooFMProvider` in the zoo; it depends on the coreai-models package, not on
173
+ CoreAIKit).
174
 
175
  ## Limits
176
 
SHA256SUMS CHANGED
@@ -52,4 +52,4 @@ b7915955d140c6c01bed43459ebc92d33c44d701270936384c41b9e41523cc6f ios-h19p/lfm3_
52
  7e226e34bface23ccf6e495d157bf77186c1c30a7659880e1a8994db528c3701 ios-h19p/tokenizer/tokenizer.json
53
  bf028f813efc6137bc07ed045cafe71820df9a5e26a38e39954706ed1e30a457 ios-h19p/tokenizer/tokenizer_config.json
54
  4d28ca14dedc0b3d0fcc2b3339f0e79931faa33874f3d24f522183a8fc70068c LICENSE
55
- 3cb5759dbe5f540da374735804ea4bea62ddd9c5866512b1546ea98f84ac0004 README.md
 
52
  7e226e34bface23ccf6e495d157bf77186c1c30a7659880e1a8994db528c3701 ios-h19p/tokenizer/tokenizer.json
53
  bf028f813efc6137bc07ed045cafe71820df9a5e26a38e39954706ed1e30a457 ios-h19p/tokenizer/tokenizer_config.json
54
  4d28ca14dedc0b3d0fcc2b3339f0e79931faa33874f3d24f522183a8fc70068c LICENSE
55
+ 089ef7119466ed5a26175f31502de64ebcabc4dc45a28fea93d621b287ddd582 README.md