Blame for README.md

---
library_name: coreai
license: other
license_name: lfm1.0
license_link: https://huggingface.co/mlboydaisuke/LFM3-1.7B-CoreAI/blob/main/LICENSE
base_model: LiquidAI/LFM3-1.7B
tags:
  - coreai
  - aimodel
  - apple-silicon
  - on-device
  - lfm3
  - hybrid
  - reasoning
  - tool-calling
pipeline_tag: text-generation
base_model_relation: quantized
---

# LFM3-1.7B — Apple Core AI (`.aimodel`)

LiquidAI's [LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B) converted to Apple's Core AI (iOS 27 /
macOS 27): a 1.7B model that reasons before it answers and calls tools in XML, in two bundles with a 4,096-token
context. The Mac bundle is one int8 `.aimodel` on the `coreai-pipelined` GPU engine that prefills a prompt 64
tokens at a time; the iPhone 18 Pro bundle is a Neural Engine build with fixed input shapes, compiled in advance for
that phone's chip (`h19p`).
[CoreAIKit](https://github.com/john-rocky/coreai-kit)'s `ChatSession` and Apple's `LanguageModelSession` drove
both in the checks below; CoreAIKit's LFM3 support is on the kit's main branch since `9e9143bc`, not yet in a release
([Use it](#use-it)).

Source: [`LiquidAI/LFM3-1.7B`](https://huggingface.co/LiquidAI/LFM3-1.7B) at revision
`ac2586a34f1a1c03329dbdcdefabd8b6478ddaa7`. Conversion code, gates and notes:
[coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo/tree/main/models/lfm3-1.7b).

## Bundles

| path | size | for | what is in it |
|---|---:|---|---|
| `gpu-pipelined/lfm3_1_7b_decode_int8hu_block32_sym_pf64/` | 2.72 GB | Mac | `lfm3_1_7b_decode_int8hu_block32_sym_pf64.aimodel`, `metadata.json`, `tokenizer/` (the source's tokenizer files; the chat template is the vendor's with eight rewrites that render the same string under transformers and parse in swift-jinja 2.3.2 and 2.3.6) |
| `ios-h19p/` | 2.16 GB | iPhone 18 Pro | `lfm3_1_7b_static_pal8_tanh_block_c4096.h19p.ane.aimodelc` (compiled for the iPhone 18 Pro's Neural Engine, `h19p`; its `resources.bin` holds 1.48 GB of weights), `metadata.json`, `tokenizer/` (the same four files) |

**Mac.** `int8hu`: the MLP and conv-mixer linears int8 per-block-32, the head untied and int8 per-block-32
symmetric, the attention projections (q, k, v, o and the gate) and the q / k norms in fp32, everything else fp16.
The `.aimodel` holds two functions over the same weights: `main` takes one token per step and `prefill` takes 64;
the pipelined engine sends the prompt through `prefill` 64 tokens at a time while more than 64 tokens remain, and
the last 1–64 tokens through `main` one at a time. On a Mac, Core AI compiles the `.aimodel` for the GPU the first
time it loads and caches the result.

**iPhone 18 Pro.** The same 24 layers rebuilt the way Apple's iOS Neural Engine path requires (fixed shapes
throughout), with each conv layer's input history kept in its own rows of the KV cache, the sliding window made from
the engine's causal mask, 8-bit palettized weights (k-means, groups of 32; the attention gate fp16) and the tied
head as an int8 table.
It holds contexts of 256 to 4,096 tokens with queries of 8, 16 or 64 tokens, and the compile puts each of its 30
transformer functions in a Neural Engine region. It is compiled for `h19p` only: no other iPhone has a bundle here.

## Speed

**Mac** (M4 Max, macOS 27.0), from a 1,024-token prompt: prefill 1,234 tok/s, 0.83 s to the first token, decode
169.5 tok/s (median of 8 runs, 2026-10-31; the published `.aimodel`, compiled on first load, run by the
coreai-models version CoreAIKit pinned then: the path an app that uses CoreAIKit takes; `llm-benchmark` with 256 generated
tokens, one process per run). Compiled ahead of time for the Mac GPU it is slower: decode 105.5 tok/s and 1.00 s to
the first token in the same window.

In that window, alternated with it on the same 1,024 prompt tokens, MLX (mlx-vlm 0.7.11) decodes LiquidAI's
8-bit weights at 189.8 tok/s and llama.cpp (b12001) their Q8_0 at 180.2, both faster than this bundle, and their
4-bit forms faster still (286.9 and 242.1); their bf16 and F16 forms are slower (113.1 and 111.1). All six reach
the first token in 0.23–0.26 s, and every form, this one included, gives transformers bf16's verdicts on the 8
questions below. What this port adds is a Neural Engine bundle for the iPhone and tool calls through Apple's
`LanguageModelSession`.

**iPhone 18 Pro** (iOS 27.2), a 768-token prompt and 256 generated tokens: prefill 590.6 tok/s, decode 47.17
tok/s (medians of 6 trials), one turn 6.71 s (768 / prefill + 255 / decode, not timed), peak footprint 2.82 GB,
thermal state nominal at the start and end of every run. In the gate app the first load on the phone took 34.9 s
and later loads 4.4 s or less; a test app built on CoreAIKit (`ChatSession(catalog:)`), with the bundle at its own path, took 32.4 s on its
first load, then 3.7 s and 0.4 s.

## What was checked

Against transformers 5.20.2 on the same weights.

The Mac bundle, on a Mac (M4 Max, macOS 27.0):

- **Teacher-forced on 17 sequences** (8 questions with thinking off and on, a 1,648-token document) against
  transformers fp32, with the `.aimodel` compiled ahead of time for the Mac GPU and the prompts fed the way the
  engine feeds them (64-token chunks through `prefill`, the rest one token at a time): top-1
  **2639 / 2682 = 98.40 %** (2,683 positions; the one where transformers' own top-2 gap is under 1e-3 is not
  counted); one token at a time throughout: 2637 / 2682. At every chunk boundary the logits agree with the
  one-token path to cos ≥ 0.99999.
- **8 questions, thinking off**, greedy: 8 / 8, the same verdicts as transformers bf16.
- **Token gate** (16 greedy tokens vs the fp32 oracle): 16 / 16, also on a 215-token prompt that rides three
  chunks; a chat turn stops on `<|im_end|>`.
- **Thinking on**, the same 8 questions, greedy, 3,584-token budget, 3 runs of fresh processes: every run 8 / 8 —
  `</think>` closes after 40–237 tokens (median 56) and the answer after it is right (run on the `main`-only
  build of the same weights: prompts this short never reach `prefill`, and `main` gives the same values).
- **A tool call**, a tool returning a fresh random value, 3 runs each through CoreAIKit's `KitLanguageModel` and the
  zoo's `ZooLanguageModel`: 6 / 6 with the value in the answer verbatim.
- **The bytes published here**: the `.aimodel` is the export re-saved without its source locations (a new
  `main.hash`); run again, its teacher-forced values equal the export's at all 2,683 positions, fed in chunks and
  one token at a time, and the 8 questions, the token gate and the stop-token check give the same results.
- **The kit's path** (the `.aimodel` specialized on load, as timed above; the teacher-forced pass above was not
  repeated on this path): the 8 questions 8 / 8 with the same
  replies as the ahead-of-time build, the token gate 16 / 16 on both prompts, and the stop-token turn stops after 6
  tokens (`1+1=2`; transformers fp32 stops after 2, at a step where its own top-2 gap is 0.005). Through
  CoreAIKit (`ChatSession` and `KitLanguageModel`), the 8 questions with thinking on (3 launches) and off, a tool
  round trip and a 3-turn chat pass.

The iPhone bundle, on an iPhone 18 Pro (iPhone19,2, iOS 27.2), through the fixed-shape (`static-shape`) engine of the
coreai-models version CoreAIKit pinned then:

- **Teacher-forced** over the replies of the same 17 sequences (721 steps, up to 64 a sequence after its prompt)
  against transformers fp32: top-1 **692 / 721 = 95.98 %** (bar 93 %; every step's fp32 top-2 gap is at least
  1e-3). The tokens equal those of Apple's own static engine on an iPhone 17 Pro on all 18 prompts, teacher-forced
  and generated, and a poisoned fixture fails, as it must.
- **8 questions, thinking off**: 8 / 8, the verdicts of transformers bf16; the stop-token turn stops after 6 tokens.
- **CoreAIKit's catalog path**: `ChatSession(catalog: "lfm3-1.7b")` takes the `ios` variant and loads `ios-h19p/`
  (copied into the kit's model store at the path a download would use, so no download ran) on the static-shape
  engine. Thinking on, the 8
  questions in 3 fresh launches: 8 / 8 each, `</think>` inside the budget and the reasoning out of the answer.
  Thinking off: 8 / 8. A tool round trip through `KitLanguageModel` and `LanguageModelSession`: 3 / 3 with the
  value verbatim. With the default configuration, "What is 17 + 25? Answer briefly." gets "17 + 25 = 42.", and a
  3-turn chat recalls its first turn, every turn ending on the stop token.
- With thinking off, 1 of the 8 questions ("How do you say 'thank you' in French?") still opens `<think>` and
  reasons for 68 tokens before its answer, which gives "Merci"; transformers bf16 and the Mac bundle answer it
  directly. Apple's engine on the iPhone 17 Pro does the same, so this comes from the Neural Engine export itself,
  not from CoreAIKit's engine.

The full table, the transcripts and how the fp16 activations set a floor on this model are on the
[zoo card](https://github.com/john-rocky/coreai-model-zoo/tree/main/models/lfm3-1.7b) and in
[the port notes](https://github.com/john-rocky/coreai-model-zoo/blob/main/knowledge/lfm3-1.7b-port.md).

## Use it

What runs today without CoreAIKit is at the end of this section. CoreAIKit's support for LFM3 — the XML tool
calls, reading a reply that starts inside the thinking block, the thinking switch (`enableThinking`) and the `lfm3-1.7b` catalog
entry — is on the kit's main branch since `9e9143bc` ([coreai-kit#83](https://github.com/john-rocky/coreai-kit/pull/83)), not yet in a CoreAIKit release (0.7.3 does
not support LFM3). The CoreAIKit checks above ran a kit build that has it; with the kit's main branch, on a Mac or an
iPhone 18 Pro:

```swift
import CoreAIKit

let chat = try await ChatSession(catalog: "lfm3-1.7b")
for try await event in await chat.streamResponse(to: "What is 17 + 25? Answer briefly.") {
    switch event {
    case .thinking(let delta): print(delta, terminator: "")   // the reasoning comes first
    case .response(let delta): print(delta, terminator: "")   // then the answer
    default: break
    }
}
```

Thinking is on by default (the chat template ends the prompt with `<think>`); `maxResponseTokens` covers the
reasoning and the answer together. `ChatSession.Configuration.enableThinking = false` renders the template's
thinking-off prompt; LiquidAI notes that turning thinking off "can be inconsistent, especially for
reasoning-heavy tasks".

Tool calls come out as XML (`<tool_call>` / `<function=NAME>` / `<parameter=KEY>`). CoreAIKit's `KitLanguageModel`
renders the tools the way the chat template does and parses these calls for Apple's `LanguageModelSession` — the
tool-call checks above.

On an iPhone the kit takes `ios-<arch>/` for the architecture Core AI reports for the phone
(`AIModel.deviceArchitectureName`: `h19p` on the iPhone 18 Pro) and falls back to `ios/`. This repo has
`ios-h19p/` only, so on any other iPhone (the iPhone 17 Pro, which reports `h18p`, included) the kit finds no
bundle and throws `variantNotFound` before it downloads anything. Both apps that ran the iPhone checks carried the
`com.apple.developer.kernel.increased-memory-limit` entitlement (peak footprint 2.82 GB in the gate app, up to 2.94 GB
through `ChatSession`); a run without it was not tried.

Without the kit: the Mac gates ran `llm-runner` built from the zoo's fork of
[apple/coreai-models](https://github.com/apple/coreai-models), whose pipelined engine carries the conv state as an
extra state and finds the `prefill` function by name, with `COREAI_CHUNK_THRESHOLD=1` and
`--inference-engine-variant coreai-pipelined --warmup off`; the iPhone checks ran the `StaticShapeEngine` of that
fork's tag `0.2.8-zoo`, the one CoreAIKit pinned then. The Mac tool-call checks also ran through the zoo's
`ZooLanguageModel` (`swift/Sources/ZooFMProvider` in the zoo; it depends on the coreai-models package, not on
CoreAIKit).

## Limits

- The engine cannot rewind the Mac bundle's conv state by position. Since coreai-models `0.2.9-zoo` (CoreAIKit's pin
  since [coreai-kit#82](https://github.com/john-rocky/coreai-kit/pull/82)), it checkpoints that state at the end of the prompt's last
  message, on the 64-token chunk grid, and CoreAIKit's `ChatSession` resumes each later turn from the checkpoint
  instead of prefilling the whole history; a history shorter than one chunk is prefilled again (in a three-turn chat
  on the kit's main branch, turn 2, after a 32-token first prompt, prefilled all of its 71 tokens, and turn 3 reused 64
  of its 103). The iPhone bundle keeps the conv history in
  the KV cache, and in the 3-turn chat its engine reused the earlier turns' prompt tokens (32, then 70).
- On the Mac the engine always runs the last prompt token through the one-token `main` function, so a prompt's
  final 1–64 tokens run one at a time at about the decode speed: a 1,024-token prompt runs 15 chunks and then 64
  single steps.
- 4,096-token context in both bundles.
- The iPhone bundle is for the iPhone 18 Pro only; the iPhone 17 Pro has no bundle here. The Mac bundle is not for
  iPhones: on an iPhone 18 Pro the `main`-only `.aimodel`, specializing on the phone (device JIT), had not built its
  engine 300 s after launch (the cause is not isolated), and on an iPhone 17 Pro the same int8 weights compiled
  ahead of time for its GPU decoded at 9.25 tok/s (median of 4 trials at thermal state serious; 10.5 in the first
  run, at fair), that compile holding fp16 copies of the int8 linears.
- On a Mac, Core AI named an ahead-of-time compile's
  cache entry after its graph and not its weights, so a second compile of the same graph with other weights ran
  with the first one's weights (not checked on an iPhone): an app that switches between such compiles should
  delete that compile-cache entry first. The Mac bundle is a JIT `.aimodel`, whose cache entry name covers its
  weights.

## License

The weights are under the **LFM Open License v1.0**
([LICENSE](https://huggingface.co/mlboydaisuke/LFM3-1.7B-CoreAI/blob/main/LICENSE), carried from
[`LiquidAI/LFM3-1.7B`](https://huggingface.co/LiquidAI/LFM3-1.7B)): Apache-2.0-style grants, with §5:
commercial use by a legal entity at or above US$10 M annual revenue is not licensed (the threshold does not
apply to a qualified non-profit's non-commercial or research use). Converted from LiquidAI's weights; not
affiliated with Apple or LiquidAI.