mlboydaisuke commited on
Commit
dae5add
·
verified ·
1 Parent(s): 89eb82c

gen-cards: regenerate Use-it block

Browse files
Files changed (1) hide show
  1. README.md +39 -0
README.md CHANGED
@@ -25,6 +25,45 @@ Part of the community Core AI model zoo: **https://github.com/john-rocky/coreai-
25
  (full card: [`zoo/lfm2.5-8b-a1b-moe.md`](https://github.com/john-rocky/coreai-model-zoo/blob/main/zoo/lfm2.5-8b-a1b-moe.md)).
26
 
27
  <!-- gen-cards:use-it begin id=lfm2.5-8b-a1b (managed by scripts/gen-cards — edit cards.json / QuickStart.swift, not this block) -->
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  <!-- gen-cards:use-it end -->
29
 
30
  ## The `gather_qmm` kernel
 
25
  (full card: [`zoo/lfm2.5-8b-a1b-moe.md`](https://github.com/john-rocky/coreai-model-zoo/blob/main/zoo/lfm2.5-8b-a1b-moe.md)).
26
 
27
  <!-- gen-cards:use-it begin id=lfm2.5-8b-a1b (managed by scripts/gen-cards — edit cards.json / QuickStart.swift, not this block) -->
28
+ ## Use it
29
+
30
+ ▶️ **Run it (source)** — the [ChatDemo runner](https://github.com/john-rocky/coreai-kit/tree/main/Examples/ChatDemo)
31
+ (GUI + CLI, one app for every chat model in the catalog):
32
+
33
+ ```bash
34
+ git clone https://github.com/john-rocky/coreai-kit
35
+ open coreai-kit/Examples/ChatDemo/ChatDemo.xcodeproj
36
+ # → Run, then pick "LFM2.5-8B-A1B (MoE)" in the model picker
37
+
38
+ # agents / headless (macOS):
39
+ cd coreai-kit/Examples/ChatDemo
40
+ swift run chat-cli --model lfm2.5-8b-a1b --prompt "What can you do, offline?"
41
+ ```
42
+
43
+ 💻 **Build with it** — complete; the glue is kit API, copy-paste runs:
44
+
45
+ ```swift
46
+ import CoreAIKit
47
+
48
+ let chat = try await ChatSession(catalog: "lfm2.5-8b-a1b")
49
+ let reply = try await chat.respond(to: prompt)
50
+ // reply: the answer, generated fully on-device
51
+ ```
52
+
53
+ The take-home is [`Examples/ChatDemo/Sources/QuickStart.swift`](https://github.com/john-rocky/coreai-kit/blob/main/Examples/ChatDemo/Sources/QuickStart.swift)
54
+ — this exact code as one typed function, no UI; the CLI is an argument shell over it, and
55
+ the GUI drives the same `ChatSession` across turns for its transcript.
56
+ Multi-turn? Hold the `ChatSession` and call `respond(to:)` per turn — it keeps the
57
+ conversation history; `streamResponse(to:)` yields tokens as they decode.
58
+
59
+ **Integration checklist**
60
+
61
+ - SPM: `https://github.com/john-rocky/coreai-kit` → product **CoreAIKit**
62
+ - Info.plist: none needed
63
+ - Entitlements: none needed (macOS)
64
+ - First run downloads the model — 9.0 GB (Mac) — then it loads from the
65
+ local cache (Application Support; progress via the `downloadProgress` callback)
66
+ - Measure in Release — Debug is ~3× slower on per-token host work
67
  <!-- gen-cards:use-it end -->
68
 
69
  ## The `gather_qmm` kernel