| --- |
| license: apache-2.0 |
| base_model: jaredpalmer/kev-0.8b |
| library_name: coreml |
| pipeline_tag: text-classification |
| tags: |
| - coreml |
| - apple-silicon |
| - decision |
| - qwen3.5 |
| --- |
| |
| # Kev-0.8B for Core ML |
|
|
| Core ML export of [jaredpalmer/kev-0.8b](https://huggingface.co/jaredpalmer/kev-0.8b) (Jared Palmer, Apache-2.0): Kev's |
| LoRA folded into [Qwen/Qwen3.5-0.8B-Base](https://huggingface.co/Qwen/Qwen3.5-0.8B-Base) in fp32 with Kev's own loader, |
| plus Kev's pointer head. Kev answers questions about a piece of text (a *state*): multiple choice, yes/no, or a score, |
| with calibrated probabilities. |
|
|
| ## Files |
|
|
| | Path | What | |
| | --- | --- | |
| | `fused/KevFused.mlpackage` | Multifunction package, weight-shared: `fused_S<state>_P<packed>_B16_K16` for states of 32–384 tokens and up to 192 packed question tokens. One call reads the state and answers all of a request's questions (up to 16 answers, 16 options each). | |
| | `fused/config.json` | Dimensions, special tokens, temperature. | |
| | `L512_K16/KevRow_fp16.mlpackage` | One question per call (state + question ≤ 512 tokens, ≤ 16 options), for questions that do not fit the fused path. | |
| | `L1024_K80/KevRow_fp16.mlpackage` | One question per call up to 1,024 tokens and 80 options (e.g. 77-way intent questions). | |
| | `L512_K16/embeddings.f16` | Token embeddings (fp16, 248,320 × 1,024), gathered on the host. | |
| | `tokenizer.json` | Qwen3.5 tokenizer. | |
|
|
| fp16, GPU (`cpuAndGPU`), iOS 18 / macOS 15. Swift: `KevFastManager` in |
| [FluidUse](https://github.com/FluidInference/FluidUse) (own Qwen BPE tokenizer, exact against Hugging Face tokenizers). |
|
|
| ## How the fused call works |
|
|
| The state's tokens come first, then every question packed end to end. A segment mask keeps each question from seeing the |
| others: it is the attention mask, the local decay of the Gated DeltaNet layers, and the triangle of their WY solve, so |
| every question restarts from the state exactly as if it were its own row. Everything except the delta-rule core runs |
| once over all positions. |
|
|
| ## Fidelity |
|
|
| Kev's own benchmark, unchanged, scored the row packages against Kev's fp32 PyTorch model on the development splits: |
|
|
| | Suite | Questions | Top answer differs | Kev fp32 | Published | Core ML fp16 | |
| | --- | ---: | ---: | ---: | ---: | ---: | |
| | transfer-v4 | 764 | 1 | 0.6479 | 0.648 | 0.6463 | |
| | decision-v7 | 1,468 | 2 | 0.8267 | 0.827 | 0.8252 | |
|
|
| The fused pass matches the row form in fp32 (0 flips, max |Δp| 3e-6 with every question packed three times), and the |
| fp16 fused functions agree with the fp16 row packages on 191 questions (0 flips, max |Δp| 0.004). |
|
|
| ## Speed and memory against the original model |
|
|
| MacBook Pro M5 Pro (24 GB). Guess Who over 80 Wikipedia people (DBpedia-14 test split): each bio is one request with 12 |
| yes/no questions, 960 decisions. The original is Kev's own checkpoint loader and serving path in PyTorch on the GPU (MPS). |
|
|
| | | Core ML (this repo, fp16, GPU) | Original, PyTorch bf16 | Original, PyTorch fp32 (default) | |
| | --- | ---: | ---: | ---: | |
| | 80 bios × 12 questions | **2.98 s** | 98.6 s | 123.9 s | |
| | per bio (12 answers, one call) | **36.6 ms** | 1,070 ms | 1,287 ms | |
| | peak memory footprint | **0.67 GB** (+1.45 GB weights mapped from disk) | 6.6 GB | 9.5 GB | |
| | weights | 1.45 GB (fused package + embeddings) | 1.79 GB (bf16 base + LoRA/head) | same | |
| | answers vs original fp32 | 0 of 960 differ | 3 of 960 differ | reference | |
|
|
| PyTorch on a Mac runs Qwen3.5's Gated DeltaNet and causal conv through transformers' reference implementations (the |
| `flash-linear-attention` / `causal_conv1d` kernels are not available there). Peak memory footprint is Activity Monitor's |
| Memory; Core ML maps its weights from disk, so count them as resident for a conservative ~2.1 GB. This backbone does not |
| suit the Neural Engine (one call: GPU 30.8 ms, CPU 186 ms, CPU + ANE 740 ms). A function left idle pays a 0.3–0.8 s |
| re-setup on its next call; warm it before latency-sensitive work. |
|
|
| Conversion code and reports: [FluidInference/mobius](https://github.com/FluidInference/mobius), |
| `models/computer-use/kev-0.8b/coreml`. |
|
|
