d1-2B gestures on Core AI (GPU and Neural Engine)
A Core AI conversion of LiquidAI/d1-2B by Liquid AI, for the d1 gestures webcam demo: one frame in, hand gesture, facial expression, hand box and face box out, in one forward pass, on the Mac's GPU or Neural Engine through Apple's Core AI (macOS 27).
This is not a general d1 conversion. The export is one fixed-shape function with the prompt of four questions baked in (
code/questions.py): a 640Γ360 frame (16:9; other shapes are centre-cropped) β the answer logits. For any other questions, use the original model: LiquidAI/d1-2B (PyTorch) or LiquidAI/d1-2B-MLX-8bit (MLX).
| File | What it is |
|---|---|
d1_gpu_fp16.aimodel/ |
the function for the GPU, fp16, 3.1 GB |
d1_ane_fp16.aimodel/ |
the same function re-authored for the Neural Engine, fp16, 3.1 GB |
prompt.json |
the baked prompt: token ids, the 12 answer positions, the 1,023 answer tokens they are scored on |
calibration.json |
Liquid AI's calibration, unchanged from LiquidAI/d1-2B |
code/ |
the export (d1_torch.py, d1_ane.py, export_*.py), the parity and speed checks, the host readout, the Swift worker |
LICENSE |
the LFM Open License v1.0, as distributed with the original weights |
Function
image float32 [1, 3, 352, 640] the 640x360 frame after the d1 image processor (resize to 640x352, (x/255 - 0.5)/0.5)
logits [12, 1023] (GPU) / [1, 1023, 1, 12] fp16 (Neural Engine)
The 12 rows are the answer positions: hand gesture (1), facial expression (1), hand box (present + 4 edges), face box (present +
4 edges). Turning them into answers (softmax over each option's tokens, Liquid AI's calibration, box medians) is d1's own
readout; code/d1_common.py runs it through mlx-vlm's d1 code, so the answers have
exactly the format of model.system_one(...).
Run it with the Swift worker (cd code/swift && swift build -c release):
.build/release/d1worker --model ../../d1_ane_fp16.aimodel --compute ane # or d1_gpu_fp16.aimodel --compute gpu
# stdin, one JSON a line: {"pixels": "/path/frame.f32"} (raw float32 [1,3,352,640]) -> {"logits": [...], "ms": ...}
or from Python with coreai.runtime (code/coreai_backend.py is the backend the demo uses).
Measured (MacBook Pro M5 Max, macOS 27.3)
Parity with the MLX reference (LiquidAI/d1-2B-MLX-8bit, system_one) on 9 frames; a frame passes with the same top choice
for hand and face, that choice's probability within 0.05, and every box edge within 3% of the frame:
| GPU fp16 | Neural Engine fp16 | |
|---|---|---|
| frames passing | 9 / 9 | 8 / 9 |
| logits PSNR vs MLX | 53β60 dB | 46.0β55.1 dB |
| worst probability / box edge difference | 0.009 / 0.017 | 0.013 / 0.024 on real frames |
The Neural Engine's one miss is a synthetic uniform mid-gray frame: same answers, a probability 0.051 off. Every real frame passes.
Speed (same frame, median of 30 after 5 warm-ups) and power (whole Mac, macmon, 20 s of back-to-back decisions):
| MLX 8-bit | Core AI GPU | Core AI Neural Engine | |
|---|---|---|---|
| model time per decision | 43 ms | 171 ms | |
| per decision, preprocessing + model + readout | 59 ms | 55 ms | 182 ms |
| first load (specialization, once) | 12 s | 56 s | |
| cached load | 0.6 s | 0.3 s | |
| Mac power above idle | 131 W | 98 W | 17 W |
| energy per decision | 8.4 J | 6.0 J | 3.2 J |
The Neural Engine build runs entirely on the Neural Engine: its compiled plan is marked mps.fullyPlacedOnANE and
mps.noGPUActivity. It is about 3Γ slower than the GPU for this 2B model and uses about half the energy per decision.
Changes from the original
Modified by Ivan Fioravanti (notice under section 4(b) of the LFM Open License). Both .aimodel files are derived from the
LiquidAI/d1-2B weights:
- re-implemented d1's decision pass for these four questions as one fixed-shape PyTorch function and exported it with Apple's
Core AI tools (
coreai-torch0.4.2,coreai-modelsexport path); weights cast from bf16 to fp16; - the Neural Engine variant is re-authored for the Neural Engine (1Γ1 convolutions, per-head attention, fp16 constants)
and, to keep fp16 precision there, carries exact rescalings folded into its weights (the vision and text residual streams,
the patch embedding and the branch outputs before scale-invariant norms are carried at larger scale) and computes
sigmoidthroughtanh, and computes each MLP in 8 chunks of its hidden channels (1.4Γ faster there). The function is mathematically the same; numbers differ only by fp16 rounding; - the prompt for the four questions, its masks and position tables, and only the 1,023 embedding rows the answers are scored on are baked into the graph; the rest of the vocabulary is not included.
No retraining or fine-tuning. Int8 weights were tried and are not shipped: no GPU speedup; on the Neural Engine, per-channel int8 brought no speedup and per-block int8 does not compile for it (Core AI then silently runs it on the GPU).
License
Same license as the original weights: LFM Open License v1.0 (LICENSE, also at
LiquidAI/d1-2B). As there, commercial use is licensed only for
entities below US$10 million in annual revenue (section 5). "d1", "LFM" and "Liquid AI" are Liquid AI's names, used here
only to describe where this comes from.
Citation
Cite the original model:
@article{liquidAI2026opend1,
author = {Liquid AI},
title = {Open d1: Edge decision models for text, vision, and audio},
journal = {Liquid AI Blog},
year = {2026},
note = {https://www.liquid.ai/blog/open-d1},
}
@article{liquidAI2026LFM3,
author = {Liquid AI},
title = {Introducing LFM3: Every Modality for Specialized Intelligence},
journal = {Liquid AI Blog},
year = {2026},
note = {https://www.liquid.ai/blog/lfm3},
}