d1-2B gestures on Core AI (GPU and Neural Engine)

A Core AI conversion of LiquidAI/d1-2B by Liquid AI, for the d1 gestures webcam demo: one frame in, hand gesture, facial expression, hand box and face box out, in one forward pass, on the Mac's GPU or Neural Engine through Apple's Core AI (macOS 27).

This is not a general d1 conversion. The export is one fixed-shape function with the prompt of four questions baked in (code/questions.py): a 640Γ—360 frame (16:9; other shapes are centre-cropped) β†’ the answer logits. For any other questions, use the original model: LiquidAI/d1-2B (PyTorch) or LiquidAI/d1-2B-MLX-8bit (MLX).

File What it is
d1_gpu_fp16.aimodel/ the function for the GPU, fp16, 3.1 GB
d1_ane_fp16.aimodel/ the same function re-authored for the Neural Engine, fp16, 3.1 GB
prompt.json the baked prompt: token ids, the 12 answer positions, the 1,023 answer tokens they are scored on
calibration.json Liquid AI's calibration, unchanged from LiquidAI/d1-2B
code/ the export (d1_torch.py, d1_ane.py, export_*.py), the parity and speed checks, the host readout, the Swift worker
LICENSE the LFM Open License v1.0, as distributed with the original weights

Function

image   float32 [1, 3, 352, 640]   the 640x360 frame after the d1 image processor (resize to 640x352, (x/255 - 0.5)/0.5)
logits  [12, 1023] (GPU) / [1, 1023, 1, 12] fp16 (Neural Engine)

The 12 rows are the answer positions: hand gesture (1), facial expression (1), hand box (present + 4 edges), face box (present + 4 edges). Turning them into answers (softmax over each option's tokens, Liquid AI's calibration, box medians) is d1's own readout; code/d1_common.py runs it through mlx-vlm's d1 code, so the answers have exactly the format of model.system_one(...).

Run it with the Swift worker (cd code/swift && swift build -c release):

.build/release/d1worker --model ../../d1_ane_fp16.aimodel --compute ane     # or d1_gpu_fp16.aimodel --compute gpu
# stdin, one JSON a line:  {"pixels": "/path/frame.f32"}   (raw float32 [1,3,352,640])  ->  {"logits": [...], "ms": ...}

or from Python with coreai.runtime (code/coreai_backend.py is the backend the demo uses).

Measured (MacBook Pro M5 Max, macOS 27.3)

Parity with the MLX reference (LiquidAI/d1-2B-MLX-8bit, system_one) on 9 frames; a frame passes with the same top choice for hand and face, that choice's probability within 0.05, and every box edge within 3% of the frame:

GPU fp16 Neural Engine fp16
frames passing 9 / 9 8 / 9
logits PSNR vs MLX 53–60 dB 46.0–55.1 dB
worst probability / box edge difference 0.009 / 0.017 0.013 / 0.024 on real frames

The Neural Engine's one miss is a synthetic uniform mid-gray frame: same answers, a probability 0.051 off. Every real frame passes.

Speed (same frame, median of 30 after 5 warm-ups) and power (whole Mac, macmon, 20 s of back-to-back decisions):

MLX 8-bit Core AI GPU Core AI Neural Engine
model time per decision 43 ms 171 ms
per decision, preprocessing + model + readout 59 ms 55 ms 182 ms
first load (specialization, once) 12 s 56 s
cached load 0.6 s 0.3 s
Mac power above idle 131 W 98 W 17 W
energy per decision 8.4 J 6.0 J 3.2 J

The Neural Engine build runs entirely on the Neural Engine: its compiled plan is marked mps.fullyPlacedOnANE and mps.noGPUActivity. It is about 3Γ— slower than the GPU for this 2B model and uses about half the energy per decision.

Changes from the original

Modified by Ivan Fioravanti (notice under section 4(b) of the LFM Open License). Both .aimodel files are derived from the LiquidAI/d1-2B weights:

  • re-implemented d1's decision pass for these four questions as one fixed-shape PyTorch function and exported it with Apple's Core AI tools (coreai-torch 0.4.2, coreai-models export path); weights cast from bf16 to fp16;
  • the Neural Engine variant is re-authored for the Neural Engine (1Γ—1 convolutions, per-head attention, fp16 constants) and, to keep fp16 precision there, carries exact rescalings folded into its weights (the vision and text residual streams, the patch embedding and the branch outputs before scale-invariant norms are carried at larger scale) and computes sigmoid through tanh, and computes each MLP in 8 chunks of its hidden channels (1.4Γ— faster there). The function is mathematically the same; numbers differ only by fp16 rounding;
  • the prompt for the four questions, its masks and position tables, and only the 1,023 embedding rows the answers are scored on are baked into the graph; the rest of the vocabulary is not included.

No retraining or fine-tuning. Int8 weights were tried and are not shipped: no GPU speedup; on the Neural Engine, per-channel int8 brought no speedup and per-block int8 does not compile for it (Core AI then silently runs it on the GPU).

License

Same license as the original weights: LFM Open License v1.0 (LICENSE, also at LiquidAI/d1-2B). As there, commercial use is licensed only for entities below US$10 million in annual revenue (section 5). "d1", "LFM" and "Liquid AI" are Liquid AI's names, used here only to describe where this comes from.

Citation

Cite the original model:

@article{liquidAI2026opend1,
  author  = {Liquid AI},
  title   = {Open d1: Edge decision models for text, vision, and audio},
  journal = {Liquid AI Blog},
  year    = {2026},
  note    = {https://www.liquid.ai/blog/open-d1},
}
@article{liquidAI2026LFM3,
  author  = {Liquid AI},
  title   = {Introducing LFM3: Every Modality for Specialized Intelligence},
  journal = {Liquid AI Blog},
  year    = {2026},
  note    = {https://www.liquid.ai/blog/lfm3},
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for ivanfioravanti/d1-2B-gestures-CoreAI

Finetuned
LiquidAI/d1-2B
Finetuned
(11)
this model