wegotomars/d3-nano-CoreAI-ANE

d3-nano on Apple Neural Engine

Core AI conversion of vllm-sr/d3-nano for typed yes/no, choice, and score decisions over text and images, using tinkerbox-ane.

This model has 2B parameters, for a larger 4B parameter model from the same family see wegotomars/d3-mini.

d3-nano accepts prompts of up to 32,768 tokens. The Gated DeltaNet and full-attention layers and the vision tower run fully on the ANE. Tokenization, image preprocessing, position tables, token-embedding lookup, and answer readout run on the CPU while your GPU stays idle and cool 😎❄️.

Requires macOS 27, Apple Silicon M4 or newer, and Python 3.12 or 3.13.

Usage

Install tinkerbox-ane>=2.2.0. Loading the model downloads only this Mac's bundle: its checkpoint folder carries the original model's code, tokenizer and configuration, and the token embeddings and vision position table the CPU uses.

import tinkerbox_ane

with tinkerbox_ane.load("d3-nano") as model:
    result = model.system_one("I was charged twice. Please refund me today.", {
        "refund": {"type": "noul", "instructions": "Is this a refund request?"},
    })
    print(result)

Images can accompany text, or be used alone:

from PIL import Image
import tinkerbox_ane

with tinkerbox_ane.load("d3-nano") as model, Image.open("receipt.jpg") as image:
    result = model.system_one(None, {
        "receipt": {"type": "noul", "instructions": "Is this a receipt?"},
    }, images=[image])
    print(result)

d3's full-attention layers let every prompt token see the question, so questions cannot share a pass. A call with several questions runs one pass per question, in order, and warns with SequentialQuestionsWarning. Each result's usage reports its tokens, Neural Engine time, and counted FLOPs.

Supported architectures

Architecure	SoC name
h16g	Apple M4
h16s	Apple M4 Pro
h16c	Apple M4 Max
h17g	Apple M5
h17s	Apple M5 Pro
h17c	Apple M5 Max
h17d	Apple M5 Ultra
h18g	Apple M6

Benchmarks

Measured October 26, 2026 on Apple M5 Pro (h17s, 24 GiB), macOS 27.0.1, one question, warm.

Metric	Result
Top-answer agreement with FP32 text reference	99.5%
Absolute probability drift, mean / maximum	0.0027 / 0.087
Official BF16 drift from FP32, mean / maximum	0.0011 / 0.025
Peak wired memory	3.8 GiB
Peak Python memory	0.76 GiB
123-token text prompt	34 ms
198-token text prompt	75 ms
284-token text prompt	110 ms
358-token text prompt	109 ms
841-token text prompt	261 ms
3,091-token text prompt	1.01 s
640 × 480 image prompt	434 ms
ANE energy, 123-token prompt	0.20 J
ANE energy, 198-token prompt	0.40 J
ANE energy, 284-token prompt	0.60 J
ANE energy, 358-token prompt	0.61 J
ANE energy, 841-token prompt	1.42 J
ANE energy, 3,091-token prompt	5.28 J
ANE energy, 640 × 480 image prompt	1.60 J

Prompts of 129 to 320 tokens take the chunked path, which keeps one set of weights resident to minimize memory instead of a wider, faster prompt function.

Original checkpoint: vllm-sr/d3-nano@462efc366229ea268cf4eed3738b3fc4415e4817. Weights, original code, and converted assets retain the Apache License 2.0. See NOTICE.