README.md

metadata

license: apache-2.0
license_link: LICENSE
base_model: vllm-sr/d3-mini
pipeline_tag: zero-shot-classification
library_name: coreai
tags:
  - vllm-sr
  - qwen3_5
  - decision
  - classification
  - system-one
  - decision-model
  - apple
  - coreai
  - ane
  - apple-neural-engine
  - fp16
  - tinkerbox

d3-mini on Apple Neural Engine

Core AI conversion of vllm-sr/d3-mini for typed yes/no, choice, and score decisions over text and images, using tinkerbox-ane.

This model has 4B parameters, for a samller 2B parameter model from the same family see wegotomars/d3-nano.

d3-mini accepts prompts of up to 32,768 tokens. The Gated DeltaNet and full-attention layers and the vision tower run fully on the ANE. Tokenization, image preprocessing, position tables, token-embedding lookup, and answer readout run on the CPU while your GPU stays idle and cool 😎❄️.

Requires macOS 27, Apple Silicon M4 or newer, and Python 3.12 or 3.13.

Usage

Install tinkerbox-ane>=2.2.0. Loading the model downloads only this Mac's bundle: its checkpoint folder carries the original model's code, tokenizer and configuration, and the token embeddings and vision position table the CPU uses.

import tinkerbox_ane

with tinkerbox_ane.load("d3-mini") as model:
    result = model.system_one("I was charged twice. Please refund me today.", {
        "refund": {"type": "noul", "instructions": "Is this a refund request?"},
    })
    print(result)

Images can accompany text, or be used alone:

from PIL import Image
import tinkerbox_ane

with tinkerbox_ane.load("d3-mini") as model, Image.open("receipt.jpg") as image:
    result = model.system_one(None, {
        "receipt": {"type": "noul", "instructions": "Is this a receipt?"},
    }, images=[image])
    print(result)

d3's full-attention layers let every prompt token see the question, so questions cannot share a pass. A call with several questions runs one pass per question, in order, and warns with SequentialQuestionsWarning. Each result's usage reports its tokens, Neural Engine time, and counted FLOPs.

Supported architectures

Architecure	SoC name
h16g	Apple M4
h16s	Apple M4 Pro
h16c	Apple M4 Max
h17g	Apple M5
h17s	Apple M5 Pro
h17c	Apple M5 Max
h17d	Apple M5 Ultra
h18g	Apple M6

Benchmarks

Measured October 26, 2026 on Apple M5 Pro (h17s, 24 GiB), macOS 27.0.1, one question, warm.

Metric	Result
Top-answer agreement with FP32 text reference	100%
Absolute probability drift, mean / maximum	0.0008 / 0.025
Official BF16 drift from FP32, mean / maximum	0.0006 / 0.018
Peak wired memory	9.0 GiB
Peak Python memory	0.76 GiB
123-token text prompt	86 ms
198-token text prompt	183 ms
284-token text prompt	276 ms
358-token text prompt	276 ms
841-token text prompt	0.65 s
3,091-token text prompt	2.47 s
640 × 480 image prompt	0.67 s
ANE energy, 123-token prompt	0.52 J
ANE energy, 198-token prompt	1.01 J
ANE energy, 284-token prompt	1.56 J
ANE energy, 358-token prompt	1.58 J
ANE energy, 841-token prompt	3.68 J
ANE energy, 3,091-token prompt	13.7 J
ANE energy, 640 × 480 image prompt	2.88 J

Prompts of 129 to 320 tokens take the chunked path, which keeps one set of weights resident to minimize memory instead of a wider, faster prompt function.

Original checkpoint: vllm-sr/d3-mini@bd0090dd758d32ba59ef5c749910a8e87fc4391f. Weights, original code, and converted assets retain the Apache License 2.0. See NOTICE.