D1A-E2B-it QAT · Core ML

What it does: decisions, chat, and photos, voice clips and video, in Core ML. It is the Core ML conversion of JohnP1/d1a-e2b-it-qat-mlx-q4, the recommended D1A-E2B-it build: the same weights, the same adapter, the same temperature. 4.6 GB.

A small decision model for Apple devices: give it a document and a few questions, and it picks the answers. Each question is yes/no, multiple choice or a score; D1A returns a calibrated probability for every option in one forward pass, with no generated text, so every answer is a valid option. With its adapter switched off the same model is Gemma 4 E2B-it QAT and chats.

Status: what it does not do yet

Verified on a Mac only (Apple M1 Max, macOS, the GPU). Not tested on an iPhone or iPad yet.
The Neural Engine is not used: every model is set up for the GPU (compute_units ALL or CPU_AND_GPU). The Neural Engine needs a different weight layout, not built.
No ready-made app or server: this repository holds the Core ML models, the host-side data and a Python reference runner (reference/d1a_coreml_qat.py). An app reproduces the host side; io.json lists every input and output and the host rules.
Limits: decisions take up to 1,536 tokens (two prefill sizes, 512 and 1,536); chat prompts up to 1,536 tokens and 2,048 positions in all (prompt plus answer); greedy decoding only in the reference runner.
Video is slow on the Mac: a 16-frame clip takes 0.9 s to encode and 8 to 11 s to answer (about 1,200 tokens at the 1,536 size) on the M1 Max. Photos and voice clips take about 0.4 s per request.

Checks

All on an Apple M1 Max, against the MLX build and against fp32 PyTorch on the CPU (the exact arithmetic), before release.

Check	Result
Decisions: 28 real requests, 42 answers	0 answers flipped against the MLX build; largest probability change against fp32 0.0017 (the MLX build itself is 0.035 from fp32)
Photos, voice and video: the playground's 14 media requests, 28 answers	0 flipped against the MLX build; largest probability change 0.040
Photo and video encoder against fp32 (18 images and videos)	per-token cosine ≥ 0.9996
Voice encoder against fp32 (4 clips)	per-token cosine 1.0
Chat: 8 photos, a "what is this?" prompt, 427 answer tokens teacher-forced	final hidden state cosine ≥ 0.99977; 1 token of 427 different (a near-tie); logit margins within 0.35
Chat: 1,000 tokens in one answer	stable (finite throughout)
The 4-bit text weights	all 276 quantized layers hold Google's QAT integers and scales exactly (no re-quantization)

The fp32 reference for chat is PyTorch on the CPU. On the same Mac, PyTorch's MPS backend gave a different fp32 answer on these image prompts (final hidden states down to cosine 0.32 from the CPU's), while PyTorch on the CPU and this Core ML build agree, so MPS was not used as a reference. The cause was a PyTorch bug in MPS embedding lookups (jonpol01/d1a#353, pytorch/pytorch#188756), which zeroed the image start and end tokens' per-layer embeddings; d1a works around it since jonpol01/d1a#354.

The model itself is the MLX build's, measured on every D1A suite: against D1A-E2B-it v0.6 pooled +0.40 [−0.21, +1.02], PR severity +3.5, PR labels +1.7, 11% faster, but transfer-v4 −5.2 [−7.9, −2.6] (task types outside training), which D1A's release rule vetoes; it replaces v0.6 by the owner's decision of 2026-10-25. Every suite: the scorecard.

What's inside

File	What	Size
`coreml/trunk_00-07` … `trunk_29-35.mlpackage`	the 35 decoder layers in 5 parts; each holds `prefill_512`, `prefill_1536` and `decode` (one token over a 2,048-position KV cache), sharing one copy of the weights	1.1 GB
`coreml/lm_head.mlpackage`	the tied output layer, int8 per row, 16 parts of 16,384 logits (chat)	0.40 GB
`coreml/vision.mlpackage`	Gemma 4's vision tower and the QAT projection: `image` (2,520 patches → 280 tokens) and `frame` (630 → 70, video)	0.34 GB
`coreml/audio.mlpackage`	Gemma 4's audio conformer and projection, fp32: 3,000 mel frames (30 s) → 750 tokens	1.18 GB
`embeddings.safetensors`	token and per-layer embeddings, 4-bit in groups of 64, read row by row on the host	1.55 GB
`head.safetensors`, `d1a_config.json`	D1A's pointer head and its calibration temperature (1.55)
`config.json`, `processor_config.json`, `tokenizer*.json`, `chat_template.jinja`	Gemma 4 E2B-it's configuration, processor and tokenizer
`io.json`	every model function's inputs and outputs, and the host rules
`reference/d1a_coreml_qat.py`	the reference runner (Python)
`goldens/`	host inputs for one decision request, one photo and one voice clip, for testing a port

How the weights are held:

The text layers: all 276 int4 linear layers exactly as Google's quantization-aware training left them (google/gemma-4-E2B-it-qat-w4a16-ct, revision 971342c0): the checkpoint's own integers and scales as Core ML block-wise int4.
The D1A adapter (LoRA rank 16) is a separate fp16 branch scaled by an input, lora: 1 for decisions, 0 for chat. At 0 the model is the QAT base exactly.
The encoders: the photo and video encoder runs in fp16, except the pixel path up to the patch embedding, which runs in fp32 (some image tokens are sensitive to fp16 pixel rounding). The voice encoder runs in fp32 for the same reason.

Run it

The reference runner uses the d1a package for D1A's request encoding and media decoding:

pip install "d1a[serve,media] @ git+https://github.com/jonpol01/d1a" "coremltools>=8"
huggingface-cli download JohnP1/d1a-e2b-it-qat-coreml --local-dir d1a-e2b-it-qat-coreml

import sys; sys.path.insert(0, "d1a-e2b-it-qat-coreml/reference")
from d1a_coreml_qat import Bundle
b = Bundle("d1a-e2b-it-qat-coreml")          # compiles the models once, into d1a-e2b-it-qat-coreml/.compiled
b.decide("The customer was charged twice for the same order and wants a refund today.",
         {"team": {"type": "choice", "instructions": "Which team handles this?",
                   "criteria": {"billing": None, "shipping": None, "support": None}}})
b.decide("", {"damaged": {"type": "noul", "instructions": "Is the parcel damaged?"}},
         media=("image", open("parcel.jpg", "rb").read()))       # or ("audio", wav bytes), ("video", mp4 bytes)
text, tokens = b.chat("What is this?", image=open("photo.png", "rb").read())

The first load compiles the Core ML models (about a minute on an M1 Max).

License

Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). Code: github.com/jonpol01/d1a, built on Kev by Jared Palmer (Apache-2.0). Japanese decision data derived from JGLUE by Yahoo Japan Corporation and Waseda University (CC BY-SA 4.0). Pull-request data from NousResearch/hermes-agent (MIT); the labelled PR dataset itself, including the recent PRs, is private.