JohnP1/d1a-e2b-it-coreml

D1A-E2B-it v0.6 · Core ML int8 (text, ≤ 512 tokens, experimental)

A small decision model for the Apple Neural Engine: give it a document and a few questions, and it picks the answers. Each question is yes/no, multiple choice or a score; D1A returns a calibrated probability for every option in one forward pass, with no generated text, so every answer is a valid option.

This is an experimental build for on-device apps that already load Gemma 4 E2B-it: the Core ML conversion of JohnP1/d1a-e2b-it v0.6, D1A-E2B v0.6's recipe on the google/gemma-4-E2B-it trunk. Where it has been measured (on the MLX 8-bit build) it is below D1A-E2B v0.6: PR severity on 487 newer PRs 72.9% against 77.2% (−4.3 [−7.8, −1.0]), change type 87.3% against 87.5%, and media 21/28 against 25/28 agreement with E4B v0.6. E4B v0.6 remains the PR labeler. For anything but an app that already ships Gemma 4 E2B-it, use JohnP1/d1a-e2b-coreml v0.6.

This is the Core ML build: text only, documents up to 512 tokens, int8 weights (per block of 64), 3.5 GB. A 4-bit build is JohnP1/d1a-e2b-it-coreml-4bit. The conversion is the one described in jonpol01/d1a#224 for E2B v0.6, unchanged.

Status

• Not verified on iPhone. The 4-bit build was: JohnP1/d1a-e2b-it-coreml-4bit gives 0 of 42 answers different from the Mac on an iPhone 17 Pro, 0.66 s per request on its GPU. On a phone, prefer that build: these int8 chunks are about twice the size.
• Mac check on an Apple M1 Max, 28 real requests of up to 512 tokens (42 questions), against the fp32 PyTorch rebuild of the 8-bit export it was converted from: p50 18.0 s, p95 28.8 s per request (28 requests), 863 s to load (first, compiled); 0 of 42 answers change (0 of 40 clear-cut ones, where the reference's top answer leads by more than 0.05); largest probability change 0.019.
• E2B v0.6's identical conversion ran on the CPU on M1-class Macs (the M1's Neural Engine did not take it); expect the same here (not re-measured), so the M1 Max timing above is not a Neural Engine timing. E2B v0.6's identical conversion took 0.96 s per request on an M4's Neural Engine; this build has not been timed on an M4.
• The model behind it is experimental: its pre-registered bar (jonpol01/d1a#227, Bar B) failed on severity and media, and most suites (transfer-v4, hard, devtools, documents, decision-v7, JGLUE) are not measured yet. It is published by the owner's decision of 2026-10-12 for the iOS proof of concept.
• Not included: photo, voice and video (their encoders are separate models, not converted), and inputs longer than 512 tokens.

<bos>: encode with it

The Gemma 4 E2B-it tokenizer adds no <bos>, but this model was trained with one at the start of every record. d1a from jonpol01/d1a#261 on (merged as b4b0f989) puts it back (bos_token_id 2; d1a_config.json records leading_ids: [2]). An app that encodes requests itself must prepend id 2 before right-padding to 512.

Run it

Needs d1a with jonpol01/d1a#261:

pip install coremltools numpy torch transformers safetensors "d1a @ git+https://github.com/jonpol01/d1a"
hf download JohnP1/d1a-e2b-it-coreml --revision v0.6 --local-dir d1a-e2b-it-coreml
python d1a-e2b-it-coreml/reference/run.py d1a-e2b-it-coreml request.json

request.json is a System One request, for example:

{"state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.",
 "questions": {"team": {"type": "choice", "instructions": "Which team should handle this?", "criteria": {"returns": null, "shipping": null, "billing": null}},
               "urgent": {"type": "noul", "instructions": "Does this need a reply today?"}}}

The first run compiles the chunks into coreml/.compiled/ (a few minutes); later runs load them.

Files

Path	What
coreml/L512_<a>-<b>.mlpackage	5 chunks of the 35-layer text model, int8 weights (per block of 64). Chunk 7-15 outputs the shared keys and values that chunks 15-35 read.
embeddings.safetensors	the token and per-layer embedding tables (MLX affine-quantized, from JohnP1/d1a-e2b-it-mlx-q8@v0.6), read row by row on the host
head.safetensors, d1a_config.json	the pointer head, the temperature (1.41), the <bos>, pad and delimiter ids
config.json, tokenizer.json, tokenizer_config.json	the Gemma 4 E2B-it model config and tokenizer
io.json	each chunk's input names
reference/	Python reference: run.py (one request), workers.py (one process per chunk), d1a_coreml.py (host-side embeddings and masks), pipeline.py (all chunks in one process)

How a request runs

1. Encode it with D1A's layout (d1a.core.encoding.encode, d1a with #261, so it starts with <bos>, id 2) and right-pad it to 512 tokens with the pad token.
2. Look up the token embeddings (scaled by √hidden) and the per-layer embeddings (scaled by √256) on the host.
3. Build the block-causal masks (full and sliding), clamped to −1e4.
4. Run the chunks in order, each in its own process. Several large Neural Engine programs in one process can crash the runtime.
5. Read the hidden states at each question's <decide> and </opt> positions, score them with the head, divide by the temperature, and take the softmax.

macOS 15 / iOS 18 or later.

D1A-E2B-it builds

The same model in four builds. Mac checks on an Apple M1 Max, the same 28 requests (42 questions, all ≤ 512 tokens), each against the fp32 PyTorch rebuild of the 8-bit export:

Build	Repository	Runs on	Size	Mac check
MLX 8-bit	JohnP1/d1a-e2b-it-mlx-q8	Mac GPU	3.5 GB + 0.9 GB media	on its card
MLX 4-bit	JohnP1/d1a-e2b-it-mlx-q4	Mac GPU	2.5 GB + 0.9 GB media	on its card
Core ML int8	JohnP1/d1a-e2b-it-coreml	Neural Engine (CPU on M1)	3.5 GB (1.8 GB chunks + 1.6 GB embeddings)	p50 18.0 s, p95 28.8 s per request (28 requests), 863 s to load (first, compiled); 0 of 42 answers change (0 of 40 clear-cut ones, where the reference's top answer leads by more than 0.05); largest probability change 0.019.
Core ML int4	JohnP1/d1a-e2b-it-coreml-4bit	Neural Engine (M1: not checked)	2.6 GB (1.0 GB chunks + 1.6 GB embeddings)	on its card

License

Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). D1A is built on Kev by Jared Palmer (Apache-2.0).