D1A-E2B-it v0.6 (experimental)

D1A is a small open decision model. One document and a set of typed questions go in (yes/no, choice, score), and a calibrated probability for every option comes out, in one forward pass, with no generated text. It speaks the TypeSafe System One API (POST /v1/systemone), so the TypeSafe SDK and existing clients work against it.

This is an experimental build for on-device apps that already load Gemma 4 E2B-it. It is D1A-E2B v0.6 trained again with one change, the trunk: google/gemma-4-E2B-it instead of the base google/gemma-4-E2B. It exists for an iOS proof of concept that already ships the -it model. It is worse than JohnP1/d1a-e2b v0.6 where it has been measured, and E4B v0.6 (JohnP1/d1a-e4b) remains the PR labeler:

• PR severity on 487 pull requests newer than all training: 72.9% against E2B v0.6's 77.2%, −4.3 [−7.8, −1.0];
• PR change type: 87.3% against 87.5%, −0.2 [−3.1, +2.7] (no measurable difference);
• photos, voice and video: all 14 demo requests answered, but 21 of 28 answers match E4B v0.6, where E2B v0.6 matches 25.

On the iPhone (2026-10-13, iPhone 17 Pro, in the Mieru app): this model runs as an unmerged LoRA adapter on the app's own Gemma 4 E2B-it 4-bit, so it is one model in memory.

• Accuracy: 0 of 42 answers differ from the Mac.
• Speed and memory: 0.52 s per request; the adapter adds 25 MB.
• Base untouched: with the adapter off, the hidden states are bit-identical to the base.

The Core ML 4-bit build also matches the Mac on the phone (0 of 42; 0.66 s on its GPU). First use case: deciding whether a new camera description is worth speaking (redundant / vague → skip / re-capture / speak). It took the intended action on 8 hand-written test cases, which is a smoke test, not a measurement.

Everything else (hard, developer-tool and long-document decisions, held-out transfer, general decisions, Japanese) is not measured yet on this build. Do not assume E2B v0.6's numbers carry over.

Release note. The bar registered for this run before training (jonpol01/d1a#227, Bar B, "good enough for apps") failed on two lines: severity −4.3 (it needed −3 or better) and media agreement 21/28 (it needed 25/28). It is published by the owner's decision of 2026-10-12 for the iOS proof of concept, as a separate repository and not as E2B's model.

<bos>: encode with it. The Gemma 4 E2B-it tokenizer adds no <bos>, but this model was trained with one at the start of every record. d1a from jonpol01/d1a#261 on (merged as b4b0f989) puts it back (bos_token_id 2). If you encode requests yourself (another runtime, an app), prepend id 2; without it the answers are not the ones measured here.

What's inside

A LoRA adapter (rank 16 on the attention and MLP projections: q, k, v, o, gate, up, down) and a small pointer head (256-dimensional) on google/gemma-4-E2B-it at revision 3e22461f.

Stage	Training data	Settings
1. General decisions (recipes/d1a-e2b.yaml, what made E2B v0.2)	decision-v7 suite: public classification datasets plus generated policy and rule data	2 epochs, lr 1e-4, batch 4 × accum 2, from the -it trunk; not published
2. Every later D1A skill, in one run (E2B v0.6's mix)	PR labels: 4,000 English (severity-stratified), 753 Japanese, 1,500 with a blast-radius label, and 479 hermes-agent PRs created 2026-10-02 to 10-05 shown twice (once in each half); hard-v1 4,000; devtools-v1 4,000; documents-v1 2,500; JGLUE 2,000; routing: agent factory 1,216, generic 1,000; decision-v7 replay 1,500: 23,427 records	from stage 1, 1 epoch, lr 5e-5, states up to 5,120 tokens, 2,929 steps (2 h 45 m)

Both stages ran in one Hugging Face Jobs job on one NVIDIA L4, about $3.51, finished 2026-10-12 18:33 JST. The mix is E2B v0.6's, byte for byte (sha256 efff4b16), screened before training against every evaluation partition, the 487 newer PRs below included. provenance.json has the details.

Calibration: a single temperature, T = 1.41 (90% interval [1.35, 1.48]), fitted on pooled rows of the decision-v7 calibration split and the PR-labelling development set (3,178 questions). Calibration error is 0.053 before and 0.020 after, out of fold; on the 487 newer PRs below (974 questions) it is 0.021. It is stored in d1a_config.json and head.pt, and there are no per-use-case temperatures.

Results

Pull requests newer than all training

487 hermes-agent pull requests created after every PR in the training mix, held back before training. All models are MLX 8-bit at their served temperatures (E2B-it 1.41, E2B v0.6 1.45), with the serving context. Differences are paired over PRs, with 95% bootstrap intervals. E4B v0.6 is shown for reference:

Question	Majority answer	E2B-it v0.6	E2B v0.6	E4B v0.6	E2B-it − E2B v0.6
Severity (P0–P4)	67.8%	72.9%	77.2%	79.1%	−4.3 [−7.8, −1.0]
Change type	51.5%	87.3%	87.5%	90.6%	−0.2 [−3.1, +2.7]
P0 recall (7 PRs)		2/7	0/7	5/7
P1 recall (23 PRs)		1/23	0/23	8/23
Security type recall (14 PRs)		11/14	12/14	9/14

It finds 3 of the 30 P0/P1 PRs where E2B v0.6 found none, but it loses more on P3 (273/330 against 295/330) and P2 (74/121 against 77/121) than it gains.

Every other suite

Not measured yet. D1A's card-suite gate (transfer-v4, hard-v1, devtools-v1, documents-v1, decision-v7, JGLUE) has not run on this build. This card will not quote numbers for them until it has.

Photos, voice and video

With the media extra, d1a.serving.serve answers questions about a photo, a voice clip or a video through Gemma 4 E2B-it's own vision and audio encoders, unchanged by the adapter (POST /v1/systemone/media). No media in training (zero-shot), measured on only the playground's 14 photo, voice and video demo requests. It answers all 14:

• 21 of 28 answers match E4B v0.6 (E2B v0.6: 25 of 28). It misses damage E4B sees (the crushed-truck photo and video, the wet-parcel video), calls two voice clips urgent (the English directions and the Japanese not-at-home message), and places the damaged-door and intact-mailbox photos "on a sidewalk".
• 19 of 28 match E2B v0.6. Unlike E2B v0.6, it calls the intact locker and mailbox photos intact.

The questions it was trained for

PR labelling works best with these questions, worded exactly so, over a document of the form title / author / stats / body / files (one - status path +added/-deleted line per file):

• type (choice): "Primary change type from files and body, not the title prefix." Options: type/bug, type/docs, type/feature, type/perf, type/refactor, type/security, type/test.
• blast (choice): "How far a mistake in this PR spreads in production." Options: review:blast-contained (one module), -moderate (one subsystem), -broad (shared helper or config), -massive (auth, permissions, or all paths).
• sev (choice): "How serious the problem this PR addresses is — not the risk of merging the diff as-is." Options P0 to P4.

The PR-labeler recipe has the exact wording, the document builder and the training and scoring scripts.

Run it

Needs d1a with jonpol01/d1a#261 (the <bos> above):

pip install "d1a[serve] @ git+https://github.com/jonpol01/d1a"
python -m d1a.serving.serve --run JohnP1/d1a-e2b-it@v0.6 --port 8009

or in-process: from d1a import D1A; D1A.load("JohnP1/d1a-e2b-it@v0.6").decide(state, questions). Builds of the same model: MLX 8-bit JohnP1/d1a-e2b-it-mlx-q8 (with the photo, voice and video encoders), MLX 4-bit JohnP1/d1a-e2b-it-mlx-q4, Core ML JohnP1/d1a-e2b-it-coreml (int8) and JohnP1/d1a-e2b-it-coreml-4bit (int4). Unless your app already loads Gemma 4 E2B-it, use JohnP1/d1a-e2b v0.6 instead.

Limits

• Experimental, and below E2B v0.6 where measured: severity −4.3 points on newer PRs, media 21/28 against 25/28 agreement with E4B v0.6. Its pre-registered bar failed (above).
• Not a PR labeller. E4B v0.6 remains the PR labeler. On the 487 newer PRs this build finds 3 of 30 P0/P1 PRs (E4B v0.6: 13 of 30). Use E4B for severity, or flag a PR when p(P0) + p(P1) is high instead of taking the top answer.
• Most suites are not measured: hard, developer-tool and long-document decisions, held-out transfer, general decisions (decision-v7) and Japanese (JGLUE). Treat its answers there as unknown quality.
• Gate tool calls through D1A's fail-safe thresholds (d1a.agents.presets.advise: deny at p ≥ 0.5, allow only at p ≥ 0.8, otherwise ask), never through the raw top answer. E2B v0.6 gave a near-tie on the playground's $4,999 payment example; this build's demo answers have not been read yet.
• <bos>: requests encoded without the leading id 2 are out of distribution (see above).
• Media: zero-shot, measured on 14 demo requests only. It misses damage on 3 of them (above).
• One temperature for every question, and no use-case temperatures.
• The recent PR labels come from one large open-source project and a few days of it. If its conventions move again, so will these numbers.
• Questions are in English. The documents can be English or Japanese (trained on both; Japanese not measured on this build).

Versions

Tag	What it is
v0.6	E2B v0.6's recipe on the Gemma 4 E2B-it trunk; experimental, Bar B of #227 failed (severity −4.3, media 21/28)

License and data

Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). Code: github.com/jonpol01/d1a, built on Kev by Jared Palmer (Apache-2.0). Japanese decision data derived from JGLUE by Yahoo Japan Corporation and Waseda University (CC BY-SA 4.0). Pull-request data from NousResearch/hermes-agent (MIT); the labelled PR dataset itself, including the recent PRs, is private.