D1A-E2B-it QAT · MLX 4-bit

This repository mirrors JohnP1/d1a-e2b-it-qat-mlx-q4 (revision f6b7ee51) byte for byte (the same files, checked by LFS sha256); only this card differs.

What it does: decisions (/v1/systemone), chat (OpenAI-format /v1/chat/completions, streaming included), and photos, voice clips and video (/v1/systemone/media). One model on Apple Silicon (MLX), 2.7 GB plus 1.0 GB of encoders. The recommended D1A-E2B-it build: it replaces D1A-E2B-it v0.6 by the owner's decision of 2026-10-25. It is better at D1A's own jobs (PR labels, routing, hard decisions), 11% faster, 4-bit and able to chat. D1A's release rule vetoes it on transfer-v4 (task types it never trained on, −5.2 points), which is shown below; pin JohnP1/d1a-e2b-it@v0.6 where that matters.

A small decision model for Apple Silicon: give it a document and a few questions, and it picks the answers. Each question is yes/no, multiple choice or a score; D1A returns a calibrated probability for every option in one forward pass, with no generated text, so every answer is a valid option.

This build sits exactly on Google's quantization-aware-trained int4 weights, google/gemma-4-E2B-it-qat-w4a16-ct (revision 971342c0). All 276 of its int4 linear layers are stored here bit for bit as MLX 4-bit: the same packed words and scales, dequantization error 0.0. Nothing is re-quantized from bf16. The D1A adapter was retrained on this exact trunk, so the 4-bit model is the model that was trained, not a rounded copy of it.

Why it was retrained: the D1A-E2B-it v0.6 adapter, moved onto the QAT trunk unchanged, lost accuracy and changed 9.6% of its clear-cut answers, because the QAT weights differ from the bf16 ones (cosine 0.992):

v0.6 adapter on	hard-v1	devtools-v1	documents-v1
the bf16 trunk (as published)	0.689	0.709	0.862
this QAT trunk, not retrained	0.669	0.683	0.857

Scorecard

Every suite, with intervals: docs/reports/2026-10-25-e2b-it-qat-scorecard.md.

Every D1A suite, the demos and the live labeler's data (scripts/quality_gate.py --all-suites, scripts/decide.py), on an Apple M1 Max, both sides as served. Δ in points with paired 95% intervals.

Against D1A-E2B-it v0.6 on the bf16 trunk (the build it replaces):

pooled over 22 suites (17,418 questions): +0.40 [−0.21, +1.02];
significant wins: PR severity on newer PRs +3.5 [+1.4, +5.7], PR labels +1.7 [+0.4, +2.9]; hard-v1 +2.2 [−0.1, +4.5];
significant loss: transfer-v4 −5.2 [−7.9, −2.6], task types outside training. D1A's release rule vetoes this build as a replacement for v0.6 (V1, V4). Released as an exception by the owner's decision of 2026-10-25: D1A's own jobs weigh more;
11% faster per decision (latency ratio 0.89).

Against the v0.6 adapter on this same QAT trunk (the retrain alone):

pooled +2.13 [+1.36, +2.89]: 8 significant wins (hard, devtools, PR labels, routing, the hand-checked PRs) and 0 significant losses; transfer-v4 −0.2. So the transfer-v4 loss comes from the 4-bit trunk, not the retrain;
one weakness the retrain added: held-out rule compositions inside transfer-v4, 4 fixes against 13 breaks (sign p 0.049).

Calibration: better than v0.6 on hard-v1 and routing; worse on some suites (PR labels test ECE 0.019 → 0.038, transfer-v4 0.057 → 0.124).

Photos, voice and video (the playground's 14 media requests): 14 of 14 answered, p50 1.6 s per request; 25 of 28 answers the same as D1A-E2B-it v0.6 (8-bit). All 3 differences are on voice clips: two urgent flags, and one intent ("the customer is not home" read as "report a damaged parcel").

What's inside

The trunk: Gemma 4 E2B-it QAT, its 276 int4 linear layers exactly as Google trained them (symmetric, groups of 32); token and per-layer embeddings 4-bit in groups of 64 (QAT fixes only the Linears' groups of 32). 2.7 GB.
The D1A adapter, unmerged (adapter/, LoRA rank 16, 97 MB). With the adapter off the model is Gemma 4 E2B-it QAT itself, so the same weights also chat.
The pointer head (head.safetensors).
Calibration temperature 1.55 in d1a_config.json, fitted on this build's own answers: 3,178 rows from the decision-v7 calibration split and the PR-label development set, the same recipe as v0.6. Expected calibration error 0.064 → 0.018 (the same out of fold). Both sets come from corpora the model trained on, which the fit records.
media/: Gemma 4's vision and audio encoders from the QAT checkpoint itself (bf16, 1.0 GB). They are the same as E2B-it's, except the vision projection, which QAT retrained for its int4 text model.

Training

D1A-E2B-it v0.6's recipe, continued on the QAT trunk (jonpol01/d1a#341):

warm start from JohnP1/d1a-e2b-it v0.6;
one epoch of v0.6's all-skills mix: 23,427 records from 11 sources (PR labels in English and Japanese, blast radius, recent PRs, hard, devtools, documents, JGLUE, routing, decision-v7);
the trunk frozen at the QAT checkpoint's values, dequantized exactly (scale × integer); learning rate 2e-5, batch 8;
2,929 steps, 12.5 hours on an Apple M1 Max (MPS).

Checks so far

Group-64 embeddings (this revision): against the earlier group-32 build of the same adapter, hard-v1 71.1 → 71.1, devtools-v1 70.6 → 70.3 (7 fixes, 10 breaks; sign p 0.63), documents-v1 86.8 → 86.8; the 14 media requests all answered, 26 of 28 the same; 0.17 GB smaller. The scorecard above was measured on the group-32 build.
Exactness: 276 of 276 quantized linear layers identical to the QAT checkpoint (words, scales and biases).
Serving (d1a.serving.serve, Apple M1 Max): the README's example request answered support (0.67) in 109 ms; with the adapter off, chat answers and streams.

Run it

Needs d1a with jonpol01/d1a#261:

pip install "d1a[serve,mlx,media] @ git+https://github.com/jonpol01/d1a"
python -m d1a.serving.serve --run mlx-community/d1a-e2b-it-qat-4bit --port 8009

A decision:

curl -s localhost:8009/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.",
  "questions": {"team": {"type": "choice", "instructions": "Which team should handle this?",
                         "criteria": {"returns": null, "shipping": null, "billing": null}}}
}'

A photo, voice clip or video: the same request with a media field. type is image (JPEG, PNG, WebP), audio (WAV, FLAC or OGG, up to 30 s) or video (MP4, MOV or WebM, up to 32 MB; 16 frames sampled evenly, the sound track not used):

curl -s localhost:8009/v1/systemone/media -H 'content-type: application/json' -d '{
  "media": {"type": "image", "data": "'"$(base64 -i parcel.jpg)"'"},
  "questions": {"damaged": {"type": "noul", "instructions": "Is the parcel damaged?"}}
}'

Chat (the adapter is switched off for it, so this is Gemma 4 E2B-it QAT itself):

curl -s localhost:8009/v1/chat/completions -H 'content-type: application/json' -d '{
  "messages": [{"role": "user", "content": "Name three things to check when a parcel arrives damaged."}]
}'

or in-process: from d1a import D1A; D1A.load("mlx-community/d1a-e2b-it-qat-4bit").decide(state, questions). Chat goes to the same server's OpenAI-format /v1/chat/completions, with the adapter off.

Limits

Release exception: transfer-v4 −5.2 (above), and the labeler replay's blast-radius answers 64.3 → 59.2. Pin JohnP1/d1a-e2b-it@v0.6 for task types far from D1A's own.
Photos, voice and video were checked on the 14 demo requests only, zero-shot as on every D1A-E2B-it build.
<bos>: the model was trained with id 2 at the start of every record; d1a adds it (leading_ids: [2]). An app that encodes requests itself must prepend it.
Questions are in English. The documents can be English or Japanese.
Everything on the D1A-E2B-it card applies: experimental, and agent tool calls go through D1A's fail-safe thresholds (d1a.agents.presets.advise).

License and data

Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). Code: github.com/jonpol01/d1a, built on Kev by Jared Palmer (Apache-2.0). Japanese decision data derived from JGLUE by Yahoo Japan Corporation and Waseda University (CC BY-SA 4.0). Pull-request data from NousResearch/hermes-agent (MIT); the labelled PR dataset itself, including the recent PRs, is private.