---
license: apache-2.0
base_model: google/gemma-4-E2B-it
base_model_relation: adapter
library_name: peft
pipeline_tag: text-classification
language: [en, ja]
datasets: [JohnP1/d1a-ja-jglue, JohnP1/d1a-routing]
tags: [d1a, decision-model, calibration, lora, gemma4, typesafe, system-one, pull-requests, on-device, experimental]
---
# D1A-E2B-it v0.6 (experimental)
D1A is a small open decision model. One document and a set of typed questions go in (yes/no, choice, score), and a calibrated probability for every option comes out, in one forward pass, with no generated text. It speaks the TypeSafe System One API (`POST /v1/systemone`), so the TypeSafe SDK and existing clients work against it.
**This is an experimental build for on-device apps that already load Gemma 4 E2B-it.** It is D1A-E2B v0.6 trained again with one change, the trunk: [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it) instead of the base google/gemma-4-E2B. It exists for an iOS proof of concept that already ships the -it model. **It is worse than [JohnP1/d1a-e2b](https://huggingface.co/JohnP1/d1a-e2b) v0.6 where it has been measured**, and E4B v0.6 ([JohnP1/d1a-e4b](https://huggingface.co/JohnP1/d1a-e4b)) remains the PR labeler:
- PR severity on 487 pull requests newer than all training: **72.9%** against E2B v0.6's 77.2%, −4.3 [−7.8, −1.0];
- PR change type: 87.3% against 87.5%, −0.2 [−3.1, +2.7] (no measurable difference);
- photos, voice and video: all 14 demo requests answered, but **21 of 28** answers match E4B v0.6, where E2B v0.6 matches 25.
**On the iPhone (2026-10-13, iPhone 17 Pro, in the Mieru app):** this model runs as an unmerged LoRA adapter on the app's
own Gemma 4 E2B-it 4-bit, so it is one model in memory.
- **Accuracy:** 0 of 42 answers differ from the Mac.
- **Speed and memory:** 0.52 s per request; the adapter adds 25 MB.
- **Base untouched:** with the adapter off, the hidden states are bit-identical to the base.
The Core ML 4-bit build also matches the Mac on the phone (0 of 42; 0.66 s on its GPU). First use case: deciding whether a
new camera description is worth speaking (redundant / vague → skip / re-capture / speak). It took the intended action on
8 hand-written test cases, which is a smoke test, not a measurement.
Everything else (hard, developer-tool and long-document decisions, held-out transfer, general decisions, Japanese) is **not measured yet** on this build. Do not assume E2B v0.6's numbers carry over.
**Release note.** The bar registered for this run before training (jonpol01/d1a#227, Bar B, "good enough for apps") failed on two lines: severity −4.3 (it needed −3 or better) and media agreement 21/28 (it needed 25/28). It is published by the owner's decision of 2026-10-12 for the iOS proof of concept, as a separate repository and not as E2B's model.
**``: encode with it.** The Gemma 4 E2B-it tokenizer adds no ``, but this model was trained with one at the start of every record. d1a from jonpol01/d1a#261 on (merged as `b4b0f989`) puts it back (bos_token_id 2). If you encode requests yourself (another runtime, an app), prepend id 2; without it the answers are not the ones measured here.
## What's inside
A LoRA adapter (rank 16 on the attention and MLP projections: q, k, v, o, gate, up, down) and a small pointer head (256-dimensional) on [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it) at revision `3e22461f`.
| Stage | Training data | Settings |
|---|---|---|
| 1. General decisions (`recipes/d1a-e2b.yaml`, what made E2B v0.2) | decision-v7 suite: public classification datasets plus generated policy and rule data | 2 epochs, lr 1e-4, batch 4 × accum 2, from the -it trunk; not published |
| **2. Every later D1A skill, in one run (E2B v0.6's mix)** | PR labels: 4,000 English (severity-stratified), 753 Japanese, 1,500 with a blast-radius label, and 479 hermes-agent PRs created 2026-10-02 to 10-05 shown twice (once in each half); hard-v1 4,000; devtools-v1 4,000; documents-v1 2,500; JGLUE 2,000; routing: agent factory 1,216, generic 1,000; decision-v7 replay 1,500: 23,427 records | from stage 1, 1 epoch, lr 5e-5, states up to 5,120 tokens, 2,929 steps (2 h 45 m) |
Both stages ran in one Hugging Face Jobs job on one NVIDIA L4, about $3.51, finished 2026-10-12 18:33 JST. The mix is E2B v0.6's, byte for byte (sha256 `efff4b16`), screened before training against every evaluation partition, the 487 newer PRs below included. `provenance.json` has the details.
Calibration: a single temperature, **T = 1.41** (90% interval [1.35, 1.48]), fitted on pooled rows of the decision-v7 calibration split and the PR-labelling development set (3,178 questions). Calibration error is 0.053 before and 0.020 after, out of fold; on the 487 newer PRs below (974 questions) it is 0.021. It is stored in `d1a_config.json` and `head.pt`, and there are no per-use-case temperatures.
## Results
### Pull requests newer than all training
487 hermes-agent pull requests created after every PR in the training mix, held back before training. All models are MLX 8-bit at their served temperatures (E2B-it 1.41, E2B v0.6 1.45), with the serving context. Differences are paired over PRs, with 95% bootstrap intervals. E4B v0.6 is shown for reference:
| Question | Majority answer | **E2B-it v0.6** | E2B v0.6 | E4B v0.6 | E2B-it − E2B v0.6 |
|---|---|---|---|---|---|
| Severity (P0–P4) | 67.8% | **72.9%** | 77.2% | 79.1% | −4.3 [−7.8, −1.0] |
| Change type | 51.5% | **87.3%** | 87.5% | 90.6% | −0.2 [−3.1, +2.7] |
| P0 recall (7 PRs) | | **2/7** | 0/7 | 5/7 | |
| P1 recall (23 PRs) | | **1/23** | 0/23 | 8/23 | |
| Security type recall (14 PRs) | | **11/14** | 12/14 | 9/14 | |
It finds 3 of the 30 P0/P1 PRs where E2B v0.6 found none, but it loses more on P3 (273/330 against 295/330) and P2 (74/121 against 77/121) than it gains.
### Every other suite
**Not measured yet.** D1A's card-suite gate (transfer-v4, hard-v1, devtools-v1, documents-v1, decision-v7, JGLUE) has not run on this build. This card will not quote numbers for them until it has.
## Photos, voice and video
With the `media` extra, `d1a.serving.serve` answers questions about a photo, a voice clip or a video through Gemma 4 E2B-it's own vision and audio encoders, unchanged by the adapter (`POST /v1/systemone/media`). No media in training (zero-shot), measured on only the playground's 14 photo, voice and video demo requests. It answers all 14:
- **21 of 28** answers match E4B v0.6 (E2B v0.6: 25 of 28). It misses damage E4B sees (the crushed-truck photo and video, the wet-parcel video), calls two voice clips urgent (the English directions and the Japanese not-at-home message), and places the damaged-door and intact-mailbox photos "on a sidewalk".
- 19 of 28 match E2B v0.6. Unlike E2B v0.6, it calls the intact locker and mailbox photos intact.
## The questions it was trained for
PR labelling works best with these questions, worded exactly so, over a document of the form `title / author / stats / body / files` (one `- status path +added/-deleted` line per file):
- `type` (choice): "Primary change type from files and body, not the title prefix." Options: type/bug, type/docs, type/feature, type/perf, type/refactor, type/security, type/test.
- `blast` (choice): "How far a mistake in this PR spreads in production." Options: review:blast-contained (one module), -moderate (one subsystem), -broad (shared helper or config), -massive (auth, permissions, or all paths).
- `sev` (choice): "How serious the problem this PR addresses is — not the risk of merging the diff as-is." Options P0 to P4.
The [PR-labeler recipe](https://github.com/jonpol01/d1a/tree/main/recipes/pr-labeler) has the exact wording, the document builder and the training and scoring scripts.
## Run it
Needs d1a with jonpol01/d1a#261 (the `` above):
```bash
pip install "d1a[serve] @ git+https://github.com/jonpol01/d1a"
python -m d1a.serving.serve --run JohnP1/d1a-e2b-it@v0.6 --port 8009
```
or in-process: `from d1a import D1A; D1A.load("JohnP1/d1a-e2b-it@v0.6").decide(state, questions)`. Builds of the same model: MLX 8-bit [JohnP1/d1a-e2b-it-mlx-q8](https://huggingface.co/JohnP1/d1a-e2b-it-mlx-q8) (with the photo, voice and video encoders), MLX 4-bit [JohnP1/d1a-e2b-it-mlx-q4](https://huggingface.co/JohnP1/d1a-e2b-it-mlx-q4), Core ML [JohnP1/d1a-e2b-it-coreml](https://huggingface.co/JohnP1/d1a-e2b-it-coreml) (int8) and [JohnP1/d1a-e2b-it-coreml-4bit](https://huggingface.co/JohnP1/d1a-e2b-it-coreml-4bit) (int4). Unless your app already loads Gemma 4 E2B-it, use [JohnP1/d1a-e2b](https://huggingface.co/JohnP1/d1a-e2b) v0.6 instead.
## Limits
- **Experimental, and below E2B v0.6 where measured**: severity −4.3 points on newer PRs, media 21/28 against 25/28 agreement with E4B v0.6. Its pre-registered bar failed (above).
- **Not a PR labeller.** E4B v0.6 remains the PR labeler. On the 487 newer PRs this build finds 3 of 30 P0/P1 PRs (E4B v0.6: 13 of 30). Use E4B for severity, or flag a PR when p(P0) + p(P1) is high instead of taking the top answer.
- **Most suites are not measured**: hard, developer-tool and long-document decisions, held-out transfer, general decisions (decision-v7) and Japanese (JGLUE). Treat its answers there as unknown quality.
- **Gate tool calls through D1A's fail-safe thresholds** (`d1a.agents.presets.advise`: deny at p ≥ 0.5, allow only at p ≥ 0.8, otherwise ask), never through the raw top answer. E2B v0.6 gave a near-tie on the playground's $4,999 payment example; this build's demo answers have not been read yet.
- **``**: requests encoded without the leading id 2 are out of distribution (see above).
- **Media**: zero-shot, measured on 14 demo requests only. It misses damage on 3 of them (above).
- One temperature for every question, and no use-case temperatures.
- The recent PR labels come from one large open-source project and a few days of it. If its conventions move again, so will these numbers.
- Questions are in English. The documents can be English or Japanese (trained on both; Japanese not measured on this build).
## Versions
| Tag | What it is |
|---|---|
| **`v0.6`** | **E2B v0.6's recipe on the Gemma 4 E2B-it trunk; experimental, Bar B of #227 failed (severity −4.3, media 21/28)** |
## License and data
Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). Code: [github.com/jonpol01/d1a](https://github.com/jonpol01/d1a), built on Kev by Jared Palmer (Apache-2.0). Japanese decision data derived from JGLUE by Yahoo Japan Corporation and Waseda University (CC BY-SA 4.0). Pull-request data from NousResearch/hermes-agent (MIT); the labelled PR dataset itself, including the recent PRs, is private.