README.md

---
license: apache-2.0
base_model: google/gemma-4-E2B-it
base_model_relation: adapter
library_name: peft
pipeline_tag: text-classification
language: [en, ja]
datasets: [JohnP1/d1a-ja-jglue, JohnP1/d1a-routing]
tags: [d1a, decision-model, calibration, lora, gemma4, typesafe, system-one, pull-requests, on-device, experimental]
---

<p align="center"><img src="https://huggingface.co/JohnP1/d1a-e2b-it/resolve/main/d1a-avatar.png" width="200" alt="D1A avatar: an anime girl with silver-white and dark purple hair, round glasses and headphones on a round #D1A badge"></p>

# D1A-E2B-it v0.6 (experimental)

D1A is a small open decision model. One document and a set of typed questions go in (yes/no, choice, score), and a calibrated probability for every option comes out, in one forward pass, with no generated text. It speaks the TypeSafe System One API (`POST /v1/systemone`), so the TypeSafe SDK and existing clients work against it.

**This is an experimental build for on-device apps that already load Gemma 4 E2B-it.** It is D1A-E2B v0.6 trained again with one change, the trunk: [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it) instead of the base google/gemma-4-E2B. It exists for an iOS proof of concept that already ships the -it model. **It is worse than [JohnP1/d1a-e2b](https://huggingface.co/JohnP1/d1a-e2b) v0.6 where it has been measured**, and E4B v0.6 ([JohnP1/d1a-e4b](https://huggingface.co/JohnP1/d1a-e4b)) remains the PR labeler:
- PR severity on 487 pull requests newer than all training: **72.9%** against E2B v0.6's 77.2%, −4.3 [−7.8, −1.0];
- PR change type: 87.3% against 87.5%, −0.2 [−3.1, +2.7] (no measurable difference);
- photos, voice and video: all 14 demo requests answered, but **21 of 28** answers match E4B v0.6, where E2B v0.6 matches 25.

**On the iPhone (2026-10-13, iPhone 17 Pro, in an iOS app):** this model runs as an unmerged LoRA adapter on the app's
own Gemma 4 E2B-it 4-bit, so it is one model in memory.
- **Accuracy:** 0 of 42 answers differ from the Mac.
- **Speed and memory:** 0.52 s per request; the adapter adds 25 MB.
- **Base untouched:** with the adapter off, the hidden states are bit-identical to the base.

The Core ML 4-bit build also matches the Mac on the phone (0 of 42; 0.66 s on its GPU). First use case: deciding whether a
new camera description is worth speaking (redundant / vague → skip / re-capture / speak). It took the intended action on
8 hand-written test cases, which is a smoke test, not a measurement.

Everything else (hard, developer-tool and long-document decisions, held-out transfer, general decisions, Japanese) is **not measured yet** on this build. Do not assume E2B v0.6's numbers carry over.

**Release note.** The bar registered for this run before training (jonpol01/d1a#227, Bar B, "good enough for apps") failed on two lines: severity −4.3 (it needed −3 or better) and media agreement 21/28 (it needed 25/28). It is published by the owner's decision of 2026-10-12 for the iOS proof of concept, as a separate repository and not as E2B's model.

**`<bos>`: encode with it.** The Gemma 4 E2B-it tokenizer adds no `<bos>`, but this model was trained with one at the start of every record. d1a from jonpol01/d1a#261 on (merged as `b4b0f989`) puts it back (bos_token_id 2). If you encode requests yourself (another runtime, an app), prepend id 2; without it the answers are not the ones measured here.

## D1A models: what each can do (2026-10-14)

What each D1A model line can do today. "Decisions" means D1A's System One API (`POST /v1/systemone`): yes/no, choice and score questions over a document, answered in one pass with calibrated probabilities. Numbers are from each model's card; "not measured" means the card has no number yet.

| | **D1A-E4B v0.6.1** | **D1A-E2B v0.6.1** | **D1A-E2B-it v0.6** | D1A-12B |
|---|---|---|---|---|
| Base model (trunk) | Gemma 4 E4B (pretrained base) | Gemma 4 E2B (pretrained base) | Gemma 4 E2B-it (instruction-tuned) | Gemma 4 12B-it QAT; **not trained yet** ([jonpol01/d1a#270](https://github.com/jonpol01/d1a/issues/270), shelved) |
| Decisions (`/v1/systemone`) | yes | yes | yes (experimental) | — |
| PR labeling, 487 PRs newer than all training: severity / change type | **79.1% / 90.6%**; P0/P1 found 13 of 30 | 77.2% / 87.5%; P0/P1 0 of 30 | 72.9% / 87.3%; P0/P1 3 of 30 ("not a PR labeller") | — |
| Skill suites (hard, developer tools, long documents, transfer, Japanese JGLUE) | measured (card) | measured (card) | measured 2026-10-15 against E4B v0.6.1 (not published) | — |
| Photos, voice and video (`/v1/systemone/media`, MLX builds with `media/`) | yes, zero-shot | yes; 25 of 28 demo answers match E4B | yes; 21 of 28 match E4B | — |
| Routing temperature (`"use_case": "routing"`) | yes, T 1.18 (from v0.6.1) | yes, T 0.95 (from v0.6.1) | no | — |
| Self-learning (outcome log, recalibration, learned flags) | yes, when served by `d1a.serving.serve` | yes | yes | — |
| **Chat / text generation** | **no**: base trunk; the MLX build has the adapter merged | **no** (same) | **yes**: build it with `scripts/export_mlx.py --unmerged` and serve it; with the adapter off it is Gemma 4 E2B-it ([Enable chat](https://huggingface.co/JohnP1/d1a-e2b-it#enable-chat), [Chat with D1A](https://github.com/jonpol01/d1a/blob/main/docs/USAGE.md#chat-with-d1a)). Also runs as an unmerged adapter in an iPhone app | planned: adapter off = Gemma 4 12B-it |
| Embeddings | — | — | from a GGUF embedder served beside it (`D1A_EMBED_GGUF`) | — |
| Builds | PyTorch `JohnP1/d1a-e4b`; MLX 8-bit `JohnP1/d1a-e4b-mlx-q8` (7.6 GB with 1.0 GB of media encoders) and mirror `mlx-community/d1a-e4b-8bit`; Core ML int8 preview `JohnP1/d1a-e4b-coreml` (text only, ≤512 tokens) | PyTorch `JohnP1/d1a-e2b`; MLX 8-bit `JohnP1/d1a-e2b-mlx-q8` (4.8 GB) and mirror `mlx-community/d1a-e2b-8bit`; Core ML int8 `JohnP1/d1a-e2b-coreml` (text only, ≤512 tokens) | PyTorch adapter `JohnP1/d1a-e2b-it`; MLX `JohnP1/d1a-e2b-it-mlx-q8` (4.8 GB) and `-mlx-q4` (3.6 GB), both merged; Core ML `JohnP1/d1a-e2b-it-coreml` and `-coreml-4bit` | 4-bit Q4_0 export ready (`scripts/export_mlx.py --q4-0`, [jonpol01/d1a#272](https://github.com/jonpol01/d1a/pull/272)) |
| Calibration temperature | 1.95 | 1.45 | 1.41 | — |

Older single-purpose builds: `JohnP1/d1a-e4b-pr-labeler-mlx-q8` (v1) and `JohnP1/d1a-e4b-routing` / `-routing-mlx-q8` (v0.1), superseded by D1A-E4B v0.6.1.

Kept current in the D1A repository: [docs/MODELS.md](https://github.com/jonpol01/d1a/blob/main/docs/MODELS.md).

## What's inside

A LoRA adapter (rank 16 on the attention and MLP projections: q, k, v, o, gate, up, down) and a small pointer head (256-dimensional) on [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it) at revision `3e22461f`.

| Stage | Training data | Settings |
|---|---|---|
| 1. General decisions (`recipes/d1a-e2b.yaml`, what made E2B v0.2) | decision-v7 suite: public classification datasets plus generated policy and rule data | 2 epochs, lr 1e-4, batch 4 × accum 2, from the -it trunk; not published |
| **2. Every later D1A skill, in one run (E2B v0.6's mix)** | PR labels: 4,000 English (severity-stratified), 753 Japanese, 1,500 with a blast-radius label, and 479 hermes-agent PRs created 2026-10-02 to 10-05 shown twice (once in each half); hard-v1 4,000; devtools-v1 4,000; documents-v1 2,500; JGLUE 2,000; routing: agent factory 1,216, generic 1,000; decision-v7 replay 1,500: 23,427 records | from stage 1, 1 epoch, lr 5e-5, states up to 5,120 tokens, 2,929 steps (2 h 45 m) |

Both stages ran in one Hugging Face Jobs job on one NVIDIA L4, about $3.51, finished 2026-10-12 18:33 JST. The mix is E2B v0.6's, byte for byte (sha256 `efff4b16`), screened before training against every evaluation partition, the 487 newer PRs below included. `provenance.json` has the details.

Calibration: a single temperature, **T = 1.41** (90% interval [1.35, 1.48]), fitted on pooled rows of the decision-v7 calibration split and the PR-labelling development set (3,178 questions). Calibration error is 0.053 before and 0.020 after, out of fold; on the 487 newer PRs below (974 questions) it is 0.021. It is stored in `d1a_config.json` and `head.pt`, and there are no per-use-case temperatures.

## Results

### Pull requests newer than all training

487 hermes-agent pull requests created after every PR in the training mix, held back before training. All models are MLX 8-bit at their served temperatures (E2B-it 1.41, E2B v0.6 1.45), with the serving context. Differences are paired over PRs, with 95% bootstrap intervals. E4B v0.6 is shown for reference:

| Question | Majority answer | **E2B-it v0.6** | E2B v0.6 | E4B v0.6 | E2B-it − E2B v0.6 |
|---|---|---|---|---|---|
| Severity (P0–P4) | 67.8% | **72.9%** | 77.2% | 79.1% | −4.3 [−7.8, −1.0] |
| Change type | 51.5% | **87.3%** | 87.5% | 90.6% | −0.2 [−3.1, +2.7] |
| P0 recall (7 PRs) | | **2/7** | 0/7 | 5/7 | |
| P1 recall (23 PRs) | | **1/23** | 0/23 | 8/23 | |
| Security type recall (14 PRs) | | **11/14** | 12/14 | 9/14 | |

It finds 3 of the 30 P0/P1 PRs where E2B v0.6 found none, but it loses more on P3 (273/330 against 295/330) and P2 (74/121 against 77/121) than it gains.

### Every other suite

**Not measured yet.** D1A's card-suite gate (transfer-v4, hard-v1, devtools-v1, documents-v1, decision-v7, JGLUE) has not run on this build. This card will not quote numbers for them until it has.

## Photos, voice and video

With the `media` extra, `d1a.serving.serve` answers questions about a photo, a voice clip or a video through Gemma 4 E2B-it's own vision and audio encoders, unchanged by the adapter (`POST /v1/systemone/media`). No media in training (zero-shot), measured on only the playground's 14 photo, voice and video demo requests. It answers all 14:
- **21 of 28** answers match E4B v0.6 (E2B v0.6: 25 of 28). It misses damage E4B sees (the crushed-truck photo and video, the wet-parcel video), calls two voice clips urgent (the English directions and the Japanese not-at-home message), and places the damaged-door and intact-mailbox photos "on a sidewalk".
- 19 of 28 match E2B v0.6. Unlike E2B v0.6, it calls the intact locker and mailbox photos intact.

## The questions it was trained for

PR labelling works best with these questions, worded exactly so, over a document of the form `title / author / stats / body / files` (one `- status path +added/-deleted` line per file):

- `type` (choice): "Primary change type from files and body, not the title prefix." Options: type/bug, type/docs, type/feature, type/perf, type/refactor, type/security, type/test.
- `blast` (choice): "How far a mistake in this PR spreads in production." Options: review:blast-contained (one module), -moderate (one subsystem), -broad (shared helper or config), -massive (auth, permissions, or all paths).
- `sev` (choice): "How serious the problem this PR addresses is — not the risk of merging the diff as-is." Options P0 to P4.

The [PR-labeler recipe](https://github.com/jonpol01/d1a/tree/main/recipes/pr-labeler) has the exact wording, the document builder and the training and scoring scripts.

## Run it

Needs d1a with jonpol01/d1a#261 (the `<bos>` above):

```bash
pip install "d1a[serve] @ git+https://github.com/jonpol01/d1a"
python -m d1a.serving.serve --run JohnP1/d1a-e2b-it@v0.6 --port 8009
```

or in-process: `from d1a import D1A; D1A.load("JohnP1/d1a-e2b-it@v0.6").decide(state, questions)`. Builds of the same model: MLX 8-bit [JohnP1/d1a-e2b-it-mlx-q8](https://huggingface.co/JohnP1/d1a-e2b-it-mlx-q8) (with the photo, voice and video encoders), MLX 4-bit [JohnP1/d1a-e2b-it-mlx-q4](https://huggingface.co/JohnP1/d1a-e2b-it-mlx-q4), Core ML [JohnP1/d1a-e2b-it-coreml](https://huggingface.co/JohnP1/d1a-e2b-it-coreml) (int8) and [JohnP1/d1a-e2b-it-coreml-4bit](https://huggingface.co/JohnP1/d1a-e2b-it-coreml-4bit) (int4). Unless your app already loads Gemma 4 E2B-it, use [JohnP1/d1a-e2b](https://huggingface.co/JohnP1/d1a-e2b) v0.6 instead.

## Enable chat

With its D1A adapter switched off, this model **is** Gemma 4 E2B-it (bit-identical hidden states), so one model in memory
can answer D1A decisions (adapter on) and OpenAI-format chat (adapter off). This needs an **unmerged** build: the published
MLX repos have the adapter merged into their weights and cannot chat. No chat build is published; build one from this
adapter on Apple Silicon (downloads the E2B-it base, ~10 GB), with d1a from jonpol01/d1a#276 on:

```bash
git clone https://github.com/jonpol01/d1a && cd d1a
uv run --extra serve python scripts/export_mlx.py --run JohnP1/d1a-e2b-it@v0.6 --q-bits 8 --q-group-size 64 \
    --unmerged --out runs/exports/d1a-e2b-it-chat
```

Serve it, with embeddings if your app needs them (mem0 does; the GGUF is `nomic-ai/nomic-embed-text-v1.5-GGUF`, file
`nomic-embed-text-v1.5.Q4_K_M.gguf`, the one LM Studio serves):

```bash
uv sync --extra serve --extra embed
D1A_EMBED_GGUF=<path to nomic-embed-text-v1.5.Q4_K_M.gguf> \
  uv run --extra serve python -m d1a.serving.serve --run runs/exports/d1a-e2b-it-chat --port 8009
```

Then point any OpenAI client at `http://localhost:8009/v1`: `POST /v1/chat/completions` (streaming, tools, JSON mode by
prompt), `POST /v1/embeddings` and `GET /v1/models`. Any model id is answered by this one model. Decisions stay on
`/v1/systemone` with the adapter on; a chat and a decision never run at the same time, so a long answer makes decisions
wait. On the 28-request bench the unmerged 8-bit build's decisions match fp32 (0 of 42 flips). Details:
[Chat with D1A](https://github.com/jonpol01/d1a/blob/main/docs/USAGE.md#chat-with-d1a).

## Limits

- **Experimental, and below E2B v0.6 where measured**: severity −4.3 points on newer PRs, media 21/28 against 25/28 agreement with E4B v0.6. Its pre-registered bar failed (above).
- **Not a PR labeller.** E4B v0.6 remains the PR labeler. On the 487 newer PRs this build finds 3 of 30 P0/P1 PRs (E4B v0.6: 13 of 30). Use E4B for severity, or flag a PR when p(P0) + p(P1) is high instead of taking the top answer.
- **Most suites are not measured**: hard, developer-tool and long-document decisions, held-out transfer, general decisions (decision-v7) and Japanese (JGLUE). Treat its answers there as unknown quality.
- **Gate tool calls through D1A's fail-safe thresholds** (`d1a.agents.presets.advise`: deny at p ≥ 0.5, allow only at p ≥ 0.8, otherwise ask), never through the raw top answer. E2B v0.6 gave a near-tie on the playground's $4,999 payment example; this build's demo answers have not been read yet.
- **`<bos>`**: requests encoded without the leading id 2 are out of distribution (see above).
- **Media**: zero-shot, measured on 14 demo requests only. It misses damage on 3 of them (above).
- One temperature for every question, and no use-case temperatures.
- The recent PR labels come from one large open-source project and a few days of it. If its conventions move again, so will these numbers.
- Questions are in English. The documents can be English or Japanese (trained on both; Japanese not measured on this build).

## Versions

| Tag | What it is |
|---|---|
| **`v0.6`** | **E2B v0.6's recipe on the Gemma 4 E2B-it trunk; experimental, Bar B of #227 failed (severity −4.3, media 21/28)** |

## License and data

Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). Code: [github.com/jonpol01/d1a](https://github.com/jonpol01/d1a), built on Kev by Jared Palmer (Apache-2.0). Japanese decision data derived from JGLUE by Yahoo Japan Corporation and Waseda University (CC BY-SA 4.0). Pull-request data from NousResearch/hermes-agent (MIT); the labelled PR dataset itself, including the recent PRs, is private.