File size: 12,924 Bytes

---
license: apache-2.0
base_model: google/gemma-4-E4B-it
base_model_relation: adapter
library_name: peft
pipeline_tag: text-classification
language: [en, ja]
datasets: [JohnP1/d1a-ja-jglue, JohnP1/d1a-routing]
tags: [d1a, decision-model, calibration, lora, gemma4, typesafe, system-one, pull-requests, chat, on-device]
---

<p align="center"><img src="https://huggingface.co/JohnP1/d1a-e4b-it/resolve/main/d1a-avatar.png" width="200" alt="D1A avatar: an anime girl with silver-white and dark purple hair, round glasses and headphones on a round #D1A badge"></p>

# D1A-E4B-it v0.6

D1A is a small open decision model. One document and a set of typed questions go in (yes/no, choice, score), and a calibrated probability for every option comes out, in one forward pass, with no generated text. It speaks the TypeSafe System One API (`POST /v1/systemone`), so the TypeSafe SDK and existing clients work against it.

**One model for decisions and chat.** This is D1A-E2B-it v0.6's recipe on the larger instruction-tuned trunk, [google/gemma-4-E4B-it](https://huggingface.co/google/gemma-4-E4B-it). With the D1A adapter on it answers decisions; with the adapter off it is Gemma 4 E4B-it exactly, so one model in memory also serves OpenAI-format chat ([Enable chat](#enable-chat)).

Measured against D1A-E2B-it v0.6 on every D1A suite (2026-10-17, both as served):
- **accuracy up on 21 of 22 suites**, pooled **+2.5 points [+1.7, +3.4]**: hard decisions +6.6, developer tools +3.2, long documents +2.3, JGLUE +2.2, PR labels (newer PRs) +1.8, and +5.1 on PR severity with the served decode offsets;
- the one significant loss is the unknowable-facts monitor (−2.9);
- **about 2.9× slower per decision** than E2B-it (a model about twice the size; p50 0.25 s on an Apple M4);
- calibration is mixed (ECE better on 9 suites, worse on 12, all by ≤ 0.03).

**`<bos>`: encode with it.** The Gemma 4 -it tokenizers add no `<bos>`, but this model was trained with one at the start of every record. d1a from jonpol01/d1a#261 on puts it back (bos_token_id 2). If you encode requests yourself (another runtime, an app), prepend id 2; without it the answers are not the ones measured here.

## D1A models: what each can do (2026-10-17)

| | **D1A-E4B-it v0.6** | D1A-E4B v0.6.1 | D1A-E2B v0.6.1 | D1A-E2B-it v0.6 |
|---|---|---|---|---|
| Base model (trunk) | Gemma 4 E4B-it (instruction-tuned) | Gemma 4 E4B (pretrained base) | Gemma 4 E2B (pretrained base) | Gemma 4 E2B-it (instruction-tuned) |
| Decisions (`/v1/systemone`) | yes | yes | yes | yes (experimental) |
| Photos, voice and video (MLX builds with `media/`) | yes, zero-shot | yes, zero-shot | yes | yes |
| **Chat / text generation** | **yes**: unmerged build, adapter off = Gemma 4 E4B-it ([Enable chat](#enable-chat)) | no | no | yes (same way) |
| Embeddings | from a GGUF embedder served beside it (`D1A_EMBED_GGUF`) | — | — | same |
| Routing temperature | no (measured, not shipped) | yes, T 1.18 | yes, T 0.95 | no |
| Calibration temperature | 1.55 | 1.95 | 1.45 | 1.41 |
| Builds | PyTorch adapter `JohnP1/d1a-e4b-it`; MLX `-mlx-q8` (7.0 GB with media) and `-mlx-q4` (4.9 GB with media), both merged | PyTorch, MLX 8-bit, Core ML int8 preview | PyTorch, MLX 8-bit, Core ML int8 | PyTorch adapter, MLX q8/q4, Core ML int8/int4 |

Kept current in the D1A repository: [docs/MODELS.md](https://github.com/jonpol01/d1a/blob/main/docs/MODELS.md).

## What's inside

A LoRA adapter (rank 16 on the attention and MLP projections: q, k, v, o, gate, up, down) and a small pointer head (256-dimensional) on [google/gemma-4-E4B-it](https://huggingface.co/google/gemma-4-E4B-it) at revision `ee0ef602`.

| Stage | Training data | Settings |
|---|---|---|
| 1. General decisions (`recipes/d1a-e2b.yaml`, what made E2B v0.2) | decision-v7 suite: public classification datasets plus generated policy and rule data | 2 epochs, lr 1e-4, batch 4 × accum 2, from the -it trunk; not published |
| **2. Every later D1A skill, in one run (E2B v0.6's mix)** | PR labels: 4,000 English (severity-stratified), 753 Japanese, 1,500 with a blast-radius label, and 479 hermes-agent PRs created 2026-10-02 to 10-05 shown twice (once in each half); hard-v1 4,000; devtools-v1 4,000; documents-v1 2,500; JGLUE 2,000; routing: agent factory 1,216, generic 1,000; decision-v7 replay 1,500: 23,427 records, none truncated | from stage 1, 1 epoch, lr 5e-5, states up to 5,120 tokens, 2,929 steps |

The mix is E2B v0.6's and E2B-it v0.6's, byte for byte (sha256 `efff4b16`), screened before training against every evaluation partition. Where it ran:
- **Stage 1:** steps 1–2,038 on one NVIDIA L4 (Hugging Face Jobs, bf16 autocast); steps 2,039–3,128 on an Apple M1 Max (MPS, fp32 forward), resumed from the saved step when the cloud job stopped.
- **Stage 2:** on the M1 Max (MPS, fp32 forward, bf16 frozen weights), finished 2026-10-17.
- LoRA and head master weights are fp32 on both machines, so only the forward precision changed between them. `provenance.json` has the details.

Calibration: a single temperature, **T = 1.55** (90% interval [1.48, 1.62]), fitted on pooled rows of the decision-v7 calibration split and the PR-labelling development set (3,178 questions). Calibration error (ECE) is 0.057 before and 0.015 after, out of fold. It is stored in `d1a_config.json`.

**No routing temperature.** A routing-specific temperature (T 1.20) was fitted and not shipped: out of fold it did not separate from the general one, and on held-out transfer it made calibration worse (ECE 0.018 → 0.031). Routing requests use T 1.55, the same outcome as E2B-it.

## Results

Every D1A suite, both models as served (MLX 8-bit, unmerged, each at its own temperature), on an Apple M4. Δ is paired with 95% bootstrap intervals; **bold** where the interval excludes 0.

| Suite | n | E2B-it v0.6 | **E4B-it v0.6** | Δ [95% CI] |
|---|---:|---:|---:|---|
| General decisions (decision-v7) | 1,264 | 84.8 | 85.7 | +0.9 [−0.7, +2.5] |
| Transfer (transfer-v4, sources never trained on) | 656 | 75.0 | 77.4 | +2.4 [−0.9, +5.9] |
| Hard decisions (hard-v1) | 1,083 | 69.2 | 75.7 | **+6.6 [+4.0, +9.2]** |
| Developer tools (devtools-v1) | 1,074 | 71.1 | 74.3 | **+3.2 [+0.9, +5.4]** |
| Long documents (documents-v1) | 920 | 86.3 | 88.6 | **+2.3 [+0.5, +4.2]** |
| Japanese JGLUE, development / test | 1,500 / 1,500 | 78.6 / 76.4 | 80.8 / 78.6 | **+2.2 / +2.2** |
| PR labels, test (newer PRs): severity / type / blast | 953 / 953 / 139 | 70.5 / 87.8 / 52.5 | 72.9 / 88.4 / 58.3 | **+2.4** / +0.5 / +5.8 |
| PR severity with the served decode offsets | 953 | 64.3 | 69.5 | **+5.1 [+3.0, +7.3]** |
| PR labels, Japanese test | 193 | 74.6 | 77.7 | +3.1 [−1.0, +7.2] |
| Routing: agent factory / generic / hand-labelled | 640 / 270 / 45 | 92.3 / 97.0 / 100 | 93.6 / 97.4 / 100 | +1.2 / +0.4 / 0.0 |
| External: TypeSafe / wanli-v2 | 102 / 1,002 | 69.6 / 68.9 | 86.3 / 72.7 | **+16.7 / +3.8** |
| Dates (monitor) | 900 | 87.0 | 90.8 | **+3.8 [+1.7, +5.8]** |
| Unknowable facts (monitor) | 525 | 69.5 | 66.7 | **−2.9 [−5.5, −0.2]** |

The live PR-labeler replay (98–193 questions) is flat on type and down on blast radius (64.3 → 58.2, not significant at this n). Against the older D1A-E4B v0.6.1 the comparison is indirect (both measured against E2B-it): E4B-it is ahead on decision-v7, transfer, hard and developer tools, and 1–2 points behind on long documents, JGLUE and dates.

## Photos, voice and video

With the `media` extra, `d1a.serving.serve` answers questions about a photo, a voice clip or a video through Gemma 4 E4B-it's own vision and audio encoders, unchanged by the adapter (`POST /v1/systemone/media`). No media in training (zero-shot).

Measured on the playground's 14 photo, voice and video demo requests (28 answers): all answered. 7 answers differ from E2B-it v0.6; read by hand, 6 are better (it now calls the crushed-truck photo and video damaged, places the intact mailbox photo "in a mailbox", and no longer calls the English directions and Japanese not-at-home voice clips urgent) and 1 is worse (the intact front-door photo placed "on a sidewalk").

## The questions it was trained for

PR labelling works best with these questions, worded exactly so, over a document of the form `title / author / stats / body / files` (one `- status path +added/-deleted` line per file):

- `type` (choice): "Primary change type from files and body, not the title prefix." Options: type/bug, type/docs, type/feature, type/perf, type/refactor, type/security, type/test.
- `blast` (choice): "How far a mistake in this PR spreads in production." Options: review:blast-contained (one module), -moderate (one subsystem), -broad (shared helper or config), -massive (auth, permissions, or all paths).
- `sev` (choice): "How serious the problem this PR addresses is — not the risk of merging the diff as-is." Options P0 to P4.

The [PR-labeler recipe](https://github.com/jonpol01/d1a/tree/main/recipes/pr-labeler) has the exact wording, the document builder and the training and scoring scripts.

## Run it

Needs d1a with jonpol01/d1a#261 (the `<bos>` above):

```bash
pip install "d1a[serve] @ git+https://github.com/jonpol01/d1a"
python -m d1a.serving.serve --run JohnP1/d1a-e4b-it@v0.6 --port 8009
```

or in-process: `from d1a import D1A; D1A.load("JohnP1/d1a-e4b-it@v0.6").decide(state, questions)`. Builds of the same model: MLX 8-bit [JohnP1/d1a-e4b-it-mlx-q8](https://huggingface.co/JohnP1/d1a-e4b-it-mlx-q8) and MLX 4-bit [JohnP1/d1a-e4b-it-mlx-q4](https://huggingface.co/JohnP1/d1a-e4b-it-mlx-q4), both with the photo, voice and video encoders and both merged (decisions only).

## Enable chat

With its D1A adapter switched off, this model **is** Gemma 4 E4B-it, so one model in memory can answer D1A decisions (adapter on) and OpenAI-format chat (adapter off). This needs an **unmerged** build: the published MLX repos have the adapter merged into their weights and cannot chat. No chat build is published; build one from this adapter on Apple Silicon (downloads the E4B-it base, ~16 GB), with d1a from jonpol01/d1a#276 on:

```bash
git clone https://github.com/jonpol01/d1a && cd d1a
uv run --extra serve python scripts/export_mlx.py --run JohnP1/d1a-e4b-it@v0.6 --q-bits 8 --q-group-size 64 \
    --unmerged --media --out runs/exports/d1a-e4b-it-chat
```

Serve it, with embeddings if your app needs them (mem0 does; the GGUF is `nomic-ai/nomic-embed-text-v1.5-GGUF`, file `nomic-embed-text-v1.5.Q4_K_M.gguf`, the one LM Studio serves):

```bash
uv sync --extra serve --extra embed
D1A_EMBED_GGUF=<path to nomic-embed-text-v1.5.Q4_K_M.gguf> \
  uv run --extra serve python -m d1a.serving.serve --run runs/exports/d1a-e4b-it-chat --port 8009
```

Then point any OpenAI client at `http://localhost:8009/v1`: `POST /v1/chat/completions` (streaming, tools, JSON mode by prompt), `POST /v1/embeddings` and `GET /v1/models`. Any model id is answered by this one model. Decisions stay on `/v1/systemone` with the adapter on. With d1a from jonpol01/d1a#291 on, a decision that arrives during a chat is answered between two of its tokens (0.24 s on this model during a 200-token answer, instead of waiting for the whole answer), and the chat's text is unchanged. Details: [Chat with D1A](https://github.com/jonpol01/d1a/blob/main/docs/USAGE.md#chat-with-d1a).

## Limits

- **Slower than E2B-it**: about 2.9× the model time per decision. Use E2B-it or E2B where latency matters more than accuracy.
- **Calibration is mixed against E2B-it**: ECE worse on 12 suites (largest +0.028 on Japanese PR test), better on 9.
- **Unknowable facts** (questions whose answer is not in the document): −2.9 against E2B-it.
- **Gate tool calls through D1A's fail-safe thresholds** (`d1a.agents.presets.advise`: deny at p ≥ 0.5, allow only at p ≥ 0.8, otherwise ask), never through the raw top answer.
- **`<bos>`**: requests encoded without the leading id 2 are out of distribution (see above).
- **Media**: zero-shot, measured on the playground's demo requests only.
- One temperature for every question, and no use-case temperatures (the routing one was measured and not shipped).
- The recent PR labels come from one large open-source project and a few days of it. If its conventions move again, so will these numbers.
- Questions are in English. The documents can be English or Japanese (trained on both).

## Versions

| Tag | What it is |
|---|---|
| **`v0.6`** | **E2B v0.6's recipe on the Gemma 4 E4B-it trunk; decisions and chat in one model** |

## License and data

Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). Code: [github.com/jonpol01/d1a](https://github.com/jonpol01/d1a), built on Kev by Jared Palmer (Apache-2.0). Japanese decision data derived from JGLUE by Yahoo Japan Corporation and Waseda University (CC BY-SA 4.0). Pull-request data from NousResearch/hermes-agent (MIT); the labelled PR dataset itself, including the recent PRs, is private.