File size: 9,968 Bytes

2859d40

---
license: apache-2.0
base_model: JohnP1/d1a-e2b-it
base_model_relation: quantized
library_name: mlx
pipeline_tag: text-classification
language: [en, ja]
tags: [d1a, decision-model, pull-requests, mlx, apple-silicon, quantized, on-device, experimental]
---

<p align="center"><img src="https://huggingface.co/JohnP1/d1a-e2b-it-mlx-q8/resolve/main/d1a-avatar.png" width="200" alt="D1A avatar: an anime girl with silver-white and dark purple hair, round glasses and headphones on a round #D1A badge"></p>

# D1A-E2B-it v0.6 · MLX 8-bit (experimental)

This repository mirrors [JohnP1/d1a-e2b-it-mlx-q8](https://huggingface.co/JohnP1/d1a-e2b-it-mlx-q8) `v0.6` byte for byte (the same files, checked by LFS sha256), under tag `v0.6`.

**A small decision model for Apple Silicon: give it a document and a few questions, and it picks the answers.** Each question
is yes/no, multiple choice or a score; D1A returns a calibrated probability for every option in one forward pass, with no
generated text, so every answer is a valid option.

**This is an experimental build for on-device apps that already load Gemma 4 E2B-it.** It is D1A-E2B v0.6's recipe on the
[google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it) trunk. Where it has been measured it is below
[JohnP1/d1a-e2b-mlx-q8](https://huggingface.co/JohnP1/d1a-e2b-mlx-q8) v0.6 (severity −4.3 points, media 21/28 against 25/28
agreement with E4B v0.6; numbers below), and E4B v0.6 ([JohnP1/d1a-e4b-mlx-q8](https://huggingface.co/JohnP1/d1a-e4b-mlx-q8))
remains the PR labeler. If your app does not already load Gemma 4 E2B-it, use E2B v0.6.

```python
from d1a import D1A   # pip install "d1a[serve,mlx,media] @ git+https://github.com/jonpol01/d1a"  (needs jonpol01/d1a#261)
m = D1A.load("mlx-community/d1a-e2b-it-8bit@v0.6")
m.decide("The customer was charged twice for the same order and wants a refund today.",
         {"team": {"type": "choice", "instructions": "Which team handles this?",
                   "criteria": {"billing": None, "shipping": None, "support": None}},
          "angry": {"type": "noul", "instructions": "Is the customer angry?"}})
# -> {"team": {"choice": ..., "probabilities": {"billing": ..., "shipping": ..., "support": ...}}, "angry": {"noul": ...}}
```

**Not a chat model.** `mlx_lm.generate` and LM Studio will load the Gemma 4 backbone and write ordinary text, but D1A's
answers come from its decision head, which only the `d1a` package uses (see Run it).

**`<bos>`: encode with it.** The Gemma 4 E2B-it tokenizer adds no `<bos>`, but this model was trained with one at the start
of every record. d1a from jonpol01/d1a#261 on (merged as `b4b0f989`) puts it back (bos_token_id 2; `d1a_config.json` records `leading_ids: [2]`). If
you encode requests yourself, prepend id 2.

[JohnP1/d1a-e2b-it](https://huggingface.co/JohnP1/d1a-e2b-it) `v0.6` for Apple Silicon:
- the adapter merged into Gemma 4 E2B-it; linear layers and token embeddings 8-bit (group 64), per-layer embeddings 4-bit;
- pointer head fp32 (`head.safetensors`);
- calibration temperature 1.41 in `d1a_config.json`;
- 3.5 GB on disk, plus 0.9 GB of photo, voice and video encoders in `media/` (unchanged from the -it base).

Mac check on an Apple M1 Max, 28 real requests of up to 512 tokens (42 questions), against the fp32 PyTorch rebuild of this
export: p50 199 ms, p95 294 ms per request (28 requests), 9 s to load; 0 of 42 answers change (0 of 40 clear-cut ones, where the reference's top answer leads by more than 0.05); largest probability change 0.015.

The scores below were measured on this build.

**Release note.** The bar registered for this run before training (jonpol01/d1a#227, Bar B, "good enough for apps") failed on
two lines: severity −4.3 (it needed −3 or better) and media agreement 21/28 (it needed 25/28). It is published by the
owner's decision of 2026-10-12 for an iOS proof of concept, as a separate repository and not as E2B's model.

## What's inside

A LoRA adapter (rank 16 on q, k, v, o, gate, up, down) and a small pointer head (256-dimensional) on
[google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it) at revision `3e22461f`, merged and quantized here.
Training: stage 1 is `recipes/d1a-e2b.yaml` (decision-v7, 2 epochs) from the -it trunk; stage 2 is E2B v0.6's run on its
mix of 23,427 records (PR labels, hard, developer-tool and long-document decisions, JGLUE, routing, decision-v7 replay),
1 epoch from stage 1. One NVIDIA L4 on Hugging Face Jobs, about $3.51. Details: the PyTorch repo's card and `provenance.json`.

Calibration: a single temperature, **T = 1.41** (90% interval [1.35, 1.48]), fitted on pooled rows of the decision-v7
calibration split and the PR-labelling development set. Calibration error is 0.053 before and 0.020 after, out of fold.

## Results

### Pull requests newer than all training

487 hermes-agent pull requests created after every PR in the training mix, held back before training. All models are MLX
8-bit at their served temperatures (E2B-it 1.41, E2B v0.6 1.45), with the serving context. Differences are paired over PRs,
with 95% bootstrap intervals. E4B v0.6 is shown for reference:

| Question | Majority answer | **E2B-it v0.6** | E2B v0.6 | E4B v0.6 | E2B-it − E2B v0.6 |
|---|---|---|---|---|---|
| Severity (P0–P4) | 67.8% | **72.9%** | 77.2% | 79.1% | −4.3 [−7.8, −1.0] |
| Change type | 51.5% | **87.3%** | 87.5% | 90.6% | −0.2 [−3.1, +2.7] |
| P0 recall (7 PRs) | | **2/7** | 0/7 | 5/7 | |
| P1 recall (23 PRs) | | **1/23** | 0/23 | 8/23 | |
| Security type recall (14 PRs) | | **11/14** | 12/14 | 9/14 | |

Calibration error on these 974 questions: 0.021.

### Every other suite

**Not measured yet.** D1A's card-suite gate (transfer-v4, hard-v1, devtools-v1, documents-v1, decision-v7, JGLUE) has not
run on this build. Do not assume E2B v0.6's numbers carry over.

## Photos, voice and video

With the `media` extra, `d1a.serving.serve` answers questions about a photo, a voice clip or a video with this same model
through Gemma 4 E2B-it's own vision and audio encoders in `media/`, unchanged by the adapter (`POST /v1/systemone/media`).
No media in training (zero-shot), measured on only the playground's 14 photo, voice and video demo requests. It answers all 14:
- **21 of 28** answers match E4B v0.6 (E2B v0.6: 25 of 28). It misses damage E4B sees (the crushed-truck photo and video,
  the wet-parcel video), calls two voice clips urgent (the English directions and the Japanese not-at-home message), and
  places the damaged-door and intact-mailbox photos "on a sidewalk".
- 19 of 28 match E2B v0.6. Unlike E2B v0.6, it calls the intact locker and mailbox photos intact.

## The questions it was trained for

PR labelling works best with these questions, worded exactly so, over a document of the form `title / author / stats / body / files` (one `- status path +added/-deleted` line per file):

- `type` (choice): "Primary change type from files and body, not the title prefix." Options: type/bug, type/docs, type/feature, type/perf, type/refactor, type/security, type/test.
- `blast` (choice): "How far a mistake in this PR spreads in production." Options: review:blast-contained (one module), -moderate (one subsystem), -broad (shared helper or config), -massive (auth, permissions, or all paths).
- `sev` (choice): "How serious the problem this PR addresses is — not the risk of merging the diff as-is." Options P0 to P4.

The [PR-labeler recipe](https://github.com/jonpol01/d1a/tree/main/recipes/pr-labeler) has the exact wording, the document builder and the training and scoring scripts.

## Run it

Needs d1a with jonpol01/d1a#261 (the `<bos>` above):

```bash
pip install "d1a[serve,mlx,media] @ git+https://github.com/jonpol01/d1a"
python -m d1a.serving.serve --run mlx-community/d1a-e2b-it-8bit@v0.6 --port 8009
```

or in-process: `from d1a import D1A; D1A.load("mlx-community/d1a-e2b-it-8bit@v0.6").decide(state, questions)`. The PyTorch
checkpoint is [JohnP1/d1a-e2b-it](https://huggingface.co/JohnP1/d1a-e2b-it) `v0.6`; a 4-bit build is
[JohnP1/d1a-e2b-it-mlx-q4](https://huggingface.co/JohnP1/d1a-e2b-it-mlx-q4).

## Limits

- **Experimental, and below E2B v0.6 where measured**: severity −4.3 points on newer PRs, media 21/28 against 25/28 agreement with E4B v0.6. Its pre-registered bar failed (above).
- **Not a PR labeller.** E4B v0.6 remains the PR labeler. On the 487 newer PRs this build finds 3 of 30 P0/P1 PRs (E4B v0.6: 13 of 30).
- **Most suites are not measured**: hard, developer-tool and long-document decisions, held-out transfer, general decisions (decision-v7) and Japanese (JGLUE).
- **Gate tool calls through D1A's fail-safe thresholds** (`d1a.agents.presets.advise`: deny at p ≥ 0.5, allow only at p ≥ 0.8, otherwise ask), never through the raw top answer. This build's demo answers have not been read yet.
- **`<bos>`**: requests encoded without the leading id 2 are out of distribution (see above).
- **Media**: zero-shot, measured on 14 demo requests only. It misses damage on 3 of them (above).
- One temperature for every question, and no use-case temperatures.
- The recent PR labels come from one large open-source project and a few days of it. If its conventions move again, so will these numbers.
- Questions are in English. The documents can be English or Japanese (trained on both; Japanese not measured on this build).

## Versions

| Tag | What it is |
|---|---|
| **`v0.6`** | **E2B v0.6's recipe on the Gemma 4 E2B-it trunk, MLX 8-bit with `media/`; experimental, Bar B of #227 failed** |

## License and data

Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). Code: [github.com/jonpol01/d1a](https://github.com/jonpol01/d1a), built on Kev by Jared Palmer (Apache-2.0). Japanese decision data derived from JGLUE by Yahoo Japan Corporation and Waseda University (CC BY-SA 4.0). Pull-request data from NousResearch/hermes-agent (MIT); the labelled PR dataset itself, including the recent PRs, is private.