Text Classification
MLX
Safetensors
English
Japanese
gemma4
d1a
decision-model
pull-requests
apple-silicon
quantized
4-bit precision
on-device
Instructions to use JohnP1/d1a-e4b-it-mlx-q4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use JohnP1/d1a-e4b-it-mlx-q4 with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download JohnP1/d1a-e4b-it-mlx-q4 --local-dir d1a-e4b-it-mlx-q4
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
D1A-E4B-it v0.6: README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,73 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: google/gemma-4-E4B-it
|
| 4 |
+
library_name: mlx
|
| 5 |
+
pipeline_tag: text-classification
|
| 6 |
+
language: [en, ja]
|
| 7 |
+
tags: [d1a, decision-model, pull-requests, mlx, apple-silicon, quantized, 4-bit, on-device]
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
+
<p align="center"><img src="https://huggingface.co/JohnP1/d1a-e4b-it-mlx-q4/resolve/main/d1a-avatar.png" width="200" alt="D1A avatar: an anime girl with silver-white and dark purple hair, round glasses and headphones on a round #D1A badge"></p>
|
| 11 |
+
|
| 12 |
+
# D1A-E4B-it v0.6 · MLX 4-bit
|
| 13 |
+
|
| 14 |
+
**A small decision model for Apple Silicon: give it a document and a few questions, and it picks the answers.** Each question
|
| 15 |
+
is yes/no, multiple choice or a score; D1A returns a calibrated probability for every option in one forward pass, with no
|
| 16 |
+
generated text, so every answer is a valid option.
|
| 17 |
+
|
| 18 |
+
This is the 4-bit MLX build of [JohnP1/d1a-e4b-it](https://huggingface.co/JohnP1/d1a-e4b-it) v0.6, with the adapter merged
|
| 19 |
+
into its weights. Scores, training and limits are on that card; how far this build moves from the served chat build is below.
|
| 20 |
+
|
| 21 |
+
```python
|
| 22 |
+
from d1a import D1A # pip install "d1a[serve,mlx,media] @ git+https://github.com/jonpol01/d1a" (needs jonpol01/d1a#261)
|
| 23 |
+
m = D1A.load("JohnP1/d1a-e4b-it-mlx-q4@v0.6")
|
| 24 |
+
m.decide("The customer was charged twice for the same order and wants a refund today.",
|
| 25 |
+
{"team": {"type": "choice", "instructions": "Which team handles this?",
|
| 26 |
+
"criteria": {"billing": None, "shipping": None, "support": None}},
|
| 27 |
+
"angry": {"type": "noul", "instructions": "Is the customer angry?"}})
|
| 28 |
+
```
|
| 29 |
+
|
| 30 |
+
**Not a chat model.** This build has the adapter merged into its weights, so it answers D1A decisions only; text generated
|
| 31 |
+
from it would come from a changed backbone, not Gemma 4 E4B-it. To chat with the same model that decides, build an unmerged
|
| 32 |
+
folder from the adapter: see [Enable chat](https://huggingface.co/JohnP1/d1a-e4b-it#enable-chat).
|
| 33 |
+
|
| 34 |
+
**`<bos>`: encode with it.** The Gemma 4 E4B-it tokenizer adds no `<bos>`, but this model was trained with one at the start
|
| 35 |
+
of every record (`d1a_config.json` records `leading_ids: [2]`). If you encode requests yourself, prepend id 2.
|
| 36 |
+
|
| 37 |
+
## What's inside
|
| 38 |
+
|
| 39 |
+
- the LoRA adapter merged into Gemma 4 E4B-it (revision `ee0ef602`); linear layers and token embeddings 4-bit (group 64), per-layer embeddings 4-bit;
|
| 40 |
+
- pointer head fp32 (`head.safetensors`);
|
| 41 |
+
- calibration temperature 1.55 in `d1a_config.json`, the run's, not refitted for this build; no use-case temperatures;
|
| 42 |
+
- 3.9 GB on disk for the text model, plus 0.9 GB of photo, voice and video encoders in `media/` (unchanged from the -it base).
|
| 43 |
+
|
| 44 |
+
## Against the served build
|
| 45 |
+
|
| 46 |
+
Same code, this build against the unmerged 8-bit chat build the Mac mini serves (`quality_gate.py`, Apple M1 Max): 145 demo requests (202 questions) and 92 PR-labeler states (276 questions).
|
| 47 |
+
- **9 of 202** demo answers and **33 of 276** labeler answers (12%) change; largest probability change 0.35;
|
| 48 |
+
- model time 0.83× the chat build's.
|
| 49 |
+
|
| 50 |
+
**This 4-bit build has not been scored on the evaluation suites.** It moves noticeably more than the 8-bit build; prefer [-mlx-q8](https://huggingface.co/JohnP1/d1a-e4b-it-mlx-q8) unless memory decides.
|
| 51 |
+
|
| 52 |
+
## Run it
|
| 53 |
+
|
| 54 |
+
```bash
|
| 55 |
+
pip install "d1a[serve,mlx,media] @ git+https://github.com/jonpol01/d1a"
|
| 56 |
+
python -m d1a.serving.serve --run JohnP1/d1a-e4b-it-mlx-q4@v0.6 --port 8009
|
| 57 |
+
```
|
| 58 |
+
|
| 59 |
+
## Limits
|
| 60 |
+
|
| 61 |
+
- Everything on the [D1A-E4B-it card](https://huggingface.co/JohnP1/d1a-e4b-it#limits) applies.
|
| 62 |
+
- **Not scored on the suites at 4 bits**: 12% of PR-labeler answers differ from the served build; expect the card's numbers to move.
|
| 63 |
+
- **`<bos>`**: requests encoded without the leading id 2 are out of distribution.
|
| 64 |
+
|
| 65 |
+
## Versions
|
| 66 |
+
|
| 67 |
+
| Tag | What it is |
|
| 68 |
+
|---|---|
|
| 69 |
+
| **`v0.6`** | **MLX 4-bit build of D1A-E4B-it v0.6 (adapter merged)** |
|
| 70 |
+
|
| 71 |
+
## License and data
|
| 72 |
+
|
| 73 |
+
Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). Code: [github.com/jonpol01/d1a](https://github.com/jonpol01/d1a), built on Kev by Jared Palmer (Apache-2.0). Japanese decision data derived from JGLUE by Yahoo Japan Corporation and Waseda University (CC BY-SA 4.0). Pull-request data from NousResearch/hermes-agent (MIT); the labelled PR dataset itself, including the recent PRs, is private.
|