| --- |
| license: apache-2.0 |
| base_model: google/gemma-4-E4B-it |
| tags: |
| - coreml |
| - apple-neural-engine |
| - decision-model |
| - d1a |
| library_name: coreml |
| pipeline_tag: text-classification |
| --- |
| |
| <p align="center"><img src="https://huggingface.co/JohnP1/d1a-e4b-it-coreml-4bit/resolve/main/d1a-avatar.png" width="200" alt="D1A avatar: an anime girl with silver-white and dark purple hair, round glasses and headphones on a round #D1A badge"></p> |
|
|
| # D1A-E4B-it v0.6 · Core ML int4 (text, ≤ 512 tokens) |
|
|
| **A small decision model for Apple devices: give it a document and a few questions, and it picks the answers.** Each question |
| is yes/no, multiple choice or a score; D1A returns a calibrated probability for every option in one forward pass, with no |
| generated text, so every answer is a valid option. |
|
|
| This is the Core ML int4 conversion of [JohnP1/d1a-e4b-it](https://huggingface.co/JohnP1/d1a-e4b-it) v0.6. It is **text only, |
| for documents up to 512 tokens**, with int4 weights (per block of 32): 2.1 GB of chunks plus 2.3 GB of embedding tables. The |
| conversion is the one used for the E2B and E2B-it Core ML builds, unchanged. |
|
|
| ## Status: read before use |
|
|
| - **Answers move at 4 bits.** Mac check on an Apple M1 Max: 28 real requests of up to 512 tokens (42 questions), against |
| the fp32 PyTorch rebuild of the 8-bit export it was converted from. |
| - **3 of 42 answers change (7.1%), all three on clear-cut questions** (where the reference's top answer leads by more than |
| 0.05); the largest probability change is 0.455. |
| - For comparison, D1A-E2B-it's int4 Core ML build changes 0 of 42. |
| - **Speed** on the Mac's GPU (CPU_AND_GPU): p50 6.6 s, p95 9.2 s per request; 469 s to load (first, compiled). |
| - **Not verified on iPhone.** No int8 Core ML build is planned for E4B-it; for answers that must not move, use the MLX 8-bit build below. |
| - **Not included:** photo, voice and video (their encoders are separate models, not converted), and inputs longer than 512 |
| tokens. |
| |
| For decisions that matter, prefer the MLX 8-bit build [JohnP1/d1a-e4b-it-mlx-q8](https://huggingface.co/JohnP1/d1a-e4b-it-mlx-q8) |
| (1 of 202 demo answers and 3 of 276 labeler answers differ from the served model). |
|
|
| ## `<bos>`: encode with it |
|
|
| The Gemma 4 E4B-it tokenizer adds no `<bos>`, but this model was trained with one at the start of every record. d1a from |
| jonpol01/d1a#261 on puts it back (bos_token_id 2; `d1a_config.json` records `leading_ids: [2]`). An app that encodes requests |
| itself must prepend id 2 before right-padding to 512. |
|
|
| ## Run it |
|
|
| ```bash |
| pip install coremltools numpy torch transformers safetensors "d1a @ git+https://github.com/jonpol01/d1a" |
| hf download JohnP1/d1a-e4b-it-coreml-4bit --revision v0.6 --local-dir d1a-e4b-it-coreml-4bit |
| python d1a-e4b-it-coreml-4bit/reference/run.py d1a-e4b-it-coreml-4bit request.json |
| ``` |
|
|
| `request.json` is a System One request, for example: |
|
|
| ```json |
| {"state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.", |
| "questions": {"team": {"type": "choice", "instructions": "Which team should handle this?", "criteria": {"returns": null, "shipping": null, "billing": null}}, |
| "urgent": {"type": "noul", "instructions": "Does this need a reply today?"}}} |
| ``` |
|
|
| The first run compiles the chunks into `coreml/.compiled/` (several minutes); later runs load them. |
|
|
| ## Files |
|
|
| | Path | What | |
| |---|---| |
| | `coreml/L512_<a>-<b>.mlpackage` | 7 chunks of the 42-layer text model, int4 weights (per block of 32) | |
| | `embeddings.safetensors` | the token and per-layer embedding tables (MLX affine-quantized, from JohnP1/d1a-e4b-it-mlx-q8@v0.6), read row by row on the host | |
| | `head.safetensors`, `d1a_config.json` | the pointer head, the temperature (1.55), the `<bos>`, pad and delimiter ids | |
| | `config.json`, `tokenizer.json`, `tokenizer_config.json` | the Gemma 4 E4B-it model config and tokenizer | |
| | `io.json` | each chunk's input names | |
| | `reference/` | Python reference: `run.py` (one request), `workers.py` (one process per chunk), `d1a_coreml.py` (host-side embeddings and masks), `pipeline.py` (all chunks in one process) | |
|
|
| macOS 15 / iOS 18 or later. |
|
|
| ## License |
|
|
| Apache-2.0. Base model: Gemma 4 by Google (Apache-2.0). D1A is built on Kev by Jared Palmer (Apache-2.0). |
|
|
