| --- |
| license: apache-2.0 |
| base_model: vllm-sr/d3-mini |
| library_name: coreml |
| pipeline_tag: zero-shot-classification |
| tags: |
| - coreml |
| - apple-silicon |
| - decision-model |
| - vision |
| - video |
| --- |
| |
| # d3-mini — Core ML |
|
|
| [vllm-sr/d3-mini](https://huggingface.co/vllm-sr/d3-mini) (4.5B, part of [Decision 3.0](https://huggingface.co/collections/vllm-sr/decision-30) by the vLLM Semantic Router team) converted to Core ML for Apple silicon: text, image and video requests on device. Same contract as upstream: one forward pass per question, the 255-way FP32 readout at the last prompt token, a probability for every option, no text generated. |
|
|
| ## Files |
|
|
| | File | What | |
| |---|---| |
| | `d3_E256` / `d3_E640` / `d3_E1536` / `d3_E4096.mlpackage` | text graph (fp16 compute, int8 weights) per prompt-length bucket (prompts up to 4,096 tokens); input = token embeddings with image/video tokens spliced in | |
| | `vision_P320` / `vision_P1536` / `vision_P3072.mlpackage` | vision tower (fp32) per patch bucket; video frames go through it one frame pair at a time (block mask) | |
| | `embeddings.f16`, `pos_embed.f32`, `readout.f32`, `tokenizer.json`, `config.json`, `coreml_config.json` | host-side tables and config | |
| | `python/` | Python host (coremltools): `d3_coreml_text.py`, `d3_coreml_image.py`, `d3_coreml_video.py` | |
|
|
| Swift: `D3VisionManager` in [FluidUse](https://github.com/FluidInference/FluidUse) loads this folder. |
|
|
| ## Checked against upstream |
|
|
| Same weights as upstream v3.1.0. Answers that differ from the upstream PyTorch runtime (bf16, MPS), same requests: |
|
|
| | | flips | |
| |---|---| |
| | Text (typed-decisions TEST, 20 requests) | 1 / 100 | |
| | Images (receipt checks) | 0 / 48 | |
| | Video (4-second street clips, 2 fps) | 0 / 28 | |
|
|
| Speed on an M5 Pro (24 GB), `CPU_AND_GPU`: ~160 ms per text question, 1.1 s per 4-second video look. Real-website browser pilot (books.toscrape.com): 17/17 goals, ~2.2 s per click. |
|
|
| ## Python |
|
|
| ```python |
| from d3_coreml_text import D3CoreMLText # d3_coreml_image.D3CoreMLImage, d3_coreml_video.D3CoreMLVideo |
| m = D3CoreMLText("<vllm-sr/d3-mini snapshot>", "<this repo>") |
| m.system_one(state="...", questions={"route": {"type": "choice", "criteria": {...}}}) |
| ``` |
| The Python host builds prompts with upstream's own runtime (`d3_runtime.py` in the upstream snapshot, run without weights), so prompts match upstream exactly. |
|
|
| License: Apache-2.0, as upstream. |
|
|
