license: apache-2.0
base_model: vllm-sr/d3-edge
library_name: coreml
pipeline_tag: zero-shot-classification
tags:
- coreml
- mlx
- apple-silicon
- decision-model
- vision
- video
d3-edge — Core ML and MLX
vllm-sr/d3-edge (0.6B, the camera-stream model of Decision 3.0 by the vLLM Semantic Router team) for Apple silicon: text, image and video requests on device, as Core ML packages and as MLX weights. Same contract as upstream: one forward pass per question, the 255-way FP32 readout at the last prompt token, a probability for every option, no text generated.
Files
| File | What |
|---|---|
d3_E256 / d3_E640 / d3_E1536 / d3_E4096.mlpackage |
Core ML text graph (fp16 compute, int8 weights) per prompt-length bucket (prompts up to 4,096 tokens); input = token embeddings with image/video tokens spliced in |
vision_P320 / vision_P1536 / vision_P3072.mlpackage |
Core ML vision tower (fp32) per patch bucket; video frames go through it one frame pair at a time (block mask) |
embeddings.f16, pos_embed.f32, readout.f32, tokenizer.json, config.json, coreml_config.json |
host-side tables and config |
mlx/ |
the upstream bf16 weights in the layout mlx-vlm loads (model.safetensors, config.json) |
python/ |
Python hosts: d3_coreml_text.py, d3_coreml_image.py, d3_coreml_video.py (coremltools) and d3_mlx_video.py (MLX, text and video) |
Checked against upstream
Same weights as upstream (revision fe71f8e1). Answers that differ from the upstream PyTorch runtime (bf16, MPS), same requests:
| Core ML | MLX | |
|---|---|---|
| Text (typed-decisions TEST, 20 requests, 100 questions) | 1 | 0 |
| Images (8 head-camera parkour frames × 4 questions) | 0 / 32 | — |
| Video (2-second parkour windows × 8 yes/no questions, 192 answers) | 3 | 3 |
Speed on an M5 Pro (24 GB), one request at a time:
Core ML (CPU_AND_GPU) |
MLX | |
|---|---|---|
| per text question | ~20 ms | ~16 ms |
| per 2-second video look, 8 questions | 440 ms | 112 ms |
| upstream PyTorch (MPS), same look | 651 ms |
MLX is faster on video because it runs all of a request's questions as one batched forward at their exact lengths; the Core ML graph runs one padded pass per question. Neither places ops on the Neural Engine: with CPU_AND_NE, Core ML falls back to the CPU for every op of this architecture.
Python
# Core ML
from d3_coreml_video import D3CoreMLVideo # d3_coreml_text.D3CoreMLText, d3_coreml_image.D3CoreMLImage
m = D3CoreMLVideo("<vllm-sr/d3-edge snapshot>", "<this folder>", max_pixels=100_352)
m.system_one(state="A short clip from a head camera.", videos=[frames_2fps], questions={"high": {"type": "noul", "instructions": "Is the camera high above the ground?"}})
# MLX (needs mlx-vlm >= 0.7.8)
from d3_mlx_video import D3MLXVideo
m = D3MLXVideo("<vllm-sr/d3-edge snapshot>", "<this folder>/mlx", max_pixels=100_352)
m.system_one(state="...", questions={...}, videos=[frames_2fps]) # or videos=None for text
Both hosts build prompts with upstream's own runtime (d3_runtime.py in the upstream snapshot, run without weights), so prompts match upstream exactly. Note for MLX: mlx-vlm's Qwen3.5 builds layer kinds from full_attention_interval, but d3-edge's full-attention layers are 2, 5, 8, 9, 10 and 11 (layer_types in config.json); d3_mlx_video.py patches the layer constructor from layer_types before loading, and loads strictly. Without that, half the layers load no weights.
License: Apache-2.0, as upstream.