d3-oQ4e

This model was quantized using oQ (oMLX v0.7.1) mixed-precision quantization with Improved oQe Quantization (jundot/omlx#4385).

A mixed-precision MLX checkpoint of vllm-sr/d3, the 27B multimodal decision model of Decision 3.0 from vLLM Semantic Router, built on Qwen/Qwen3.8-27B. It reads text or JSON with optional images and videos and returns a probability for every option of choice, yes/no and score questions without generating text. It includes the language backbone, the vision encoder and the code readout.

The code readout (readout.safetensors, FP32) and decision_config.json are copied unchanged from the source. oMLX serves this checkpoint through POST /v1/systemone; Decision 3.0 support (jundot/omlx#4470) is newer than oMLX v0.7.1.dev1.

The effective average is 4.880 bits/weight for the complete checkpoint.

Quantization and Bit Distribution

oQ4e allocates bits by measured layer sensitivity, then rounds each group with an importance matrix (128 samples x 512 tokens) and a weighted least-squares refit of its scale and bias.

Storage format	Logical weights	Share of total	Tensor storage	Effective bits/weight
Affine 4-bit, group 64	21.075B	80.790%	11.041 GiB	4.50
Affine 5-bit, group 64	4.547B	17.429%	2.911 GiB	5.50
BF16	0.463B	1.776%	0.863 GiB	16.00
FP32	0.001B	0.005%	0.005 GiB	32.00
Total	26.087B	100%	14.820 GiB	4.880

Storage Breakdown

Component	Logical weights	Storage
Language backbone	25.625B	14.986 GB / 13.957 GiB
Vision encoder	0.461B	0.921 GB / 0.858 GiB
Code readout	0.001B	0.005 GB / 0.005 GiB
Total	26.087B	15.913 GB / 14.820 GiB

License

Apache-2.0, following the source model. See LICENSE and NOTICE.