chexone-4bit-mlx

MLX quantization of StanfordAIMI/CheXOne for Apple Silicon.

Variant: Affine int4
Disk size: 2945 MB
Quantized by: sahilchachra

Note on effective bpw: mlx-vlm's quantizers only act on the language tower's linear weights. The vision encoder and embeddings stay at the source dtype (bf16), so the headline variant name reflects the LM-tower quantization while the on-disk size averages the two halves of the model.

Benchmark results

Evaluated on Apple M4 Pro with MLX. Model loaded once; performance and quality measured in a single pass.

Performance

This model FP16 baseline
Decode tok/s (avg, long traces) 116.94 38.67
Peak memory (GB) 3.84 7.876
Disk size (MB) 2945 7175

Quality

Benchmark This model FP16 baseline n
VQA-RAD (radiology VQA, accuracy) 20.0% 36.7% 30

Context scaling (decode tok/s)

Context length Decode tok/s
~128 tokens 118.4
~256 tokens 116.2
~512 tokens 117.4
~1024 tokens 115.8

Usage

pip install mlx-vlm
from mlx_vlm import load, generate

model, processor = load("sahilchachra/chexone-4bit-mlx")
response = generate(model, processor, prompt="Describe this image.",
                    image="path/to/image.jpg", max_tokens=256, verbose=True)

All variants in this collection

Model Variant
sahilchachra/chexone-4bit-mlx Affine int4 โ† this model
sahilchachra/chexone-8bit-mlx Affine int8
sahilchachra/chexone-mxfp4-mlx Block float MX FP4

Notes

  • Requires Apple Silicon (M1 or later) with MLX
  • Benchmarks run on Apple M4 Pro, 24 GB unified memory
  • License: see StanfordAIMI/CheXOne for the original model's license

Original model

See StanfordAIMI/CheXOne for full model details and intended use.

Downloads last month
16
Safetensors
Model size
4B params
Tensor type
U32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for sahilchachra/chexone-4bit-mlx

Base model

Qwen/Qwen2.5-3B
Quantized
(5)
this model