chexone-8bit-mlx

MLX quantization of StanfordAIMI/CheXOne for Apple Silicon.

Variant: Affine int8
Disk size: 4416 MB
Quantized by: sahilchachra

Note on effective bpw: mlx-vlm's quantizers only act on the language tower's linear weights. The vision encoder and embeddings stay at the source dtype (bf16), so the headline variant name reflects the LM-tower quantization while the on-disk size averages the two halves of the model.

Benchmark results

Evaluated on Apple M4 Pro with MLX. Model loaded once; performance and quality measured in a single pass.

Performance

This model FP16 baseline
Decode tok/s (avg, long traces) 70.44 38.67
Peak memory (GB) 5.216 7.876
Disk size (MB) 4416 7175

Quality

Benchmark This model FP16 baseline n
VQA-RAD (radiology VQA, accuracy) 30.0% 36.7% 30

Context scaling (decode tok/s)

Context length Decode tok/s
~128 tokens 71.3
~256 tokens 70.8
~512 tokens 71.0
~1024 tokens 68.7

Usage

pip install mlx-vlm
from mlx_vlm import load, generate

model, processor = load("sahilchachra/chexone-8bit-mlx")
response = generate(model, processor, prompt="Describe this image.",
                    image="path/to/image.jpg", max_tokens=256, verbose=True)

All variants in this collection

Model Variant
sahilchachra/chexone-4bit-mlx Affine int4
sahilchachra/chexone-8bit-mlx Affine int8 โ† this model
sahilchachra/chexone-mxfp4-mlx Block float MX FP4

Notes

  • Requires Apple Silicon (M1 or later) with MLX
  • Benchmarks run on Apple M4 Pro, 24 GB unified memory
  • License: see StanfordAIMI/CheXOne for the original model's license

Original model

See StanfordAIMI/CheXOne for full model details and intended use.

Downloads last month
12
Safetensors
Model size
4B params
Tensor type
U32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for sahilchachra/chexone-8bit-mlx

Base model

Qwen/Qwen2.5-3B
Quantized
(5)
this model