msluszniak's picture
Drop MLX: 5.13x/2.25x slower than Core ML fp16 with no size win
0cbd6b4 verified
|
Raw History Blame Contribute Delete
1.97 kB

MLX is not shipped for this model

The MLX artifacts for this model were removed deliberately. The exporter in export-scripts is kept so the decision can be re-tested against a future ExecuTorch/MLX release, but no .pte or config.json is published here.

Why

Measured on device (iPhone 16, ExecuTorch 1.4.1, 3 warmup runs + 15 timed runs, median, identical input shapes allocated from each model's own schema):

model Core ML fp16 MLX fp16 MLX penalty
rfdetr-nano detector 8.4 ms 43.1 ms 5.13x slower
rfdetr-nano segmentation 31.4 ms 70.5 ms 2.25x slower

Both backends delegate fully and return finite output; MLX is simply slower.

There is no size argument either. The MLX and Core ML builds are within 1 MB of each other (detector 55.7 vs 54.7 MB, segmentation 62.0 vs 62.4 MB), so unlike CLIP and distiluse, where MLX int8 ships precisely because it is half the size of the Core ML build, there is nothing here to trade the latency against.

Why RF-DETR is not a special case

MLX runs on the Metal GPU; Core ML lowers to the ANE, which is purpose-built for this kind of network. MLX was measured behind Core ML on every convolutional vision model tested, by 2.25x to 19.6x. Its real advantage is transformer workloads, where weight-only quantization shrinks the artifact substantially: CLIP vision is 97% nn.Linear and goes 351 MB to 98 MB, and distiluse goes 541 MB to 140 MB. RF-DETR is a hybrid conv/transformer model and lands on the wrong side of that line.

A correction to the earlier figure

An earlier note put the RF-DETR MLX penalty at about 33% (Core ML 51 ms vs MLX 68 ms). That comparison is superseded and should not be quoted: it was measured on an iPhone SE 3rd gen, and against the Core ML int8 build that has since been replaced by fp16, which is roughly 4x faster. The table above remeasures both backends at the same precision on the same device.