MLX builds for Apple Silicon (8-bit, 4-bit and 3-bit mixed, with vision)
#3
by Paris99 - opened
Thanks for releasing JEV-35B under Apache-2.0. We've published MLX builds for Macs:
- netprojects/JEV-35B-mlx-8bit (36 GB)
- netprojects/JEV-35B-mlx-4bit-mixed (21 GB: experts 4-bit, other layers 8-bit)
- netprojects/JEV-35B-mlx-3bit-mixed (17 GB: experts 3-bit, other layers 8-bit)
All three keep the vision tower and router gates in bf16. jev35b-mlx reproduces the System 1 read-out of your serve_decide.py (slot logits + head bias, per-kind temperature from calibration.json), as a Python decide() and the same /v1/decide endpoint.
Against the original bf16 weights, on 231 decisions from the JevBench public set, the builds pick the same answer on 228 / 225 / 224 of 231 (mean KL 0.002 / 0.004 / 0.012). On an M5 Max a warm decision takes about 100–165 ms.
Happy to adjust anything if you'd like the read-out handled differently.
Paris Michalas, NetProjects