MLX builds for Apple Silicon (6-bit, 8-bit, 4-bit, bf16)

#7
by Paris99 - opened

Hi, and thank you for releasing SenseNova-Loop-9B.

We converted the release to MLX for Apple Silicon, from revision 329a65f. The port runs the model the way your code does: layers 4 to 27 run twice with shared weights, with the two-stream mHC routers kept in float32 and never quantized, and your YaRN setting for 512K context.

MLX builds (stock mlx-lm 0.32+, Python 3.11+, trust_remote_code):

Code, parity script and tests: https://github.com/internet-projects/sensenova-loop-mlx (v0.2.0)

from mlx_lm import load, generate
model, tokenizer = load("netprojects/SenseNova-Loop-9B-MLX-6bit", trust_remote_code=True)

What we checked on a MacBook Pro M5 Max:

  • Agreement with the original model (same next token as the PyTorch reference): bf16 98.6%, 8-bit 97.6%, 6-bit 96.7%, 4-bit 84.0%.
  • Chat and thinking mode: every MLX build answered short chat and thinking-mode prompts correctly.
  • loop_gate: the release adds a loop_gate.logits tensor that the reference code never reads, so the builds leave it out. Please tell us if that changes.
  • Long context: the 6-bit build found a pass code hidden in a 64K-token prompt.

The cards quote only your own benchmark numbers. The builds are Apache-2.0, with your licence and a NOTICE crediting SenseTime / SenseNova. They are published by NetProjects, and we are not affiliated with SenseTime or SenseNova. If you would like anything changed on the cards, please tell us here.

Paris Michalas, NetProjects

Sign up or log in to comment