Apple Silicon

#6
by devi9br - opened

llama.cpp or apple silicon?

Thanks for your interest! We expect to release llama.cpp support tomorrow, with Apple Silicon support coming soon as well. We appreciate your patience and support.

For Apple Silicon there are MLX builds of this release now, which run in stock mlx-lm (Python 3.11+, mlx-lm 0.32+): https://huggingface.co/netprojects/SenseNova-Loop-9B-MLX-6bit (also 8-bit, 4-bit and bf16, linked from the card).

from mlx_lm import load, generate
model, tokenizer = load("netprojects/SenseNova-Loop-9B-MLX-6bit", trust_remote_code=True)

(Community build by NetProjects, not affiliated with SenseNova.)

Sign up or log in to comment