llama.cpp build

#8
by ClaudioItaly - opened

Failed to load model: The installed llama.cpp does not recognise this GGUF's model architecture ('sensenova_loop'). Check the model card for its required runtime. If the model is newer than this llama.cpp build, updating llama.cpp may add support; a model requiring a custom runtime may still be incompatible.

Upvoting for llama.cpp support β€” I see SGLang and vLLM PRs are open but not llama, which is strange, because it's a 9B model so presumably it targets laptops/phones. But people run vLLM or SGLang much less in that field than they do llama.cpp/derivatives, simply because neither vLLM nor SGLang support Vulkan (nor OpenCL for that matter).

So you basically have consumer-level LLM that most consumers are unable to run πŸ˜…

Sign up or log in to comment