https://huggingface.co/mrcuddle/Siren-Personality-2x12B
Mixtral MoE 2x12B model, not quanted at all yet. Would love to see Imatrix quants from you
Mixtral MoE 2x12B model, not quanted at all yet. Would love to see Imatrix quants from you
error/1 ValueError Can not map tensor
I think theres some kind of issue with the model. Ask @nicoboss
I believe (More like Anthropic's Sonnet believes) that the issue is with the names of the layers/experts. I'm absolutely not sure what this means, because I'm nowhere close to being an expert, but maybe it's something that's worth looking into. OP doesn't respond to the report on the community page for more than 15 hours at the time of me writing this, so not sure if he will be able to explain what's wrong
I believe (More like Anthropic's Sonnet believes) that the issue is with the names of the layers/experts. I'm absolutely not sure what this means, because I'm nowhere close to being an expert, but maybe it's something that's worth looking into. OP doesn't respond to the report on the community page for more than 15 hours at the time of me writing this, so not sure if he will be able to explain what's wrong
Nico is a bit busy, and i am not a expert myself too
Siren-Personality-2x12B-MoE is a Mixture of Experts (MoE) model created by combining two specialized 12B parameter models based on the Mistral-Nemo-2407 architecture.
that's the problem. llamacpp works on a predefined architecture, so when you are doing frankenstein like this, it might not work in some cases