This behavior seems specific to this model too, other models I have installed load my 7900XTX to 100% when working. This model splits 60 to 80% to the CPU, regardless of the settings. I typically set my models up with JanAI, with the GPU layers set to -1 or 100. MTP on or off. It makes no difference, it constantly dumps to my CPU. I love the prose, but when it loads so much to the CPU, I barely get 10-15 tok/s with it.
I was going to try LM Studio, to see if I get the same effect. But does anyone know why this happens?
Same problem, 80%gpu 20%cpu, which makes 2x slower gen speed(seemingly). Ryzen iGPU 32gb unified memory. All other models run 100% gpu speed. vulkan llama.cpp
I cycled back a few llama.cpp versions; From b9967 to b9743 to b9568, without any change. I also swapped the backend from 'win-vulkan-common_CPUs-x64', my usual default, to 'Win-vulkan-x64', also without success. I also installed LM Studio Bionic, loaded the model and tried it, and got the same CPU/GPU split. Other models don't do this, including 'DavidAU/Qwen3.6-27B-Heretic-Uncensored-FINETUNE-NEO-CODE-Di-IMatrix-MAX-GGUF', which has been my goto writer agent lately.
I'd really like to figure this out, if anyone has any ideas, since this Fable Fusion version writes noticeably better prose. But this split to the CPU craters the tok/s.
Yea, we need a solution, because this model might be a good replacement for stock qwen27b. But because my system already is very slow, no real gpu just igpu, 7ts/s for q4km mtp, I don't want it more slower than that
I would probably suggest to try the latest beta build of llama.cpp. i use an nvidia 5090 and i have no cpu offloading. but it may have something to do with this line of the model card:
In addition the output tensor (10-20% of output) was modified to full precision - 16 bit - for all quants.
it could be that whatever runtime loader you guys are using or however amd's handling of it, it is not compatible. but try the latest llama.cpp beta build to rule that out.
I would probably suggest to try the latest beta build of llama.cpp. i use an nvidia 5090 and i have no cpu offloading. but it may have something to do with this line of the model card:
In addition the output tensor (10-20% of output) was modified to full precision - 16 bit - for all quants.
it could be that whatever runtime loader you guys are using or however amd's handling of it, it is not compatible. but try the latest llama.cpp beta build to rule that out.
Interesting. I grabbed llama.cpp 'b10107' and will test it when I have time later today.
