Any plans for Gemma 4 26B-A4B DFlash FP8 with vision support?
Hi StillDeadcode,
First of all, thank you for your excellent Gemma 4 26B-A4B DFlash FP8 conversion!
I'm currently running your model with Brilliance/Radiance on Windows 11 using:
- 2Γ AMD Radeon AI PRO R9700 (32 GB each)
- Tensor Parallelism: 2
- 256K context with FP8 KV cache
- DFlash Adaptive (depth 15)
- Expert-tiered placement
The model works very well, and I've successfully tested the full 256K context, including Needle-in-a-Haystack tests.
The only thing preventing me from using it as my main everyday model is the lack of vision support.
I noticed that the current .rad container is marked as text-only, even though the original Gemma 4 26B-A4B supports image understanding.
Are you planning to release a multimodal version of this model with vision support while retaining DFlash speculative decoding?
Also, is this mainly a matter of including the vision encoder in the conversion, or are additional changes to Radiance required?
I currently use the QAT Q4_K_XL version in llama.cpp when I need vision, but I'd really prefer to use your faster Radiance/DFlash implementation for everything.
Thanks again for your work!
Hey there, vision is generally supported, just need to be included in the container. You should easily be able to do a repack of the existing container and add the ViT to it, just ask your agent to do this using the existing quant, rad-convert and the original visions tower. Feel free to upload the result
Ah sorry i read your question wrong, i though this was about the 31B, vision support is currently missing for the 26B A4B, feel free to open a pull request to add it to the repo, should be easy