Any plans for Gemma 4 26B-A4B DFlash FP8 with vision support?

#1
by kaczy44 - opened

Hi StillDeadcode,

First of all, thank you for your excellent Gemma 4 26B-A4B DFlash FP8 conversion!

I'm currently running your model with Brilliance/Radiance on Windows 11 using:

  • 2Γ— AMD Radeon AI PRO R9700 (32 GB each)
  • Tensor Parallelism: 2
  • 256K context with FP8 KV cache
  • DFlash Adaptive (depth 15)
  • Expert-tiered placement

The model works very well, and I've successfully tested the full 256K context, including Needle-in-a-Haystack tests.

The only thing preventing me from using it as my main everyday model is the lack of vision support.

I noticed that the current .rad container is marked as text-only, even though the original Gemma 4 26B-A4B supports image understanding.

Are you planning to release a multimodal version of this model with vision support while retaining DFlash speculative decoding?

Also, is this mainly a matter of including the vision encoder in the conversion, or are additional changes to Radiance required?

I currently use the QAT Q4_K_XL version in llama.cpp when I need vision, but I'd really prefer to use your faster Radiance/DFlash implementation for everything.

Thanks again for your work!

Hey there, vision is generally supported, just need to be included in the container. You should easily be able to do a repack of the existing container and add the ViT to it, just ask your agent to do this using the existing quant, rad-convert and the original visions tower. Feel free to upload the result

Ah sorry i read your question wrong, i though this was about the 31B, vision support is currently missing for the 26B A4B, feel free to open a pull request to add it to the repo, should be easy

Sign up or log in to comment