---
license: apache-2.0
license_link: https://ai.google.dev/gemma/docs/gemma_4_license
base_model:
- google/gemma-4-26B-A4B-it
- z-lab/gemma-4-26B-A4B-it-DFlash
pipeline_tag: text-generation
tags:
- radiance
- rocm
- amd
- rdna4
- fp8
- moe
- speculative-decoding
- dflash
---

# Gemma 4 26B-A4B-it + DFlash, FP8 experts — radiance container

[google/gemma-4-26B-A4B-it](https://huggingface.co/google/gemma-4-26B-A4B-it) as a single `.rad`
container for the [radiance](https://codeberg.org/StillDeadcode/radiance) inference engine (AMD RDNA4,
ROCm), with z-lab's [DFlash](https://huggingface.co/z-lab/gemma-4-26B-A4B-it-DFlash) block-diffusion
drafter merged in for speculative decoding. radiance's Gemma 4 26B-A4B support is by Perire.

| | |
|---|---|
| File | `gemma4-26b-a4b-dflash-fp8.rad` — 33.94 GiB |
| Weights | the routed experts' gate_up projections FP8 E4M3 per 128×128 block (bf16 scales); their down projections, the attention, the dense MLP, the router, the norms and the tied embedding bf16 as the checkpoint ships them |
| Speculator | DFlash, drafting 15 tokens a step. Its linears and its head are FP8 E4M3 per 128×128 block; the drafter only proposes, and the model verifies every token it emits |
| Context | 262,144 tokens (the checkpoint's trained length) |
| Modalities | text |
| Needs | radiance 1.5.0 or newer |

## Serve

```sh
radiance --model gemma4-26b-a4b-dflash-fp8.rad --tp 2 --max-model-len 65536 --kv-cache-dtype fp8 \
    --max-num-seqs 8 --max-num-batched-tokens 2048 --draft-window adaptive --host 0.0.0.0 --port 8000
```

The Docker image is `stilldeadcode/radiance`. The server speaks the OpenAI API (`/v1/chat/completions`,
`/v1/completions`), with tool calls and structured output. `--draft-window adaptive` lets the drafter's
window follow how much of it is accepted; without it every step drafts the container's 15 tokens.

Measured on two Radeon AI PRO R9700 (gfx1201), one sequence:

| | prose | code |
|---|---|---|
| fixed window (15) | 113.7 tok/s | 216.7 tok/s |
| `--draft-window adaptive` | 159.0 tok/s | 249.3 tok/s |

An 11.5K-token prompt is prefilled in 1.0 s.

## How this file was made

```sh
rad-convert google/gemma-4-26B-A4B-it --draft-model z-lab/gemma-4-26B-A4B-it-DFlash \
    --recipe gemma4-26b-a4b-df.recipe -o gemma4-26b-a4b-dflash-fp8.rad
```

The recipe (`gemma4-26b-a4b-df.recipe` in this repository) names what is quantised; everything it does
not name is the checkpoint's bf16:

```
blk.*.ffn_gate_up_exps.*.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dflash.fc.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dflash.blk.*.attn_q.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dflash.blk.*.attn_k.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dflash.blk.*.attn_v.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dflash.blk.*.attn_output.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dflash.blk.*.ffn_gate_up.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dflash.blk.*.ffn_down.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dflash.draft_head.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
```