Instructions to use google/diffusiongemma-26B-A4B-it with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use google/diffusiongemma-26B-A4B-it with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="google/diffusiongemma-26B-A4B-it")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("google/diffusiongemma-26B-A4B-it")
model = AutoModelForMultimodalLM.from_pretrained("google/diffusiongemma-26B-A4B-it", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use google/diffusiongemma-26B-A4B-it with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "google/diffusiongemma-26B-A4B-it"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "google/diffusiongemma-26B-A4B-it",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/google/diffusiongemma-26B-A4B-it

SGLang

How to use google/diffusiongemma-26B-A4B-it with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "google/diffusiongemma-26B-A4B-it" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "google/diffusiongemma-26B-A4B-it",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "google/diffusiongemma-26B-A4B-it" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "google/diffusiongemma-26B-A4B-it",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use google/diffusiongemma-26B-A4B-it with Docker Model Runner:
```
docker model run hf.co/google/diffusiongemma-26B-A4B-it
```

ADD FAST-dLLM mode for more speed

by VaLtEc-BoY - opened Jun 12

Discussion

VaLtEc-BoY

Jun 12

https://nvlabs.github.io/Fast-dLLM/

LFGuys

Jun 13

good

TomLucidor

about 1 month ago

How would this work, or if it is relevant at all when we also have DFlash/DDTree types + Nemotron's Diffusion systems?

VaLtEc-BoY

about 1 month ago

Unfortunately, I don't know how they work, but I think it would be equivalent to MTP (multiple token predictions)....I believe anything that accelerates this model architecture should be studied, considering that today many tokens are used and artificial intelligence uses quite a lot, taking longer to display a response... I believe that any diffusion model is better with some kind of TURBO, as happens in video and image creators... it must be something equivalent to that.

VaLtEc-BoY

about 1 month ago

•

edited about 1 month ago

For now I've been trying to quantize this model https://huggingface.co/nvidia/Nemotron-Labs-Diffusion-VLM-8B/ because it already has VL and would obviously run on weaker cards... if you want to try using it, it even fits a 16GB GPU.. well, unfortunately quantizing these models is a bit difficult. I'm trying to create a cell in Colab to make some quantized versions to release to the community, including a heretical, uncensored version... however, I don't know when I'll be able to do it... I tried to do it with the help of mradermacher, but unfortunately I believe it won't work. Nvidia is difficult to quantize diffusion text models; you have to know how to quantize them because it's equivalent to quantizing an image or video model. It's not a simple process, and I'm not the most qualified. Even so, with the help of opencode/glm5.2 and antigravity, I'm figuring out a way to make it viable, maybe even here on HF. I'll study the best and cheapest way...

TomLucidor

30 days ago

@VaLtEc-BoY please help out in general with parsers/templates if you can. If there are already issues with AR models then dLLMs are practically unreachable for anyone (yes this goes for nvidia models as well) https://github.com/raullenchai/Rapid-MLX/discussions/1062#discussioncomment-17598198

VaLtEc-BoY

30 days ago

I’m going to look into that; the models really do have flaws in their templates. One thing that’s particularly annoying about Gemma 12b is that, despite being powerful, it can't maintain a simple footnote without needing a lot of reinforcement—something that doesn't happen with Qwen 3.5/3.6. I'll see if I can add something to address that.

VaLtEc-BoY

29 days ago

https://drive.google.com/drive/folders/1Ci-7BVBP6XTARRYk8zcXmFWsQFogtV4P?usp=drive_link ,Okay, I’ve redone the Jinja templates with corrections for various models; you just need to rename the file to chat_template.jinja for the specific model you're using. I don't know if it will work, but it's worth a try. You'll need to replace the original when running with Safetensors or call the Jinja template according to the execution parameters of the engine being used (like Llama, etc.). I can't guarantee it will work, but it's better than waiting for Google's fixes, which take a long time to arrive.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment