Instructions to use DevelopingDad/Gemma4-12B-it-Fast with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DevelopingDad/Gemma4-12B-it-Fast with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="DevelopingDad/Gemma4-12B-it-Fast")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("DevelopingDad/Gemma4-12B-it-Fast")
model = AutoModelForMultimodalLM.from_pretrained("DevelopingDad/Gemma4-12B-it-Fast", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DevelopingDad/Gemma4-12B-it-Fast with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DevelopingDad/Gemma4-12B-it-Fast"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DevelopingDad/Gemma4-12B-it-Fast",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/DevelopingDad/Gemma4-12B-it-Fast

SGLang

How to use DevelopingDad/Gemma4-12B-it-Fast with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DevelopingDad/Gemma4-12B-it-Fast" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DevelopingDad/Gemma4-12B-it-Fast",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DevelopingDad/Gemma4-12B-it-Fast" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DevelopingDad/Gemma4-12B-it-Fast",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use DevelopingDad/Gemma4-12B-it-Fast with Docker Model Runner:
```
docker model run hf.co/DevelopingDad/Gemma4-12B-it-Fast
```

Gemma4-12B-it-Fast / benchmarks /README.md

DevelopingDad

Update weight-only checkpoint and generic DGX Spark serving package

231d9a6 verified 5 days ago

preview code

Raw History Blame Contribute Delete

2.91 kB

# Generic native API screen — October 23, 2026

This is a fresh, self-contained diagnostic of the weight-only NVFP4/FP8 target
with higher-precision activations, MTP8 and FP8 KV on one NVIDIA DGX Spark GB10.
It runs directly against a native vLLM 0.31.0 API, using the native chat-template
behavior with system-block whitespace compatibility. No external datasets, custom instructions, automatic repairs or
application middleware are used. The model server was already warm.

The measured serving settings were 65,536 total tokens, eight slots,
4,096 batched tokens, 60 GiB FP8 KV and prefix caching enabled. The three
measured generation requests were sequential and used a unique `cache_salt`
to prevent prefix reuse. This is not a concurrency or long-context test.

| Prompt | Prompt tokens | Completion tokens | First content, seconds | Approximate completion tok/s after first content | Finish |
| --- | ---: | ---: | ---: | ---: | --- |
| explanation | 33 | 512 | 0.076 | 55.0 | length |
| comparison | 37 | 512 | 0.065 | 59.7 | length |
| code | 40 | 512 | 0.123 | 89.1 | length |

All three generation requests reached their 512-token ceiling. These numbers
measure capped generation only and do not establish complete or correct answers.
The rate divides reported completion tokens by the interval from first visible
content to stream completion; usage may include terminal/control tokens. It is
not the conservative delivery calculation from the earlier package. No median,
minimum, warmup exclusion or statistical speedup is inferred from three rows.

Four bounded protocol/function checks passed:

- Opt-in reasoning on one arithmetic expression, with a 2,048-token thinking
  cap and 4,096-token total completion allowance; correct final integer.
- Explicit strict JSON-schema response.
- Forced generic function call with the expected argument.
- Continuation from a synthetic function result.

These are deliberately small checks. They do not establish broad arithmetic,
tool-selection, instruction-following, factual or multimodal quality. Generic
results contain counts, timing and verdicts, not deployment identifiers.

## Reproduce

After starting the native server:

```sh
python3 benchmarks/evaluate.py --base-url http://127.0.0.1:8000/v1 --output generic-results.json
```

The script uses only Python's standard library. It issues seven completion
requests: three generation probes and four function/protocol checks. Request
schemas and prompts are in the script. A failed check gives a nonzero exit code.
Use [results.json](results.json) for the saved machine-readable receipt.

Target tensor/config hashes are recorded in [the artifact manifest](../serving/artifact-manifest.json)
and [tensor integrity report](../tensor-integrity.json). The portable launcher
was checked without restarting the measured server; this receipt does not claim
a new GPU start of the exact portable wrapper.

	# Generic native API screen — October 23, 2026

	This is a fresh, self-contained diagnostic of the weight-only NVFP4/FP8 target
	with higher-precision activations, MTP8 and FP8 KV on one NVIDIA DGX Spark GB10.
	It runs directly against a native vLLM 0.31.0 API, using the native chat-template
	behavior with system-block whitespace compatibility. No external datasets, custom instructions, automatic repairs or
	application middleware are used. The model server was already warm.

	The measured serving settings were 65,536 total tokens, eight slots,
	4,096 batched tokens, 60 GiB FP8 KV and prefix caching enabled. The three
	measured generation requests were sequential and used a unique `cache_salt`
	to prevent prefix reuse. This is not a concurrency or long-context test.

	\| Prompt \| Prompt tokens \| Completion tokens \| First content, seconds \| Approximate completion tok/s after first content \| Finish \|
	\| --- \| ---: \| ---: \| ---: \| ---: \| --- \|
	\| explanation \| 33 \| 512 \| 0.076 \| 55.0 \| length \|
	\| comparison \| 37 \| 512 \| 0.065 \| 59.7 \| length \|
	\| code \| 40 \| 512 \| 0.123 \| 89.1 \| length \|

	All three generation requests reached their 512-token ceiling. These numbers
	measure capped generation only and do not establish complete or correct answers.
	The rate divides reported completion tokens by the interval from first visible
	content to stream completion; usage may include terminal/control tokens. It is
	not the conservative delivery calculation from the earlier package. No median,
	minimum, warmup exclusion or statistical speedup is inferred from three rows.

	Four bounded protocol/function checks passed:

	- Opt-in reasoning on one arithmetic expression, with a 2,048-token thinking
	cap and 4,096-token total completion allowance; correct final integer.
	- Explicit strict JSON-schema response.
	- Forced generic function call with the expected argument.
	- Continuation from a synthetic function result.

	These are deliberately small checks. They do not establish broad arithmetic,
	tool-selection, instruction-following, factual or multimodal quality. Generic
	results contain counts, timing and verdicts, not deployment identifiers.

	## Reproduce

	After starting the native server:

	```sh
	python3 benchmarks/evaluate.py --base-url http://127.0.0.1:8000/v1 --output generic-results.json
	```

	The script uses only Python's standard library. It issues seven completion
	requests: three generation probes and four function/protocol checks. Request
	schemas and prompts are in the script. A failed check gives a nonzero exit code.
	Use [results.json](results.json) for the saved machine-readable receipt.

	Target tensor/config hashes are recorded in [the artifact manifest](../serving/artifact-manifest.json)
	and [tensor integrity report](../tensor-integrity.json). The portable launcher
	was checked without restarting the measured server; this receipt does not claim
	a new GPU start of the exact portable wrapper.