Instructions to use DevelopingDad/Gemma4-12B-it-Fast with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DevelopingDad/Gemma4-12B-it-Fast with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="DevelopingDad/Gemma4-12B-it-Fast")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("DevelopingDad/Gemma4-12B-it-Fast")
model = AutoModelForMultimodalLM.from_pretrained("DevelopingDad/Gemma4-12B-it-Fast", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DevelopingDad/Gemma4-12B-it-Fast with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DevelopingDad/Gemma4-12B-it-Fast"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DevelopingDad/Gemma4-12B-it-Fast",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/DevelopingDad/Gemma4-12B-it-Fast

SGLang

How to use DevelopingDad/Gemma4-12B-it-Fast with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DevelopingDad/Gemma4-12B-it-Fast" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DevelopingDad/Gemma4-12B-it-Fast",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DevelopingDad/Gemma4-12B-it-Fast" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DevelopingDad/Gemma4-12B-it-Fast",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use DevelopingDad/Gemma4-12B-it-Fast with Docker Model Runner:
```
docker model run hf.co/DevelopingDad/Gemma4-12B-it-Fast
```

Gemma4-12B-it-Fast / provenance.json

DevelopingDad

Update weight-only checkpoint and generic DGX Spark serving package

231d9a6 verified 9 days ago

Raw History Blame Contribute Delete

4.53 kB

{
  "schema": "gemma4-fast-public-provenance/v2",
  "publisher": "DevelopingDad",
  "model": "DevelopingDad/Gemma4-12B-it-Fast",
  "parent": "DevelopingDad/gemma-4-12b-it-trial62",
  "parent_revision": "568aa9764b4f695fb18881b97f6d4ee8b725f75c",
  "upstream_family": "google/gemma-4-12B-it",
  "upstream_license": "Apache-2.0",
  "release_license": "Apache-2.0",
  "source_inventory_sha256": "2687864388a46a4cfa06f41a60d8bd74c3d068cf95a0fe17a6163db13bef5d28",
  "conversion_receipt_sha256": "e360d1cd28465776b5cea350d0ee1312bd69b2462826697b27ca8e3ef3506f8f",
  "model_safetensors": {
    "bytes": 9304948400,
    "sha256": "f416260268dad318eda7d3a9aef91e4962a70fe95bd58af2cb6092e64c5bed5a"
  },
  "quantization": {
    "mlp": "NVFP4 groups16, imatrix MSE",
    "attention": "FP8 per-channel weights; higher-precision activations",
    "kv": "calibrated scalar FP8",
    "tied_embedding_head": "original BF16 retained byte-for-byte",
    "activations": "weight-only W4A16 MLP / W8A16 attention; activation quantization disabled"
  },
  "calibration": {
    "dataset": "allenai/c4",
    "revision": "607bd4c8450a42878aa9ddc051a65a055450ef87",
    "rows": 128,
    "seed": 91027,
    "max_tokens": 1024,
    "shuffle_buffer": 512,
    "purpose": "post-training quantization calibration; no benchmark/quality prompts"
  },
  "quality_scope": "generic bounded native API diagnostics; comprehensive quality qualification incomplete",
  "runtime_overlay": "db9527a46873454610df6dbedf79a36d6bf1a7f6",
  "packages": {
    "api_files": {
      "compressed-tensors/compressed_tensors/quantization/lifecycle/initialize.py": "67adec4337c194d3082e3e1ff316b2566c27abe684df3b8d9cb6e0a1156678d9",
      "compressed-tensors/compressed_tensors/quantization/quant_args.py": "1014b6ce8c925fc810509a68b71b3031204fb5150641c51b945e44d9363595bf",
      "compressed-tensors/compressed_tensors/quantization/quant_scheme.py": "314b5c0b0644f300d170de4d27d6eeb9b824ee78777f92f8041421104245c1ac",
      "llmcompressor/llmcompressor/entrypoints/oneshot.py": "029b18d0c7695bbecced710264aa689d833b6f8225a5f644c9e2481651106afc",
      "llmcompressor/llmcompressor/modifiers/quantization/quantization/base.py": "2bf6b4db0c2452c0c5729fcb464178ebacf60661bb483b1af44b9ae4302835ee",
      "llmcompressor/llmcompressor/modifiers/quantization/quantization/mixin.py": "3655c940d58e622cec47a68eab1da5b3ea81b4368287f42426594af5cbc5b313",
      "llmcompressor/llmcompressor/observers/imatrix.py": "71145923b217d821b2867b66ee29dba6a4c4fc6217f403c3d7f45019eef29709"
    },
    "source_tree_sha256": {
      "compressed_tensors": "1f896af0dfdf3a6cdc68b96f5a88951709aff7552a5a8559040ad67c44a69e08",
      "llmcompressor": "0f4d70b67d4aa06e45afcb6f9bba3f0eb43bae370db01ee2ab08fc53ad6be7f3"
    },
    "wheels": {
      "compressed_tensors-0.19.0-py3-none-any.whl": "5c2dcfd9cd820521906e23e096a51d68755cb88b67e19b1a5c9e11fa69637fb2",
      "llmcompressor-0.14.0-py3-none-any.whl": "be59276d4b239cbde9d11fd1ea064417306b430fb5a9412b140cce514976a0ea"
    },
    "source_snapshot_label": "audited conversion-package source snapshot"
  },
  "runtime_image": {
    "checked_utc": "2026-10-08T22:57:42.301061+00:00",
    "status": "verified_local_image_registry_binding",
    "public_registry_reference": "vllm/vllm-openai@sha256:3f7dd5b777d34d1724456ce71f87385dca288c3bb23029ab27dee358f5d2b971",
    "local_image_id": "sha256:bbe7045055707d1027079ed01cca40812578a235c56032d85c16108ad473384c",
    "architecture": "arm64",
    "os": "linux",
    "vllm_source_commit": "db9527a46873454610df6dbedf79a36d6bf1a7f6"
  },
  "updated_utc_date": "2026-10-23",
  "checkpoint_parent": {
    "repository": "DevelopingDad/Gemma4-12B-it-Fast",
    "revision": "64c6387bcb0976f2a4ba4a7adb71fe1e41cf2b9e",
    "sha256": "63536b8ec8456dddf8bbb7a71148bf2ab8499394c60226de2039e8846903a04b"
  },
  "checkpoint_update": {
    "retained_tensors": 1245,
    "retained_tensor_data_bytes": 9304785760,
    "removed_unused_mlp_activation_scale_tensors": 144,
    "retained_tensor_bytes_identical_to_parent": true,
    "additional_training": false,
    "additional_weight_quantization": false
  },
  "serving": {
    "total_context_tokens": 65536,
    "sequence_slots": 8,
    "batched_tokens": 4096,
    "kv_cache_dtype": "fp8",
    "kv_cache_bytes": 64424509440,
    "speculative_tokens": 8,
    "prefix_caching": true,
    "chat_template": "native-format compatibility; only system-block whitespace differs from preserved root native template",
    "default_thinking": false,
    "automatic_restart": false
  }
}

	{
	"schema": "gemma4-fast-public-provenance/v2",
	"publisher": "DevelopingDad",
	"model": "DevelopingDad/Gemma4-12B-it-Fast",
	"parent": "DevelopingDad/gemma-4-12b-it-trial62",
	"parent_revision": "568aa9764b4f695fb18881b97f6d4ee8b725f75c",
	"upstream_family": "google/gemma-4-12B-it",
	"upstream_license": "Apache-2.0",
	"release_license": "Apache-2.0",
	"source_inventory_sha256": "2687864388a46a4cfa06f41a60d8bd74c3d068cf95a0fe17a6163db13bef5d28",
	"conversion_receipt_sha256": "e360d1cd28465776b5cea350d0ee1312bd69b2462826697b27ca8e3ef3506f8f",
	"model_safetensors": {
	"bytes": 9304948400,
	"sha256": "f416260268dad318eda7d3a9aef91e4962a70fe95bd58af2cb6092e64c5bed5a"
	},
	"quantization": {
	"mlp": "NVFP4 groups16, imatrix MSE",
	"attention": "FP8 per-channel weights; higher-precision activations",
	"kv": "calibrated scalar FP8",
	"tied_embedding_head": "original BF16 retained byte-for-byte",
	"activations": "weight-only W4A16 MLP / W8A16 attention; activation quantization disabled"
	},
	"calibration": {
	"dataset": "allenai/c4",
	"revision": "607bd4c8450a42878aa9ddc051a65a055450ef87",
	"rows": 128,
	"seed": 91027,
	"max_tokens": 1024,
	"shuffle_buffer": 512,
	"purpose": "post-training quantization calibration; no benchmark/quality prompts"
	},
	"quality_scope": "generic bounded native API diagnostics; comprehensive quality qualification incomplete",
	"runtime_overlay": "db9527a46873454610df6dbedf79a36d6bf1a7f6",
	"packages": {
	"api_files": {
	"compressed-tensors/compressed_tensors/quantization/lifecycle/initialize.py": "67adec4337c194d3082e3e1ff316b2566c27abe684df3b8d9cb6e0a1156678d9",
	"compressed-tensors/compressed_tensors/quantization/quant_args.py": "1014b6ce8c925fc810509a68b71b3031204fb5150641c51b945e44d9363595bf",
	"compressed-tensors/compressed_tensors/quantization/quant_scheme.py": "314b5c0b0644f300d170de4d27d6eeb9b824ee78777f92f8041421104245c1ac",
	"llmcompressor/llmcompressor/entrypoints/oneshot.py": "029b18d0c7695bbecced710264aa689d833b6f8225a5f644c9e2481651106afc",
	"llmcompressor/llmcompressor/modifiers/quantization/quantization/base.py": "2bf6b4db0c2452c0c5729fcb464178ebacf60661bb483b1af44b9ae4302835ee",
	"llmcompressor/llmcompressor/modifiers/quantization/quantization/mixin.py": "3655c940d58e622cec47a68eab1da5b3ea81b4368287f42426594af5cbc5b313",
	"llmcompressor/llmcompressor/observers/imatrix.py": "71145923b217d821b2867b66ee29dba6a4c4fc6217f403c3d7f45019eef29709"
	},
	"source_tree_sha256": {
	"compressed_tensors": "1f896af0dfdf3a6cdc68b96f5a88951709aff7552a5a8559040ad67c44a69e08",
	"llmcompressor": "0f4d70b67d4aa06e45afcb6f9bba3f0eb43bae370db01ee2ab08fc53ad6be7f3"
	},
	"wheels": {
	"compressed_tensors-0.19.0-py3-none-any.whl": "5c2dcfd9cd820521906e23e096a51d68755cb88b67e19b1a5c9e11fa69637fb2",
	"llmcompressor-0.14.0-py3-none-any.whl": "be59276d4b239cbde9d11fd1ea064417306b430fb5a9412b140cce514976a0ea"
	},
	"source_snapshot_label": "audited conversion-package source snapshot"
	},
	"runtime_image": {
	"checked_utc": "2026-10-08T22:57:42.301061+00:00",
	"status": "verified_local_image_registry_binding",
	"public_registry_reference": "vllm/vllm-openai@sha256:3f7dd5b777d34d1724456ce71f87385dca288c3bb23029ab27dee358f5d2b971",
	"local_image_id": "sha256:bbe7045055707d1027079ed01cca40812578a235c56032d85c16108ad473384c",
	"architecture": "arm64",
	"os": "linux",
	"vllm_source_commit": "db9527a46873454610df6dbedf79a36d6bf1a7f6"
	},
	"updated_utc_date": "2026-10-23",
	"checkpoint_parent": {
	"repository": "DevelopingDad/Gemma4-12B-it-Fast",
	"revision": "64c6387bcb0976f2a4ba4a7adb71fe1e41cf2b9e",
	"sha256": "63536b8ec8456dddf8bbb7a71148bf2ab8499394c60226de2039e8846903a04b"
	},
	"checkpoint_update": {
	"retained_tensors": 1245,
	"retained_tensor_data_bytes": 9304785760,
	"removed_unused_mlp_activation_scale_tensors": 144,
	"retained_tensor_bytes_identical_to_parent": true,
	"additional_training": false,
	"additional_weight_quantization": false
	},
	"serving": {
	"total_context_tokens": 65536,
	"sequence_slots": 8,
	"batched_tokens": 4096,
	"kv_cache_dtype": "fp8",
	"kv_cache_bytes": 64424509440,
	"speculative_tokens": 8,
	"prefix_caching": true,
	"chat_template": "native-format compatibility; only system-block whitespace differs from preserved root native template",
	"default_thinking": false,
	"automatic_restart": false
	}
	}