shieldgemma-9b-smoothquant-w8a8

SmoothQuant + GPTQ W8A8 INT8 checkpoint of google/shieldgemma-9b at immutable revision b8b636016df4540721a098c7aab91c97ec6ee508.

Quantization configuration

• Smoothing algorithm: SmoothQuant
• Smoothing strength (alpha): 0.5
• W8A8 quantizer: GPTQ
• Weight precision: INT8
• Activation precision: dynamic per-token INT8
• Targets: all linear layers except lm_head
• Source and non-quantized module dtype: BF16
• KV cache quantization: disabled
• Implementation: llmcompressor==0.8.1
• Serialization: compressed-tensors==0.12.2

Calibration

• Frozen samples: 500 (250 English and 250 Portuguese)
• Input format: raw text, without model-specific chat templates
• Maximum sequence length: 2048 tokens
• Random seed: 42
• Calibration SHA-256: 65ea2fdb18470289314defc0fb94e27d4877b62143569615a1ce01745607ce1f
• C4 source revision: 1588ec454efa1a09f29cd18ddd04fe05fc8653a2

The resolved base revision, full quantization configuration, package versions, architecture mapping inventory, tensor inventory, and file hashes are included in the checkpoint metadata. vLLM smoke-test evidence is added before publication.

ShieldGemma serialization note

The base checkpoint ties the output projection to model.embed_tokens.weight and does not serialize a separate lm_head.weight. This checkpoint preserves that representation. A redundant, byte-identical lm_head.weight produced by the compression save path is removed before validation; no quantized tensor is changed or omitted.

Gemma2 RMSNorm stores an offset and evaluates norm(x) * (1 + weight). llmcompressor==0.8.1 applies generic SmoothQuant scaling directly to weight, which would not preserve this function. During calibration the pipeline therefore converts the offset to the effective gamma, applies the same alpha=0.5 scaling, and converts it back before serialization. This is an architecture compatibility transformation, not a different quantization configuration.

vLLM validation

This checkpoint passed four fixed English/Portuguese safe/unsafe smoke cases in BF16 with vLLM 0.31.0 on an NVIDIA A40. vLLM loaded the serialized compressed-tensors W8A8 configuration directly; no on-the-fly quantization was used. BF16 is the selected runtime compute dtype for the study. The test host used NVIDIA's cuda-compat-13-0 forward-compatibility package because vLLM 0.31.0 ships with a CUDA 13.0 PyTorch build and the host driver exposes CUDA 12.8.