immanuelpeter/Gemma4-31B-Vision

Gemma4-31B Vision

This repository packages the Tower and Projector from Gemma 4 31B.

Contents

File	Tensors	What it holds
model.safetensors	355	Tower, extracted from model.vision_tower.* in model-00001-of-00002.safetensors
projector.safetensors	1	Linear(1152, 5376) weight from model.embed_vision.*
projector_config.json		Projector shapes
config.json		Vision-only Gemma4VisionModel (model_type: gemma4_vision)
preprocessor_config.json		Gemma 4 image preprocessing configuration
projector.py		Projector loader used by examples/inference.py

Architecture

Component	Details
Tower	27 layers, 1152 hidden, 16 heads, 4304 intermediate, patch size 16, 3x3 pooling kernel
Projector	scale-free RMSNorm(1152) then Linear(1152, 5376) with no bias

The RMSNorm has no learned scale, so the packaged Projector tensor is that one Linear weight.

Usage

See examples/inference.py for image feature extraction.

Validation

The parity script compares the 355 Tower tensors and the Projector Linear with the pinned parent using torch.equal.

Reproduction

The export script reads model.vision_tower.* (355 tensors) and model.embed_vision.* (1 tensor) from model-00001-of-00002.safetensors of google/gemma-4-31B-it. It strips the prefixes and writes the original BF16 tensors.

Credits

Google DeepMind released the Gemma 4 31B weights and the native Transformers implementation. The Gemma team also wrote a technical report that describes the vision encoder in more detail.

License

Apache License 2.0, the same license as the source model.