PixtralViT-Large-4

This repository packages the Tower and Projector from Mistral Large 4.

Contents

File	Tensors	What it holds
model.safetensors	272	Pixtral vision encoder, from the parent's vision_encoder.*
projector.safetensors	4	pre_mm_projector_norm, patch_merger and vision_language_adapter
modeling_pixtral_large4.py		PixtralVisionModel subclass following the parent's numeric order
projector.py, projector_config.json		Projector shapes and loader
config.json, preprocessor_config.json		Vision configuration and PixtralImageProcessor settings
examples/inference.py		Single-image extraction example
export_manifest.json		Source pin, shard SHA256 and tensor name map
LICENSE, LICENSE-CODE, NOTICE		Weight license, code license and attribution

Architecture

Component	Details
Tower	30 blocks, 2048 hidden, 16 heads of 128, SwiGLU 6144, RMSNorm (eps 1e-5), 14 px patches, axial 2D RoPE, no class token, no final norm
Token compression	2x2 patch merge inside the Projector
Projector	RMSNorm(2048), 2x2 patch merge to 8192, Linear(8192, 2048), Linear(2048, 6144), GELU, Linear(6144, 6144), no biases

At 448 pixels the bench reports 1,024 Tower tokens of width 2048, 256 merged tokens of width 2048 and 256 projected tokens of width 6144.

Usage

See examples/inference.py for image feature extraction.

Validation

The release tests compare all 276 tensors with the pinned parent checkpoint using torch.equal. Fixed-image Tower, merged and projected outputs also match the parent implementation bit-for-bit in FP32 and BF16 at 448 and 1540 pixels.

Reproduction

The export script range-reads the 276 vision tensors from the shard the pinned index names, renames them to HF Pixtral names and reorders each wq and wk head from interleaved rotary pairs to rotate-half order. It writes the original BF16 tensors. The script pins the parent revision.

Credits

Mistral AI released the Mistral Large 4 weights and the mistral-inference reference code.

License

The weights keep the Mistral Ecosystem Open-Weight License from the source model. The Python files are MIT; see LICENSE-CODE.