Extract pinned MiniMax M3.1 Flash Tower and Projector with exact source tensors

LICENSE

@@ -0,0 +1,17 @@
+MINIMAX COMMUNITY LICENSE
+Copyright (c) 2026 MiniMax
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software for non-commercial purposes, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or provide copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+1. The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+2. If the Software (or any derivative works thereof) is used for any Commercial Use for your products or services: 
+  1. you shall prominently display “Built with MiniMax M3.1-Flash” on a related website, user interface, blogpost, about page or product documentation. 
+  2. you shall obtain a separate, prior written authorization from MiniMax by contacting api@minimax.io with the subject line “M3.1-Flash licensing - authorization request”, if such products and services generate more than 20 million US dollars (or equivalent in other currencies) in yearly revenue; otherwise, you only need to send a one-time notice to api@minimax.io with the subject “M3.1-Flash licensing — notice”.
+3. “Commercial Use” means any use of the Software or any derivative work thereof that is primarily intended for commercial advantage or monetary compensation, which includes, without limitation: (i) offering products or services to third parties for a fee, which utilize, incorporate, or rely on the Software or its derivatives, (ii) the commercial use of APIs provided by or for the Software or its derivatives, including to support or enable commercial products, services, or operations, whether in a cloud-based, hosted, or other similar environment, and (iii) the deployment or provision of the Software or its derivatives that have been subjected to post-training, fine-tuning, instruction-tuning, or any other form of modification, for any commercial purpose.
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+Appendix: Prohibited Uses 
+You agree you will not use, or allow others to use, the Software or any derivatives of the Software to:
+1. Generate or disseminate content prohibited by applicable laws or regulations.
+2. Assist with, engage in or otherwise support any military purpose.
+3. Exploit, harm, or attempt to exploit or harm minors.
+4. Generate or disseminate false or misleading information with the intent to cause harm.
+5. Promote discrimination, hate speech, or harmful behavior against individuals or groups based on race or ethnic origin, religion, disability, age, nationality and national origin, veteran status, sexual orientation, gender or gender identity, caste, immigration status, or any other characteristic that is associated with systemic discrimination or marginalization.

NOTICE

@@ -0,0 +1,5 @@
+MiniMax-M3.1-Flash Vision
+Extracted from MiniMaxAI/MiniMax-M3.1-Flash at 54e4e69dfc452b1e94a426c24a4fdeeaa8e3ad5e.
+Copyright MiniMax. Distributed under the accompanying MiniMax Community License.
+Tower and Projector tensors are preserved exactly. State-dictionary keys are adapted for native Transformers classes.
+This is a standalone vision extraction, not the complete language model.

README.md

@@ -0,0 +1,58 @@
+---
+license: other
+license_name: minimax-community
+license_link: LICENSE
+pipeline_tag: image-feature-extraction
+library_name: transformers
+base_model:
+- MiniMaxAI/MiniMax-M3.1-Flash
+tags:
+- vision
+---
+
+# MiniMax-M3.1-Flash Vision
+
+Standalone Tower and Projector extracted from MiniMaxAI/MiniMax-M3.1-Flash,
+revision `54e4e69dfc452b1e94a426c24a4fdeeaa8e3ad5e`. No language-model weights
+are included. The accompanying MiniMax Community License applies to these weights.
+
+The Tower has 32 layers, width 1280, 16 heads, MLP width 5120, patch size 14,
+temporal patch size 2, and 3D RoPE. The nominal image size is 672.
+At 448 pixels it produces 1024 Tower tokens of width 1280. The bench merges
+2x2 spatial neighbors into 256 tokens of width 5120. The Projector outputs
+256 tokens of width 6144.
+
+The bundle contains 515 Tower tensors in `model.safetensors` and eight Projector
+tensors in `projector.safetensors`. All tensors preserve their pinned source
+values and dtypes. `export_manifest.json` records checkpoint identity and file hashes.
+Shard selection comes from the pinned source index, rather than M3 shard numbers.
+
+## Use
+
+Requires Transformers 5.17.0, PyTorch and Pillow. See `examples/inference.py`.
+Load the Tower with `MiniMaxM3VLVisionModel.from_pretrained`; use `projector.py`
+to load the separate Projector. The native parent processor uses dynamic resizing.
+The benchmark uses a bicubic short-side resize and center crop at 448, CLIP
+normalization and two repeated frames.
+
+## Validation
+
+`scripts/validate_minimax_m3.py` in vision-tower-bench independently loads the
+pinned source tensors into native HF parent classes. Every Tower and Projector
+tensor matches exactly. On fixed synthetic images at 56 and 448 pixels, all
+eight Tower depths, merged tokens and projected tokens match exactly in BF16.
+Changing nominal image size between 672 and 2016 leaves those outputs unchanged.
+The report records the Transformers source hash. The pinned HF parent repository
+contains no separate modeling Python file; this check uses the native HF implementation.
+
+MiniMax M3 and M3.1-Flash share this architecture and preprocessing, but their
+weights differ. Benchmark results and the complete tensor drift report are maintained
+in [vision-tower-bench](https://github.com/immanuel-peter/vision-tower-bench).
+No benchmark score is implied by this release.
+
+## Reproduce
+
+`python scripts/export_minimax_m3_vision.py --checkpoint minimax_m31_flash`
+
+MiniMax released the parent weights and native Transformers implementation.
+See LICENSE and NOTICE.

config.json

@@ -0,0 +1,36 @@
+{
+  "architectures": [
+    "MiniMaxM3VLVisionModel"
+  ],
+  "attention_dropout": 0.0,
+  "dtype": "bfloat16",
+  "hidden_act": "gelu",
+  "hidden_size": 1280,
+  "image_size": 672,
+  "img_token_compression_config": {
+    "image_token_compression_method": "patch_merge",
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "initializer_factor": 1.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 5120,
+  "layer_norm_eps": 1e-05,
+  "model_type": "minimax_m3_vl_vision",
+  "num_attention_heads": 16,
+  "num_channels": 3,
+  "num_hidden_layers": 32,
+  "patch_size": 14,
+  "position_embedding_type": "rope",
+  "projection_dim": 6144,
+  "rope_mode": "3d",
+  "rope_parameters": {
+    "rope_theta": 10000.0,
+    "rope_type": "axial"
+  },
+  "spatial_merge_size": 2,
+  "temporal_patch_size": 2,
+  "transformers_version": "5.17.0",
+  "vision_segment_max_frames": 4,
+  "vocab_size": 32000
+}

examples/inference.py

@@ -0,0 +1,43 @@
+import argparse
+from pathlib import Path
+import sys
+
+from PIL import Image
+import torch
+from transformers import MiniMaxM3VLImageProcessor, MiniMaxM3VLVisionModel
+
+sys.path.insert(0, str(Path(__file__).absolute().parents[1]))
+from projector import load_projector
+
+
+MODEL_ID = "immanuelpeter/MiniMax-M3.1-Flash-Vision"
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Extract MiniMax-M3 visual features.")
+    parser.add_argument("image", type=Path)
+    parser.add_argument("--model", default=MODEL_ID)
+    args = parser.parse_args()
+
+    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+    dtype = torch.bfloat16 if device.type == "cuda" else torch.float32
+    tower = MiniMaxM3VLVisionModel.from_pretrained(args.model, dtype=dtype).to(device).eval()
+    processor = MiniMaxM3VLImageProcessor.from_pretrained(args.model)
+    projector = load_projector(args.model)
+    projector.to(device=device, dtype=dtype).eval()
+    inputs = processor(images=Image.open(args.image).convert("RGB"), return_tensors="pt")
+
+    with torch.inference_mode():
+        merged = tower(
+            pixel_values=inputs["pixel_values"].to(device=device, dtype=dtype),
+            grid_thw=inputs["image_grid_thw"].to(device),
+        ).last_hidden_state
+        tokens = merged.reshape(-1, merged.shape[-1])
+        projected = projector(tokens)
+
+    print("tower", merged.shape)
+    print("projected", projected.shape)
+
+
+if __name__ == "__main__":
+    main()

export_manifest.json

@@ -0,0 +1,14 @@
+{
+  "source_repo": "MiniMaxAI/MiniMax-M3.1-Flash",
+  "source_revision": "54e4e69dfc452b1e94a426c24a4fdeeaa8e3ad5e",
+  "tower_tensors": 515,
+  "projector_tensors": 8,
+  "files": {
+    "model.safetensors": "95e3731e26887d066a0b2291a1574da40a36aba5ba43e476746b9c16a64bcf08",
+    "projector.safetensors": "81fe7c8be0a2e91b26040ab23523713d1b775eadcdee5ac0021593762d3e6332",
+    "config.json": "9c3a6dac0654055594fdde1f148400331f3e55cb72ffe1b7a6806db07ab93a79",
+    "projector_config.json": "672b312ca5fb10324a41936294814367fb371a3891cc6d8ee8df3d22fc0c559c",
+    "preprocessor_config.json": "c95c459d449a2dacd974bcfef8986b2734afd55e73eb05a7aaad30542cf78b6e",
+    "LICENSE": "8c4978e3e0fe0349ff929cc8450463cba3039c4c9fc144d8bd5382ab00df247a"
+  }
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95e3731e26887d066a0b2291a1574da40a36aba5ba43e476746b9c16a64bcf08
+size 1265435224

preprocessor_config.json

@@ -0,0 +1,32 @@
+{
+  "processor_class": "AutoImageProcessor",
+  "auto_map": {
+    "AutoImageProcessor": "image_processor.MiniMaxM3VLImageProcessor",
+    "AutoProcessor": "processing_minimax.MiniMaxVLProcessor",
+    "AutoVideoProcessor": "video_processor.MiniMaxM3VLVideoProcessor"
+  },
+  "process_image_mode": "dynamic_res",
+  "image_mean": [
+    0.48145466,
+    0.4578275,
+    0.40821073
+  ],
+  "image_std": [
+    0.26862954,
+    0.26130258,
+    0.27577711
+  ],
+  "size": [
+    672,
+    672
+  ],
+  "patch_size": 14,
+  "img_token_compression_config": {
+    "image_token_compression_threshold": 1.1,
+    "image_token_compression_method": "patch_merge",
+    "max_image_resolution": 1008,
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "add_start_end_special_tokens": true
+}

projector.py

@@ -0,0 +1,36 @@
+import json
+from pathlib import Path
+
+import torch
+from huggingface_hub import hf_hub_download
+from safetensors.torch import load_file
+from torch import nn
+
+
+class Projector(nn.Module):
+    def __init__(self, config: dict):
+        super().__init__()
+        self.spatial_merge_size = 2
+        self.linear_1 = nn.Linear(config["input_size"], config["hidden_size"], bias=True)
+        self.linear_2 = nn.Linear(config["hidden_size"], config["output_size"], bias=True)
+        self.merge_linear_1 = nn.Linear(config["merged_hidden_size"], config["hidden_size"], bias=True)
+        self.merge_linear_2 = nn.Linear(config["hidden_size"], config["output_size"], bias=True)
+
+    def forward(self, image_features: torch.Tensor) -> torch.Tensor:
+        hidden = torch.nn.functional.gelu(self.linear_1(image_features))
+        hidden = self.linear_2(hidden)
+        hidden = hidden.reshape(hidden.shape[0] // (self.spatial_merge_size ** 2), -1)
+        return self.merge_linear_2(torch.nn.functional.gelu(self.merge_linear_1(hidden)))
+
+
+def load_projector(model: str | Path) -> Projector:
+    path = Path(model)
+    if path.is_dir():
+        config_path = path / "projector_config.json"
+        weights_path = path / "projector.safetensors"
+    else:
+        config_path = Path(hf_hub_download(str(model), "projector_config.json"))
+        weights_path = Path(hf_hub_download(str(model), "projector.safetensors"))
+    projector = Projector(json.loads(config_path.read_text()))
+    projector.load_state_dict(load_file(weights_path))
+    return projector

projector.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81fe7c8be0a2e91b26040ab23523713d1b775eadcdee5ac0021593762d3e6332
+size 468763376

projector_config.json

@@ -0,0 +1,6 @@
+{
+  "input_size": 1280,
+  "hidden_size": 6144,
+  "output_size": 6144,
+  "merged_hidden_size": 24576
+}