File size: 2,595 Bytes
accba69 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | ---
license: other
license_name: minimax-community
license_link: LICENSE
pipeline_tag: image-feature-extraction
library_name: transformers
tags:
- vision
base_model:
- MiniMaxAI/MiniMax-M3
---
# MiniMax-M3 Vision
This repository packages the Tower and Projector from
[MiniMax M3](https://huggingface.co/MiniMaxAI/MiniMax-M3).
## Contents
| File | Tensors | What it holds |
|---|---|---|
| `model.safetensors` | 515 | MiniMax-M3 vision tower |
| `projector.safetensors` | 8 | `multi_modal_projector` and remapped `patch_merge_mlp` |
| `projector_config.json`, `projector.py` | | Projector shapes and loader |
| `config.json`, `preprocessor_config.json` | | Vision-only model and image-processing configuration |
| `LICENSE` | | MiniMax Community License |
## Architecture
| Component | Details |
|---|---|
| Tower | CLIP-style ViT, 32 layers, 1280 hidden, 16 heads, 5120 intermediate, patch size 14, Conv3d patch embed, 3D RoPE, GELU, image size 2016 |
| Token compression | 2x2 spatial merge, temporal patch size 2 |
| Projector | `Linear(1280, 6144)`, GELU, `Linear(6144, 6144)`, flatten four 6144-wide patches, `Linear(24576, 6144)`, GELU, `Linear(6144, 6144)` |
The class is `MiniMaxM3VLVisionModel` (`model_type` `minimax_m3_vl_vision`).
`last_hidden_state` is the raw 1280-wide Tower tokens. At 448 the bench reports 1280-wide
Tower tokens, 5120-wide merged tokens (2x2 of 1280), and 6144-wide projected tokens.
`projector_config.json` records `input_size` 1280, `hidden_size` 6144, `output_size` 6144,
and `merged_hidden_size` 24576.
## Usage
See [`examples/inference.py`](examples/inference.py) for image feature extraction.
## Validation
The [parity script](https://github.com/immanuel-peter/vision-tower-bench/blob/main/tests/test_parity.py)
compares all 515 Tower tensors and eight Projector tensors with the pinned parent
checkpoint using `torch.equal`.
## Reproduction
The [export script](https://github.com/immanuel-peter/vision-tower-bench/blob/main/scripts/export_minimax_m3_vision.py)
reads `vision_tower.vision_model.*` from shard 59 of `MiniMaxAI/MiniMax-M3`.
It remaps `encoder.layers` to `layers` and `embeddings.patch_embedding` to `embeddings.proj`,
then writes 515 Tower tensors. The Projector comes from `multi_modal_projector.*` and `patch_merge_mlp.*`
in shards 26 and 59, with merge linear names remapped, for eight tensors. The original BF16 weights
are preserved.
## Credits
MiniMax released the MiniMax-M3 weights and the native Transformers implementation.
## License
[MiniMax Community License](LICENSE), the same license as the source model.
|