---
license: mit
pipeline_tag: image-feature-extraction
library_name: transformers
tags:
- vision
- feature-extraction
- mimo
base_model:
- XiaomiMiMo/MiMo-V2.6-Pro-RL
---

# MiMoViT-2.6-Pro

This repository packages the vision Tower and merger from [MiMo V2.6 Pro](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RL).

## Contents

| File | Tensors | What it holds |
|---|---|---|
| `model.safetensors` | 364 | Tower blocks, patch embedding and merger, from the parent's `visual.*` |
| `modeling_mimo_vision.py` | | Vision classes extracted from the parent, loaded with `trust_remote_code` |
| `config.json`, `preprocessor_config.json` | | Vision configuration and the parent's `Qwen2VLImageProcessor` settings |
| `export_manifest.json` | | Source pin and the SHA256 of every tensor |
| `LICENSE`, `LICENSE-CODE`, `NOTICE` | | Weight and code licenses |

## Architecture

| Component | Details |
|---|---|
| Tower | 28 blocks, 1280 hidden, SwiGLU 4608, 32 query heads, 8 KV heads, head dimension 64, RMSNorm, 2D RoPE |
| Patches | 2 frames x 16 x 16 pixels; a still image is repeated as two frames |
| Attention | Full attention at blocks 0, 9, 18 and 27; the other blocks use a ±64-token window and a per-head sink bias on key column 0 |
| Merger | `LayerNorm(1280)`, 2x2 concatenation to 5120, `Linear(5120, 5120)`, GELU, `Linear(5120, 6144)`, no biases |

At 448 the bench reports 784 Tower tokens of width 1280 (restored to raster
order), 196 `merged` tokens of width 5120, and 196 `projected` tokens of width 6144.

## Usage

See [`examples/inference.py`](examples/inference.py) for image feature extraction.
`AutoModel.from_pretrained(..., trust_remote_code=True)` loads `modeling_mimo_vision.py`.

## Validation

The [release parity test](https://github.com/immanuel-peter/vision-tower-bench/blob/main/tests/test_release_parity.py) compares every exported tensor with the pinned parent using `torch.equal`, then checks all stages against the parent's own `MiMoVisionTransformer`.

## Reproduction

The [export script](https://github.com/immanuel-peter/vision-tower-bench/blob/main/scripts/export_mimo26_vision.py) reads the `visual.*` tensors from the pinned parent and writes the original BF16 tensors.

## Credits

Xiaomi released the MiMo V2.6 Pro weights and the vision implementation.

## License

Weights are MIT, as declared on the source model card. `modeling_mimo_vision.py` keeps its Apache-2.0 header; see [`LICENSE-CODE`](LICENSE-CODE) and [`NOTICE`](NOTICE).