---
license: other
license_name: vips
license_link: LICENSE
language:
- en
base_model:
- Qwen/Qwen3-VL-8B-Instruct
datasets:
- nyu-visionx/VSI-Bench
tags:
- vips
- spatial-understanding
- video-question-answering
- visual-priors
- lora
- arxiv:2607.15054
---

# Beyond Single Expert: Harmonizing Diverse Visual Priors in MLLMs for Spatial Understanding

**NeurIPS 2026**

Xiao Lin, Xiaohu Huang, Kai Han

Visual AI Lab, The University of Hong Kong

[Paper](https://arxiv.org/abs/2607.15054) · [Project page](https://visual-ai.github.io/vips/) · [Code](https://github.com/Visual-AI/ViPS)

## Checkpoints

| Location | Description |
| --- | --- |
| `vips/` | ViPS checkpoint. |
| `vggt_multihead.pt` | Prior proxy checkpoint for prior feature extraction. |

## Download

ViPS requires [Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct). Run these commands from the [code repository](https://github.com/Visual-AI/ViPS) root:

```bash
hf download leeiioolin/ViPS --local-dir checkpoints
hf download Qwen/Qwen3-VL-8B-Instruct --local-dir models/Qwen3-VL-8B-Instruct
```

See the [code README](https://github.com/Visual-AI/ViPS#readme) for setup and usage.

## Evaluation of the ViPS checkpoint

VSI-Bench results:

| Model | Overall | Obj. Count | Abs. Dist. | Obj. Size | Room Size | Rel. Dist. | Rel. Dir. | Route Plan | Appr. Order |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| ViPS | **63.7** | 69.6 | 54.7 | 71.1 | 72.5 | 60.1 | 81.9 | 45.9 | 53.6 |

## License

| Component | License |
|:--|:--|
| Original ViPS code and original adapter / fusion-module contributions | [MIT](LICENSE) |
| Qwen-derived components | [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0) |
| Prior Proxy (`vggt_multihead.pt`, derived from VGGT-1B) | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) — non-commercial |

See [LICENSE](LICENSE) for attribution, scope, and full terms.

## Citation

```bibtex
@article{Lin2026ViPS,
  author  = {Xiao Lin and Xiaohu Huang and Kai Han},
  title   = {Beyond Single Expert: Harmonizing Diverse Visual Priors in MLLMs for Spatial Understanding},
  journal = {arXiv preprint arXiv:2607.15054},
  year    = {2026}
}
```