File size: 2,215 Bytes
c6a35a1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 | ---
license: other
license_name: vips
license_link: LICENSE
language:
- en
base_model:
- Qwen/Qwen3-VL-8B-Instruct
datasets:
- nyu-visionx/VSI-Bench
tags:
- vips
- spatial-understanding
- video-question-answering
- visual-priors
- lora
- arxiv:2607.15054
---
# Beyond Single Expert: Harmonizing Diverse Visual Priors in MLLMs for Spatial Understanding
**NeurIPS 2026**
Xiao Lin, Xiaohu Huang, Kai Han
Visual AI Lab, The University of Hong Kong
[Paper](https://arxiv.org/abs/2607.15054) · [Project page](https://visual-ai.github.io/vips/) · [Code](https://github.com/Visual-AI/ViPS)
## Checkpoints
| Location | Description |
| --- | --- |
| `vips/` | ViPS checkpoint. |
| `vggt_multihead.pt` | Prior proxy checkpoint for prior feature extraction. |
## Download
ViPS requires [Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct). Run these commands from the [code repository](https://github.com/Visual-AI/ViPS) root:
```bash
hf download leeiioolin/ViPS --local-dir checkpoints
hf download Qwen/Qwen3-VL-8B-Instruct --local-dir models/Qwen3-VL-8B-Instruct
```
See the [code README](https://github.com/Visual-AI/ViPS#readme) for setup and usage.
## Evaluation of the ViPS checkpoint
VSI-Bench results:
| Model | Overall | Obj. Count | Abs. Dist. | Obj. Size | Room Size | Rel. Dist. | Rel. Dir. | Route Plan | Appr. Order |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| ViPS | **63.7** | 69.6 | 54.7 | 71.1 | 72.5 | 60.1 | 81.9 | 45.9 | 53.6 |
## License
| Component | License |
|:--|:--|
| Original ViPS code and original adapter / fusion-module contributions | [MIT](LICENSE) |
| Qwen-derived components | [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0) |
| Prior Proxy (`vggt_multihead.pt`, derived from VGGT-1B) | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) — non-commercial |
See [LICENSE](LICENSE) for attribution, scope, and full terms.
## Citation
```bibtex
@article{Lin2026ViPS,
author = {Xiao Lin and Xiaohu Huang and Kai Han},
title = {Beyond Single Expert: Harmonizing Diverse Visual Priors in MLLMs for Spatial Understanding},
journal = {arXiv preprint arXiv:2607.15054},
year = {2026}
}
```
|