metadata
license: other
license_name: vips
license_link: LICENSE
language:
- en
base_model:
- Qwen/Qwen3-VL-8B-Instruct
datasets:
- nyu-visionx/VSI-Bench
tags:
- vips
- spatial-understanding
- video-question-answering
- visual-priors
- lora
- arxiv:2607.15054
Beyond Single Expert: Harmonizing Diverse Visual Priors in MLLMs for Spatial Understanding
NeurIPS 2026
Xiao Lin, Xiaohu Huang, Kai Han
Visual AI Lab, The University of Hong Kong
Paper · Project page · Code
Checkpoints
| Location | Description |
|---|---|
vips/ |
ViPS checkpoint. |
vggt_multihead.pt |
Prior proxy checkpoint for prior feature extraction. |
Download
ViPS requires Qwen3-VL-8B-Instruct. Run these commands from the code repository root:
hf download leeiioolin/ViPS --local-dir checkpoints
hf download Qwen/Qwen3-VL-8B-Instruct --local-dir models/Qwen3-VL-8B-Instruct
See the code README for setup and usage.
Evaluation of the ViPS checkpoint
VSI-Bench results:
| Model | Overall | Obj. Count | Abs. Dist. | Obj. Size | Room Size | Rel. Dist. | Rel. Dir. | Route Plan | Appr. Order |
|---|---|---|---|---|---|---|---|---|---|
| ViPS | 63.7 | 69.6 | 54.7 | 71.1 | 72.5 | 60.1 | 81.9 | 45.9 | 53.6 |
License
| Component | License |
|---|---|
| Original ViPS code and original adapter / fusion-module contributions | MIT |
| Qwen-derived components | Apache-2.0 |
Prior Proxy (vggt_multihead.pt, derived from VGGT-1B) |
CC BY-NC 4.0 — non-commercial |
See LICENSE for attribution, scope, and full terms.
Citation
@article{Lin2026ViPS,
author = {Xiao Lin and Xiaohu Huang and Kai Han},
title = {Beyond Single Expert: Harmonizing Diverse Visual Priors in MLLMs for Spatial Understanding},
journal = {arXiv preprint arXiv:2607.15054},
year = {2026}
}