README.md

---
license: mit
library_name: pytorch
tags:
- Robotics
- Vision-Language-Action
- Manipulation
- Multimodal
- Sensor-Fusion
- Diffusion
language:
- en
pipeline_tag: robotics
arxiv:
- 2606.17598
---
<div align="center">
  <span style="font-size:32px;">MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulation</span>
</div>
<p align="center">
  <a href="https://arxiv.org/abs/2606.17598"><img src="https://img.shields.io/badge/arXiv-Paper-red?logo=arxiv&logoColor=white" alt="arXiv"></a>
  <a href="https://github.com/microsoft/MuseVLA"><img src="https://img.shields.io/badge/Code-GitHub-181717?logo=github&logoColor=white" alt="Code Repository"></a>
  <a href="https://huggingface.co/microsoft/MuseVLA"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue" alt="Hugging Face Model"></a>
  <a href="https://huggingface.co/datasets/microsoft/MuseVLA-dataset"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Dataset-blue" alt="Hugging Face Dataset"></a>
</p>

MuseVLA is an adaptive multimodal sensing Vision-Language-Action (VLA) model
for robotic manipulation. Built on top of
[VITRA](https://github.com/microsoft/VITRA), MuseVLA treats novel sensors as
on-demand tools: it first selects the modality needed for a task, grounds the
selected sensor observation in the RGB image, and then generates robot actions.
The model supports thermal, acoustic, and mmWave radar sensing in addition to
RGB observations. MuseVLA achieves an average success rate of **80.6%** across
thermal-, audio-, and radar-guided manipulation tasks, as well as **66.7%**
average success on unseen sensor-guided tasks.


All our [code](https://github.com/microsoft/MuseVLA) and
[pre-trained model weights](https://huggingface.co/microsoft/MuseVLA) are
licensed under the MIT license.

Please refer to our [paper](https://arxiv.org/abs/2606.17598),
[code repository](https://github.com/microsoft/MuseVLA), and
[dataset](https://huggingface.co/datasets/microsoft/MuseVLA-dataset) for more
details.


## Model Summary

- **Model type:** Vision-Language-Action Model
- **Sensor modalities:** RGB, thermal, acoustic, and mmWave radar
- **Language(s) (NLP):** en
- **License:** MIT
- **Training Dataset:** [MuseVLA-dataset](https://huggingface.co/datasets/microsoft/MuseVLA-dataset)
- **Repository:** [https://github.com/microsoft/MuseVLA](https://github.com/microsoft/MuseVLA)
- **Paper:** [MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulation](https://arxiv.org/abs/2606.17598)

## Citation

```bibtex
@misc{liu2026musevlaadaptivemultimodalsensing,
  title={MuseVLA: An Adaptive Multimodal Sensing Vision-Language-Action Model for Robotic Manipulation},
  author={Xingyuming Liu and Ruichun Ma and Heyu Guo and Qixiu Li and Qingwen Yang and Lin Luo and Shiqi Jiang and Chenren Xu and Jiaolong Yang and Baining Guo},
  year={2026},
  eprint={2606.17598},
  archivePrefix={arXiv},
  primaryClass={cs.RO},
  url={https://arxiv.org/abs/2606.17598},
}
```