Abstract
A vision-language model trained on nearly one million clinical neuroimaging studies interprets complete examinations, discusses findings in natural language, and outperforms existing general and medical models in diagnosis, localization, reporting, and progression assessment.
Medical imaging is essential for diagnosing disease, guiding treatment and monitoring patients over time. Although artificial intelligence has advanced image interpretation, existing medical models often address restricted tasks, while general-purpose proprietary models lack access to private clinical imaging required to achieve medical expertise. Developing models that can interpret complex examinations and communicate their findings in the context of clinical decisions remains a major challenge. Here, we show that Neuro-VL, a vision-language model trained on nearly one million neuroimaging studies from routine clinical care, can interpret complete imaging examinations and engage in natural language conversations about their findings. Across internal and international datasets, Neuro-VL outperformed frontier general-purpose and specialized medical models in diagnosing, localizing, and describing abnormalities, as well as generating comprehensive radiology reports. The model also compared serial examinations to assess disease progression and more reliably resisted misleading clinical assumptions than leading frontier models from OpenAI and Google. These results extend health system learning beyond individual predictions to interactive image interpretation and longitudinal disease monitoring. They demonstrate how the collective clinical experience of health systems can be used to develop medical AI with specialized expertise and broad clinical capabilities.
Community
Neuro-VL is a machine intelligence for human neuroimaging designed to reason over images and clinical context and communicate its interpretations through natural-language conversation. We developed Neuro-VL using nearly 1 million neuroimaging studies from 450,000 patients, comprising 10 million volumetric scans. Neuro-VL employs HLIP as the visual encoder that incorporates a hierarchical attention mechanism inspired by the inherent structure of neuroimaging data. This visual encoder was pretrained using radiology reports to extract semantically rich visual representations from complex neuroimaging studies. Building on these representations, Neuro-VL generates robust radiological interpretations accompanied by auditable reasoning traces.
Resources:
- Neuro-VL code: https://github.com/MLNeurosurg/neuro-vl
- Model weights: https://huggingface.co/mlinslab/neuro-vl-8b-thinking
- NeuroBench evaluation benchmark: https://huggingface.co/datasets/mlinslab/neurobench
- NeuroBench evaluation tools with LLM judges: https://github.com/MLNeurosurg/neurobench
Get this paper in your agent:
hf papers read 2611.15094 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper