Upload folder using huggingface_hub

91e2b20 verified 4 months ago

9.34 kB

---
license: cc-by-nc-sa-4.0
base_model:
- OpenGVLab/InternVL3_5-2B
base_model_relation: finetune
datasets:
- QiZishi/OphReason-Vision
language:
- en
tags:
- ophthalmology
- multimodal
- reasoning
- reinforcement-learning
---
<p align="center">
  <img src="figures/模型图标.png" width="120" alt="OphVLM-R1 project icon">
</p>

<h1 align="center">OphVLM-R1</h1>

<p align="center"><strong>Efficient Ophthalmic Reasoning via Curriculum Reinforcement Learning</strong></p>

<p align="center">
  <a href="https://huggingface.co/QiZishi/OphVLM-R1/blob/main/README.md">English</a> · <a href="https://huggingface.co/QiZishi/OphVLM-R1/blob/main/README_zh.md">简体中文</a>
</p>

<p align="center">
  <a href="https://qizishi.github.io/OphVLM-R1/"><img src="https://img.shields.io/badge/Project-Page-2b6cb0?style=for-the-badge" alt="Project page"></a>
  <a href="https://github.com/QiZishi/OphAgent/"><img src="https://img.shields.io/badge/Code-OphAgent-181717?style=for-the-badge&logo=github" alt="OphAgent code"></a>
</p>
<p align="center">
  <a href="https://huggingface.co/QiZishi/OphVLM-R1"><img src="https://img.shields.io/badge/🤗%20Model-Hugging%20Face-FFD21E?style=for-the-badge" alt="Model on Hugging Face"></a>
  <a href="https://www.modelscope.cn/models/MoonNight/OphVLM-R1"><img src="https://img.shields.io/badge/Model-ModelScope-624AFF?style=for-the-badge" alt="Model on ModelScope"></a>
</p>
<p align="center">
  <a href="https://huggingface.co/datasets/QiZishi/OphReason-Vision"><img src="https://img.shields.io/badge/🤗%20Dataset-Hugging%20Face-FFD21E?style=for-the-badge" alt="Dataset on Hugging Face"></a>
  <a href="https://www.modelscope.cn/datasets/MoonNight/OphReason-Vision"><img src="https://img.shields.io/badge/Dataset-ModelScope-624AFF?style=for-the-badge" alt="Dataset on ModelScope"></a>
</p>

## Overview

**OphVLM-R1** is a lightweight 2B-parameter vision-language model for ophthalmic multimodal reasoning. Starting from InternVL3.5-2B, it first acquires ophthalmic knowledge through LoRA supervised fine-tuning (SFT), then develops progressively harder clinical reasoning skills through four-stage curriculum reinforcement learning. Group Sequence-level Policy Optimization (GSPO) and hard-sample dynamic backtracking are used to improve optimization over long reasoning trajectories and difficult long-tail cases.

This README focuses on the model training pipeline, architecture and algorithms, and experimental results. For dataset construction and distribution details, see the OphReason-Vision dataset repositories below.

## Project Resources

- **Project page**: <https://qizishi.github.io/OphVLM-R1/>
- **OphVLM-R1 model**: [Hugging Face](https://huggingface.co/QiZishi/OphVLM-R1) | [ModelScope](https://www.modelscope.cn/models/MoonNight/OphVLM-R1)
- **OphReason-Vision dataset**: [Hugging Face](https://huggingface.co/datasets/QiZishi/OphReason-Vision) | [ModelScope](https://www.modelscope.cn/datasets/MoonNight/OphReason-Vision)
- **OphAgent system**: [GitHub](https://github.com/QiZishi/OphAgent/)

## Model and Training Framework

OphVLM-R1 uses InternVL3.5-2B as its backbone. Its 2B parameter scale targets deployment in resource-constrained settings while retaining multimodal clinical reasoning capacity. Training consists of two stages.

![Two-stage training pipeline](figures/two_stage_training.png)

### Stage 1: LoRA Supervised Fine-Tuning

The cold-start stage injects ophthalmic domain knowledge with Low-Rank Adaptation:

$$ W_0 + \Delta W = W_0 + BA, \qquad h = W_0x + \frac{\alpha}{r}BAx $$
where \\(B \in \mathbb{R}^{d \times r}\\), \\(A \in \mathbb{R}^{r \times k}\\), and \\(r \ll \min(d,k)\\). The SFT objective is
$$ \mathcal{L}_{\mathrm{SFT}}=-\mathbb{E}\left[\sum_{t=1}^{\lvert y\rvert}\log\pi_\phi(y_t\mid x,y_{\lt t})\right]. $$

| Setting | Value |
|---|---|
| Training subset | 3,418 cold-start samples |
| LoRA rank / scaling | \\(r=64\\), \\(\alpha=128\\) |
| Target projections | \\(W_q\\), \\(W_k\\), \\(W_v\\), \\(W_o\\) |
| Learning rate | \\(1\times10^{-4}\\) with cosine annealing |
| Batch size / epochs | 32 / 3 |
| Trainable parameters | Approximately 0.5% |

### Stage 2: Curriculum Reinforcement Learning

GSPO computes the policy ratio at sequence level rather than independently clipping token-level ratios:

$$ s_i(\phi)=\left(\frac{\pi_\phi(y_i\mid x)}{\pi_{\phi_{\mathrm{old}}}(y_i\mid x)}\right)^{\frac{1}{\lvert y_i\rvert}}=\exp\left(\frac{1}{\lvert y_i\rvert}\sum_{t=1}^{\lvert y_i\rvert}\log\frac{\pi_\phi(y_{i,t}\mid x,y_{i,\lt t})}{\pi_{\phi_{\mathrm{old}}}(y_{i,t}\mid x,y_{i,\lt t})}\right). $$
The PPO-style objective uses group-normalized advantages:
$$ \mathcal{J}_{\mathrm{GSPO}}(\phi)=\mathbb{E}\left[\frac{1}{G}\sum_{i=1}^{G}\min\left(s_i\hat A_i,\operatorname{clip}(s_i,1-\varepsilon,1+\varepsilon)\hat A_i\right)\right]. $$
Each stage uses a mixed reward
$$ r(x,y)=\lambda_1r_{\mathrm{rule}}(x,y)+\lambda_2r_{\mathrm{judge}}(x,y), $$
where the judge reward is produced by Intern-S1-mini. The curriculum follows increasing clinical complexity:
1. **Lesion Localization** — single-image visual perception.
2. **Multi-image Selection** — cross-image comparison.
3. **Report Generation** — structured, long-form synthesis.
4. **Knowledge Q&A** — integration of visual findings and clinical knowledge.
| Setting | Value |
|---|---|
| Group size / clipping | \\(G=8\\), \\(\varepsilon=0.2\\) |
| Learning rate | \\(5\times10^{-6}\\) |
| KL coefficient | \\(\beta_{\mathrm{KL}}=0.04\\) |
| Reward weights | \\(\lambda_1=0.6\\), \\(\lambda_2=0.4\\) |
| Training length | 2 epochs per curriculum stage |

### Hard-Sample Dynamic Backtracking

Prompts that repeatedly receive low rewards during the most recent \\(k=5\\) rounds are resampled on-policy:

$$ P_{\mathrm{sample}}(x,y)\propto f(x,y)^\beta, $$
where \\(f(x,y)\\) is the consecutive failure count and \\(\beta=2.0\\). Resampled prompts always receive fresh rollouts, and the resampling share is capped at 30% of each batch.
### Training Stack
- **Hardware**: 8 NVIDIA GeForce RTX 4090 GPUs (24 GB each).
- **SFT**: 4 GPUs.
- **Reinforcement learning**: 6 GPUs for training and 2 GPUs for vLLM rollout generation.
- **Software**: ms-swift, DeepSpeed ZeRO-3, AdamW, vLLM, and EvalScope.
## Experiments
### Evaluation Benchmarks
| Benchmark | Samples | Tasks / sources | Role |
|---|---:|---:|---|
| In-Domain | 2,000 | 4 tasks | Held-out clinical reasoning |
| Fundus-MMBench | 620 | 31 tasks | Fine-grained fundus analysis |
| OmniMedVQA-Eye | 10,044 | 11 sources | Out-of-domain VQA |
### Main Results
Accuracy is reported in percent. The cross-benchmark average is reference-only because the benchmarks differ in task format, difficulty, and random baseline; per-benchmark comparisons are primary.
| Model | In-Domain | Fundus | Omni-Eye | Avg.* |
|---|---:|---:|---:|---:|
| InternVL3.5-2B | 34.50 | 36.61 | 55.47 | 42.19 |
| InternVL3.5-4B | 36.23 | 42.10 | 77.51 | 51.95 |
| MedVLM-R1-2B | 27.80 | 20.81 | 68.06 | 38.89 |
| Lingshu-7B | **44.20** | 41.29 | 87.42 | **57.64** |
| HuatuoGPT-Vision-7B | 38.30 | 28.06 | 71.78 | 46.05 |
| FundusExpert-8B | 31.20 | **54.84** | 64.71 | 50.25 |
| OphthaReason-Intern-2B | 31.00 | 35.48 | 79.61 | 48.70 |
| OphthaReason-Qwen-3B | 36.60 | 38.87 | 86.86 | 54.11 |
| **OphVLM-R1-2B (ours)** | 38.40 | 42.58 | **88.24** | 56.41 |
OphVLM-R1 reaches 88.24% on OmniMedVQA-Eye and 42.58% on Fundus-MMBench. Its 56.41% reference average is 4.46 percentage points above InternVL3.5-4B and 2.30 points above OphthaReason-Qwen-3B. Comparisons with off-the-shelf 7B/8B models should be interpreted cautiously because data exposure and parameter scale are not controlled.
### Ablation Results
| Configuration | In-Domain | Fundus | Omni-Eye | \\(\Delta\\) Omni |
|---|---:|---:|---:|---:|
| **OphVLM-R1 (full)** | **38.40** | **42.58** | **88.24** | — |
| SFT only | 37.52 | 34.47 | 62.03 | -26.21 |
| SFT + RL one-shot | 37.96 | 38.62 | 78.14 | -10.10 |
| SFT + RL shuffled | 37.73 | 37.85 | 76.48 | -11.76 |
| Without Stage 1 | 38.14 | 40.43 | 85.62 | -2.62 |
| Without Stage 2 | 38.02 | 40.17 | 85.13 | -3.11 |
| Without Stage 3 | 37.88 | 39.72 | 84.38 | -3.86 |
| Without Stage 4 | 38.07 | 40.31 | 85.47 | -2.77 |
| Token-level GRPO | 37.84 | 39.76 | 84.52 | -3.72 |
| Without hard-sample backtracking | 38.11 | 40.83 | 86.12 | -2.12 |
The ablations show that ordered curriculum RL, sequence-level optimization, and hard-sample backtracking make independent contributions. All reported results are single runs without confidence intervals or significance tests. In addition, the theoretical variance-reduction argument for GSPO is exact only under i.i.d. per-token log-ratio assumptions and is approximate for correlated natural-language tokens.
## Citation
```bibtex
@inproceedings{qi2026ophvlm,
  title={OphVLM-R1: Efficient Ophthalmic Reasoning via Curriculum Reinforcement Learning},
  author={Qi, Zishi and Hu, Xiaoya and Pan, Huilin and Gao, Ang and Hou, Jiaxin and Li, Jiankun and Qian, Yongao},
  booktitle={Proceedings of the World Artificial Intelligence Conference Academic (WAICA)},
  year={2026}
}
```
## Acknowledgements
We thank [OpenGVLab](https://github.com/OpenGVLab/InternVL) for the InternVL foundation model and the ophthalmologists who contributed to data review and quality assurance.

	---
	license: cc-by-nc-sa-4.0
	base_model:
	- OpenGVLab/InternVL3_5-2B
	base_model_relation: finetune
	datasets:
	- QiZishi/OphReason-Vision
	language:
	- en
	tags:
	- ophthalmology
	- multimodal
	- reasoning
	- reinforcement-learning
	---

	<p align="center">
	<img src="figures/模型图标.png" width="120" alt="OphVLM-R1 project icon">
	</p>

	<h1 align="center">OphVLM-R1</h1>

	<p align="center"><strong>Efficient Ophthalmic Reasoning via Curriculum Reinforcement Learning</strong></p>

	<p align="center">
	<a href="https://huggingface.co/QiZishi/OphVLM-R1/blob/main/README.md">English</a> · <a href="https://huggingface.co/QiZishi/OphVLM-R1/blob/main/README_zh.md">简体中文</a>
	</p>

	<p align="center">
	<a href="https://qizishi.github.io/OphVLM-R1/"><img src="https://img.shields.io/badge/Project-Page-2b6cb0?style=for-the-badge" alt="Project page"></a>
	<a href="https://github.com/QiZishi/OphAgent/"><img src="https://img.shields.io/badge/Code-OphAgent-181717?style=for-the-badge&logo=github" alt="OphAgent code"></a>
	</p>
	<p align="center">
	<a href="https://huggingface.co/QiZishi/OphVLM-R1"><img src="https://img.shields.io/badge/🤗%20Model-Hugging%20Face-FFD21E?style=for-the-badge" alt="Model on Hugging Face"></a>
	<a href="https://www.modelscope.cn/models/MoonNight/OphVLM-R1"><img src="https://img.shields.io/badge/Model-ModelScope-624AFF?style=for-the-badge" alt="Model on ModelScope"></a>
	</p>
	<p align="center">
	<a href="https://huggingface.co/datasets/QiZishi/OphReason-Vision"><img src="https://img.shields.io/badge/🤗%20Dataset-Hugging%20Face-FFD21E?style=for-the-badge" alt="Dataset on Hugging Face"></a>
	<a href="https://www.modelscope.cn/datasets/MoonNight/OphReason-Vision"><img src="https://img.shields.io/badge/Dataset-ModelScope-624AFF?style=for-the-badge" alt="Dataset on ModelScope"></a>
	</p>

	## Overview

	OphVLM-R1 is a lightweight 2B-parameter vision-language model for ophthalmic multimodal reasoning. Starting from InternVL3.5-2B, it first acquires ophthalmic knowledge through LoRA supervised fine-tuning (SFT), then develops progressively harder clinical reasoning skills through four-stage curriculum reinforcement learning. Group Sequence-level Policy Optimization (GSPO) and hard-sample dynamic backtracking are used to improve optimization over long reasoning trajectories and difficult long-tail cases.

	This README focuses on the model training pipeline, architecture and algorithms, and experimental results. For dataset construction and distribution details, see the OphReason-Vision dataset repositories below.

	## Project Resources

	- Project page: <https://qizishi.github.io/OphVLM-R1/>
	- OphVLM-R1 model: [Hugging Face](https://huggingface.co/QiZishi/OphVLM-R1) \| [ModelScope](https://www.modelscope.cn/models/MoonNight/OphVLM-R1)
	- OphReason-Vision dataset: [Hugging Face](https://huggingface.co/datasets/QiZishi/OphReason-Vision) \| [ModelScope](https://www.modelscope.cn/datasets/MoonNight/OphReason-Vision)
	- OphAgent system: [GitHub](https://github.com/QiZishi/OphAgent/)

	## Model and Training Framework

	OphVLM-R1 uses InternVL3.5-2B as its backbone. Its 2B parameter scale targets deployment in resource-constrained settings while retaining multimodal clinical reasoning capacity. Training consists of two stages.

	![Two-stage training pipeline](figures/two_stage_training.png)

	### Stage 1: LoRA Supervised Fine-Tuning

	The cold-start stage injects ophthalmic domain knowledge with Low-Rank Adaptation:

	$$ W_0 + \Delta W = W_0 + BA, \qquad h = W_0x + \frac{\alpha}{r}BAx $$

	where \\(B \in \mathbb{R}^{d \times r}\\), \\(A \in \mathbb{R}^{r \times k}\\), and \\(r \ll \min(d,k)\\). The SFT objective is

	$$ \mathcal{L}_{\mathrm{SFT}}=-\mathbb{E}\left[\sum_{t=1}^{\lvert y\rvert}\log\pi_\phi(y_t\mid x,y_{\lt t})\right]. $$

	\| Setting \| Value \|
	\|---\|---\|
	\| Training subset \| 3,418 cold-start samples \|
	\| LoRA rank / scaling \| \\(r=64\\), \\(\alpha=128\\) \|
	\| Target projections \| \\(W_q\\), \\(W_k\\), \\(W_v\\), \\(W_o\\) \|
	\| Learning rate \| \\(1\times10^{-4}\\) with cosine annealing \|
	\| Batch size / epochs \| 32 / 3 \|
	\| Trainable parameters \| Approximately 0.5% \|

	### Stage 2: Curriculum Reinforcement Learning

	GSPO computes the policy ratio at sequence level rather than independently clipping token-level ratios:

	$$ s_i(\phi)=\left(\frac{\pi_\phi(y_i\mid x)}{\pi_{\phi_{\mathrm{old}}}(y_i\mid x)}\right)^{\frac{1}{\lvert y_i\rvert}}=\exp\left(\frac{1}{\lvert y_i\rvert}\sum_{t=1}^{\lvert y_i\rvert}\log\frac{\pi_\phi(y_{i,t}\mid x,y_{i,\lt t})}{\pi_{\phi_{\mathrm{old}}}(y_{i,t}\mid x,y_{i,\lt t})}\right). $$

	The PPO-style objective uses group-normalized advantages:

	$$ \mathcal{J}_{\mathrm{GSPO}}(\phi)=\mathbb{E}\left[\frac{1}{G}\sum_{i=1}^{G}\min\left(s_i\hat A_i,\operatorname{clip}(s_i,1-\varepsilon,1+\varepsilon)\hat A_i\right)\right]. $$

	Each stage uses a mixed reward

	$$ r(x,y)=\lambda_1r_{\mathrm{rule}}(x,y)+\lambda_2r_{\mathrm{judge}}(x,y), $$

	where the judge reward is produced by Intern-S1-mini. The curriculum follows increasing clinical complexity:

	1. Lesion Localization — single-image visual perception.
	2. Multi-image Selection — cross-image comparison.
	3. Report Generation — structured, long-form synthesis.
	4. Knowledge Q&A — integration of visual findings and clinical knowledge.

	\| Setting \| Value \|
	\|---\|---\|
	\| Group size / clipping \| \\(G=8\\), \\(\varepsilon=0.2\\) \|
	\| Learning rate \| \\(5\times10^{-6}\\) \|
	\| KL coefficient \| \\(\beta_{\mathrm{KL}}=0.04\\) \|
	\| Reward weights \| \\(\lambda_1=0.6\\), \\(\lambda_2=0.4\\) \|
	\| Training length \| 2 epochs per curriculum stage \|

	### Hard-Sample Dynamic Backtracking

	Prompts that repeatedly receive low rewards during the most recent \\(k=5\\) rounds are resampled on-policy:

	$$ P_{\mathrm{sample}}(x,y)\propto f(x,y)^\beta, $$

	where \\(f(x,y)\\) is the consecutive failure count and \\(\beta=2.0\\). Resampled prompts always receive fresh rollouts, and the resampling share is capped at 30% of each batch.

	### Training Stack

	- Hardware: 8 NVIDIA GeForce RTX 4090 GPUs (24 GB each).
	- SFT: 4 GPUs.
	- Reinforcement learning: 6 GPUs for training and 2 GPUs for vLLM rollout generation.
	- Software: ms-swift, DeepSpeed ZeRO-3, AdamW, vLLM, and EvalScope.

	## Experiments

	### Evaluation Benchmarks

	\| Benchmark \| Samples \| Tasks / sources \| Role \|
	\|---\|---:\|---:\|---\|
	\| In-Domain \| 2,000 \| 4 tasks \| Held-out clinical reasoning \|
	\| Fundus-MMBench \| 620 \| 31 tasks \| Fine-grained fundus analysis \|
	\| OmniMedVQA-Eye \| 10,044 \| 11 sources \| Out-of-domain VQA \|

	### Main Results

	Accuracy is reported in percent. The cross-benchmark average is reference-only because the benchmarks differ in task format, difficulty, and random baseline; per-benchmark comparisons are primary.

	\| Model \| In-Domain \| Fundus \| Omni-Eye \| Avg.* \|
	\|---\|---:\|---:\|---:\|---:\|
	\| InternVL3.5-2B \| 34.50 \| 36.61 \| 55.47 \| 42.19 \|
	\| InternVL3.5-4B \| 36.23 \| 42.10 \| 77.51 \| 51.95 \|
	\| MedVLM-R1-2B \| 27.80 \| 20.81 \| 68.06 \| 38.89 \|
	\| Lingshu-7B \| 44.20 \| 41.29 \| 87.42 \| 57.64 \|
	\| HuatuoGPT-Vision-7B \| 38.30 \| 28.06 \| 71.78 \| 46.05 \|
	\| FundusExpert-8B \| 31.20 \| 54.84 \| 64.71 \| 50.25 \|
	\| OphthaReason-Intern-2B \| 31.00 \| 35.48 \| 79.61 \| 48.70 \|
	\| OphthaReason-Qwen-3B \| 36.60 \| 38.87 \| 86.86 \| 54.11 \|
	\| OphVLM-R1-2B (ours) \| 38.40 \| 42.58 \| 88.24 \| 56.41 \|

	OphVLM-R1 reaches 88.24% on OmniMedVQA-Eye and 42.58% on Fundus-MMBench. Its 56.41% reference average is 4.46 percentage points above InternVL3.5-4B and 2.30 points above OphthaReason-Qwen-3B. Comparisons with off-the-shelf 7B/8B models should be interpreted cautiously because data exposure and parameter scale are not controlled.

	### Ablation Results

	\| Configuration \| In-Domain \| Fundus \| Omni-Eye \| \\(\Delta\\) Omni \|
	\|---\|---:\|---:\|---:\|---:\|
	\| OphVLM-R1 (full) \| 38.40 \| 42.58 \| 88.24 \| — \|
	\| SFT only \| 37.52 \| 34.47 \| 62.03 \| -26.21 \|
	\| SFT + RL one-shot \| 37.96 \| 38.62 \| 78.14 \| -10.10 \|
	\| SFT + RL shuffled \| 37.73 \| 37.85 \| 76.48 \| -11.76 \|
	\| Without Stage 1 \| 38.14 \| 40.43 \| 85.62 \| -2.62 \|
	\| Without Stage 2 \| 38.02 \| 40.17 \| 85.13 \| -3.11 \|
	\| Without Stage 3 \| 37.88 \| 39.72 \| 84.38 \| -3.86 \|
	\| Without Stage 4 \| 38.07 \| 40.31 \| 85.47 \| -2.77 \|
	\| Token-level GRPO \| 37.84 \| 39.76 \| 84.52 \| -3.72 \|
	\| Without hard-sample backtracking \| 38.11 \| 40.83 \| 86.12 \| -2.12 \|

	The ablations show that ordered curriculum RL, sequence-level optimization, and hard-sample backtracking make independent contributions. All reported results are single runs without confidence intervals or significance tests. In addition, the theoretical variance-reduction argument for GSPO is exact only under i.i.d. per-token log-ratio assumptions and is approximate for correlated natural-language tokens.

	## Citation

	```bibtex
	@inproceedings{qi2026ophvlm,
	title={OphVLM-R1: Efficient Ophthalmic Reasoning via Curriculum Reinforcement Learning},
	author={Qi, Zishi and Hu, Xiaoya and Pan, Huilin and Gao, Ang and Hou, Jiaxin and Li, Jiankun and Qian, Yongao},
	booktitle={Proceedings of the World Artificial Intelligence Conference Academic (WAICA)},
	year={2026}
	}
	```

	## Acknowledgements

	We thank [OpenGVLab](https://github.com/OpenGVLab/InternVL) for the InternVL foundation model and the ophthalmologists who contributed to data review and quality assurance.