Update model card

f4b1928 verified 27 days ago

4.96 kB

---
base_model: OpenGVLab/InternVL3-9B
base_model_relation: finetune
library_name: transformers
license: mit
tags:
  - cvrr
  - custom_code
  - latent-reasoning
  - visual-question-answering
  - arxiv:2609.06746
---
# InternVL3-9B-CVRR

<p align="center">
  <a href="https://arxiv.org/abs/2609.06746"><strong>📄 Paper</strong></a> ·
  <a href="https://github.com/dmis-lab/CVRR"><strong>💻 Code</strong></a>
</p>

## Introduction

Latent visual reasoning is not just about storing visual information in hidden
states. That information should matter for the model's answer. When an answer
decoder can access the original multimodal context through a parallel path,
the presence of informative latent states alone does not establish their use.

Our work, **Reason Through the Latent! Making Latent Visual Reasoning Necessary**,
introduces **Causal Visual Recurrent Reasoning (CVRR)**. CVRR separates visual
access during reasoning from visual access during answering, focusing on three
design principles:

**👁️ Preserving pretrained visual competence**  
Reasoning starts from the native image-conditioned question representation,
preserving visual information already integrated by the pretrained VLM.

**🔁 Refining reasoning with persistent visual evidence**  
A shared native decoder layer updates the question state while retaining access
to fixed visual evidence. The recurrent transition is adapted with LoRA and
answer-token supervision.

**🔒 Making the reasoning state the visual answer interface**  
The upper answer decoder receives the final recurrent question state, without
direct access to the original visual rows or original multimodal prefix caches.
The lower-layer prefix context used for answer generation follows a text-only path.

## Model Checkpoints

### Final Models

We release CVRR checkpoints based on the following pretrained vision-language
backbones. Each repository contains the native backbone weights, the merged
recurrent transition, and the custom inference code; no separate LoRA download
is required.

- 🤗 [Qwen2.5-VL-7B-CVRR](https://huggingface.co/dmis-lab/Qwen2.5-VL-7B-CVRR)
- 🤗 [Qwen2.5-VL-32B-CVRR](https://huggingface.co/dmis-lab/Qwen2.5-VL-32B-CVRR)
- 🤗 [Qwen3-VL-8B-CVRR](https://huggingface.co/dmis-lab/Qwen3-VL-8B-CVRR)
- 🤗 [Gemma3-4B-CVRR](https://huggingface.co/dmis-lab/Gemma3-4B-CVRR)
- 🤗 [Gemma3-12B-CVRR](https://huggingface.co/dmis-lab/Gemma3-12B-CVRR)
- 🤗 [Gemma3-27B-CVRR](https://huggingface.co/dmis-lab/Gemma3-27B-CVRR)
- 🤗 [Gemma4-12B-CVRR](https://huggingface.co/dmis-lab/Gemma4-12B-CVRR)
- 🤗 [Gemma4-31B-CVRR](https://huggingface.co/dmis-lab/Gemma4-31B-CVRR)
- 🤗 [InternVL3-9B-CVRR](https://huggingface.co/dmis-lab/InternVL3-9B-CVRR)
- 🤗 [InternVL3-38B-CVRR](https://huggingface.co/dmis-lab/InternVL3-38B-CVRR)

This checkpoint is based on **`OpenGVLab/InternVL3-9B`**.

## Loading

Install the model-specific dependencies from [requirements.txt](https://huggingface.co/dmis-lab/InternVL3-9B-CVRR/blob/main/requirements.txt), then load directly from the Hub:

```bash
pip install torch==2.9.1 transformers==4.57.6 peft==0.17.1 timm einops accelerate safetensors huggingface_hub Pillow numpy sentencepiece
```

The first `from_pretrained` call downloads and caches the complete model package.

```python
import torch
from PIL import Image
from transformers import AutoModelForImageTextToText
model = AutoModelForImageTextToText.from_pretrained(
    "dmis-lab/InternVL3-9B-CVRR",
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map="cuda:0",
).eval()
image = Image.open("example.jpg").convert("RGB")
inputs = model.prepare_inputs(
    image,
    "What is the dominant color? A. Red B. Blue C. Green D. Yellow. Answer with the option letter.",
)
answer_ids = model.generate(
    **inputs,
    do_sample=False,
    max_new_tokens=64,
    eos_token_id=model.tokenizer.eos_token_id,
)
print(model.tokenizer.decode(answer_ids[0], skip_special_tokens=True))
```

This model supports strict first-answer-token readout and deterministic greedy answer generation. The default recurrent update coefficient is `beta=0.5`.

Use one complete model replica per GPU. This custom Transformers loader does
not support automatic model sharding, generic `save_pretrained()` reserialization,
or direct loading through vLLM/SGLang. Pin the Hub revision for reproducible use.

## License

Please follow the upstream model's terms and the included
[`NOTICE.txt`](NOTICE.txt) and [`licenses/`](licenses/) attribution files.
The component licenses remain applicable to their respective materials.

## Citation

```bibtex
@misc{park2026reasonlatentmakinglatent,
      title={Reason Through the Latent! Making Latent Visual Reasoning Necessary},
      author={Suhyeong Park and Junha Jung and Jaewoo Kang},
      year={2026},
      eprint={2609.06746},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2609.06746},
}
```

	---
	base_model: OpenGVLab/InternVL3-9B
	base_model_relation: finetune
	library_name: transformers
	license: mit
	tags:
	- cvrr
	- custom_code
	- latent-reasoning
	- visual-question-answering
	- arxiv:2609.06746
	---

	# InternVL3-9B-CVRR

	<p align="center">
	<a href="https://arxiv.org/abs/2609.06746"><strong>📄 Paper</strong></a> ·
	<a href="https://github.com/dmis-lab/CVRR"><strong>💻 Code</strong></a>
	</p>

	## Introduction

	Latent visual reasoning is not just about storing visual information in hidden
	states. That information should matter for the model's answer. When an answer
	decoder can access the original multimodal context through a parallel path,
	the presence of informative latent states alone does not establish their use.

	Our work, Reason Through the Latent! Making Latent Visual Reasoning Necessary,
	introduces Causal Visual Recurrent Reasoning (CVRR). CVRR separates visual
	access during reasoning from visual access during answering, focusing on three
	design principles:

	👁️ Preserving pretrained visual competence
	Reasoning starts from the native image-conditioned question representation,
	preserving visual information already integrated by the pretrained VLM.

	🔁 Refining reasoning with persistent visual evidence
	A shared native decoder layer updates the question state while retaining access
	to fixed visual evidence. The recurrent transition is adapted with LoRA and
	answer-token supervision.

	🔒 Making the reasoning state the visual answer interface
	The upper answer decoder receives the final recurrent question state, without
	direct access to the original visual rows or original multimodal prefix caches.
	The lower-layer prefix context used for answer generation follows a text-only path.

	## Model Checkpoints

	### Final Models

	We release CVRR checkpoints based on the following pretrained vision-language
	backbones. Each repository contains the native backbone weights, the merged
	recurrent transition, and the custom inference code; no separate LoRA download
	is required.

	- 🤗 [Qwen2.5-VL-7B-CVRR](https://huggingface.co/dmis-lab/Qwen2.5-VL-7B-CVRR)
	- 🤗 [Qwen2.5-VL-32B-CVRR](https://huggingface.co/dmis-lab/Qwen2.5-VL-32B-CVRR)
	- 🤗 [Qwen3-VL-8B-CVRR](https://huggingface.co/dmis-lab/Qwen3-VL-8B-CVRR)
	- 🤗 [Gemma3-4B-CVRR](https://huggingface.co/dmis-lab/Gemma3-4B-CVRR)
	- 🤗 [Gemma3-12B-CVRR](https://huggingface.co/dmis-lab/Gemma3-12B-CVRR)
	- 🤗 [Gemma3-27B-CVRR](https://huggingface.co/dmis-lab/Gemma3-27B-CVRR)
	- 🤗 [Gemma4-12B-CVRR](https://huggingface.co/dmis-lab/Gemma4-12B-CVRR)
	- 🤗 [Gemma4-31B-CVRR](https://huggingface.co/dmis-lab/Gemma4-31B-CVRR)
	- 🤗 [InternVL3-9B-CVRR](https://huggingface.co/dmis-lab/InternVL3-9B-CVRR)
	- 🤗 [InternVL3-38B-CVRR](https://huggingface.co/dmis-lab/InternVL3-38B-CVRR)

	This checkpoint is based on `OpenGVLab/InternVL3-9B`.

	## Loading

	Install the model-specific dependencies from [requirements.txt](https://huggingface.co/dmis-lab/InternVL3-9B-CVRR/blob/main/requirements.txt), then load directly from the Hub:

	```bash
	pip install torch==2.9.1 transformers==4.57.6 peft==0.17.1 timm einops accelerate safetensors huggingface_hub Pillow numpy sentencepiece
	```

	The first `from_pretrained` call downloads and caches the complete model package.

	```python
	import torch
	from PIL import Image
	from transformers import AutoModelForImageTextToText

	model = AutoModelForImageTextToText.from_pretrained(
	"dmis-lab/InternVL3-9B-CVRR",
	trust_remote_code=True,
	dtype=torch.bfloat16,
	device_map="cuda:0",
	).eval()

	image = Image.open("example.jpg").convert("RGB")
	inputs = model.prepare_inputs(
	image,
	"What is the dominant color? A. Red B. Blue C. Green D. Yellow. Answer with the option letter.",
	)
	answer_ids = model.generate(
	**inputs,
	do_sample=False,
	max_new_tokens=64,
	eos_token_id=model.tokenizer.eos_token_id,
	)
	print(model.tokenizer.decode(answer_ids[0], skip_special_tokens=True))
	```

	This model supports strict first-answer-token readout and deterministic greedy answer generation. The default recurrent update coefficient is `beta=0.5`.

	Use one complete model replica per GPU. This custom Transformers loader does
	not support automatic model sharding, generic `save_pretrained()` reserialization,
	or direct loading through vLLM/SGLang. Pin the Hub revision for reproducible use.

	## License

	Please follow the upstream model's terms and the included
	[`NOTICE.txt`](NOTICE.txt) and [`licenses/`](licenses/) attribution files.
	The component licenses remain applicable to their respective materials.

	## Citation

	```bibtex
	@misc{park2026reasonlatentmakinglatent,
	title={Reason Through the Latent! Making Latent Visual Reasoning Necessary},
	author={Suhyeong Park and Junha Jung and Jaewoo Kang},
	year={2026},
	eprint={2609.06746},
	archivePrefix={arXiv},
	primaryClass={cs.AI},
	url={https://arxiv.org/abs/2609.06746},
	}
	```