Reviso / README.md
TengfeiLiuCoder's picture
Add files using upload-large-folder tool
7423a36 verified
|
Raw History Blame Contribute Delete
2.04 kB
---
library_name: transformers
base_model: Qwen/Qwen3.5-9B
tags:
- video-captioning
- agentic
- arxiv:2611.03602
---
# Reviso: Not All Visual Evidence Deserves Equal Attention in Video Captioning
[Paper](https://arxiv.org/abs/2611.03602) | [Code](https://github.com/pkucs-Ltf/Reviso)
![Overview of Reviso: CFP observation planning, tool-assisted captioning, supervised fine-tuning, and GRPO.](assets/framework.png)
Reviso is an agentic video captioning model that learns what matters in each video and actively gathers evidence to describe it. From a global preview, it builds a hierarchical **Context–Focus–Points (CFP)** observation plan, then uses temporal revisiting, image cropping, and OCR to inspect important details. Across multiple rounds, it updates its understanding and plan before generating the final caption.
This release contains the **Checkpoint**: full model weights in FP32, stored in nine `safetensors` shards (approximately 37.64 GB), together with the tokenizer and processor.(Base model :Qwen 3.5 9B)
## Installation
Use Python 3.12. The PyTorch command below uses CUDA 12.6; select another compatible build from the [PyTorch installation instructions](https://pytorch.org/get-started/previous-versions/) if needed.
```bash
pip install torch==2.10.0 torchvision==0.25.0 --index-url https://download.pytorch.org/whl/cu126
pip install "transformers==5.10.4" "accelerate>=1.9,<2" safetensors pillow huggingface_hub
```
## Instructions
Load the complete checkpoint directory with Transformers:
```python
import torch
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration
model_path = "./Reviso" # Use "." when running inside this checkpoint directory.
processor = AutoProcessor.from_pretrained(model_path)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
model_path,
dtype=torch.bfloat16,
device_map="auto",
).eval()
```
The example loads the FP32 weights in BF16 on compatible hardware.
[project repository](https://github.com/pkucs-Ltf/Reviso).