RAVEC / README.md

Release verified RAVEC-7B M1 step100 Actor (Table 2 Avg 61.00)

bc9139e verified 2 days ago

2.82 kB

---
library_name: transformers
pipeline_tag: image-text-to-text
base_model:
- Qwen/Qwen2.5-VL-3B-Instruct
- Qwen/Qwen2.5-VL-7B-Instruct
tags:
- ravec
- qwen2_5_vl
- multimodal
- reinforcement-learning
license: other
license_name: qwen-research
license_link: https://huggingface.co/Jack04810/RAVEC/blob/main/RAVEC-3B/LICENSE
---
# RAVEC: main-result checkpoints

This repository releases the verified 3B and 7B checkpoints associated with Table 2 of RAVEC.

| Directory | Base model | Checkpoint | Availability |
|---|---|---|---|
| `RAVEC-3B` | Qwen2.5-VL-3B-Instruct | F0, step 175 | Uploaded |
| `RAVEC-7B` | Qwen2.5-VL-7B-Instruct | M1, step 100 Actor | Uploaded |

The 7B release is the exact M1 step100 Actor export used by all fourteen recorded evaluation runs. All 15 original files were verified against the evaluation-time SHA-256 hashes; see `RAVEC-7B/checkpoint_manifest.json`.
The current paper and table-generation metadata identify the 3B step175 export as Actor/student.
Older records use `EMA_G3R` / `EMA` labels. The released 3B files match the original
checkpoint manifest byte-for-byte; `RAVEC-3B/checkpoint_manifest.json` records their SHA-256 hashes.

## Reported results

Avg@8 is mean accuracy over eight generations per question. Avg is the equal-weight
mean of the ten listed benchmarks. Scores are percentages.

| Model | Geometry3K | MathVista | We-Math | MathVerse | LogicVista | MMMU-Pro | MathVerse-V | C-VQA-Real | MARS-Bench | POPE | Avg |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| RAVEC-3B | 33.28 | 59.78 | 61.68 | 57.43 | 41.67 | 28.38 | 54.44 | 66.54 | 53.05 | 87.98 | 54.42 |
| RAVEC-7B | 43.20 | 66.44 | 70.73 | 67.53 | 46.28 | 36.09 | 65.03 | 70.69 | 55.99 | 87.99 | 61.00 |


See `evaluation_results.json` for available score precision, checkpoint selection, and source hashes.

## Loading the 3B checkpoint

```python
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
repo = "Jack04810/RAVEC"
processor = AutoProcessor.from_pretrained(repo, subfolder="RAVEC-3B")
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    repo, subfolder="RAVEC-3B", torch_dtype="auto", device_map="auto"
)
```

The export records Transformers 4.51.3. Use a Transformers version supporting Qwen2.5-VL.

## Loading the 7B checkpoint

Use the same loading example with `subfolder="RAVEC-7B"` for both the processor and model. See [RAVEC-7B/README.md](RAVEC-7B/README.md) for details and the original evaluation summary.

## License

The 3B checkpoint is derived from Qwen2.5-VL-3B-Instruct and retains the
Qwen Research License; see `RAVEC-3B/LICENSE` and `RAVEC-3B/NOTICE`.

The 7B checkpoint is derived from Qwen2.5-VL-7B-Instruct under Apache-2.0; see `RAVEC-7B/LICENSE`. The two model directories retain their respective base-model licenses.

	---
	library_name: transformers
	pipeline_tag: image-text-to-text
	base_model:
	- Qwen/Qwen2.5-VL-3B-Instruct
	- Qwen/Qwen2.5-VL-7B-Instruct
	tags:
	- ravec
	- qwen2_5_vl
	- multimodal
	- reinforcement-learning
	license: other
	license_name: qwen-research
	license_link: https://huggingface.co/Jack04810/RAVEC/blob/main/RAVEC-3B/LICENSE
	---

	# RAVEC: main-result checkpoints

	This repository releases the verified 3B and 7B checkpoints associated with Table 2 of RAVEC.

	\| Directory \| Base model \| Checkpoint \| Availability \|
	\|---\|---\|---\|---\|
	\| `RAVEC-3B` \| Qwen2.5-VL-3B-Instruct \| F0, step 175 \| Uploaded \|
	\| `RAVEC-7B` \| Qwen2.5-VL-7B-Instruct \| M1, step 100 Actor \| Uploaded \|

	The 7B release is the exact M1 step100 Actor export used by all fourteen recorded evaluation runs. All 15 original files were verified against the evaluation-time SHA-256 hashes; see `RAVEC-7B/checkpoint_manifest.json`.
	The current paper and table-generation metadata identify the 3B step175 export as Actor/student.
	Older records use `EMA_G3R` / `EMA` labels. The released 3B files match the original
	checkpoint manifest byte-for-byte; `RAVEC-3B/checkpoint_manifest.json` records their SHA-256 hashes.

	## Reported results

	Avg@8 is mean accuracy over eight generations per question. Avg is the equal-weight
	mean of the ten listed benchmarks. Scores are percentages.

	\| Model \| Geometry3K \| MathVista \| We-Math \| MathVerse \| LogicVista \| MMMU-Pro \| MathVerse-V \| C-VQA-Real \| MARS-Bench \| POPE \| Avg \|
	\|---\|---:\|---:\|---:\|---:\|---:\|---:\|---:\|---:\|---:\|---:\|---:\|
	\| RAVEC-3B \| 33.28 \| 59.78 \| 61.68 \| 57.43 \| 41.67 \| 28.38 \| 54.44 \| 66.54 \| 53.05 \| 87.98 \| 54.42 \|
	\| RAVEC-7B \| 43.20 \| 66.44 \| 70.73 \| 67.53 \| 46.28 \| 36.09 \| 65.03 \| 70.69 \| 55.99 \| 87.99 \| 61.00 \|


	See `evaluation_results.json` for available score precision, checkpoint selection, and source hashes.

	## Loading the 3B checkpoint

	```python
	from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration

	repo = "Jack04810/RAVEC"
	processor = AutoProcessor.from_pretrained(repo, subfolder="RAVEC-3B")
	model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
	repo, subfolder="RAVEC-3B", torch_dtype="auto", device_map="auto"
	)
	```

	The export records Transformers 4.51.3. Use a Transformers version supporting Qwen2.5-VL.

	## Loading the 7B checkpoint

	Use the same loading example with `subfolder="RAVEC-7B"` for both the processor and model. See [RAVEC-7B/README.md](RAVEC-7B/README.md) for details and the original evaluation summary.

	## License

	The 3B checkpoint is derived from Qwen2.5-VL-3B-Instruct and retains the
	Qwen Research License; see `RAVEC-3B/LICENSE` and `RAVEC-3B/NOTICE`.

	The 7B checkpoint is derived from Qwen2.5-VL-7B-Instruct under Apache-2.0; see `RAVEC-7B/LICENSE`. The two model directories retain their respective base-model licenses.