Video-Text-to-Text
Transformers
Safetensors
qwen3_vl
image-text-to-text
video-understanding
reinforcement-learning
adaptive-reasoning
qwen3-vl
Instructions to use MaplesWCT/EVAThink with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MaplesWCT/EVAThink with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("MaplesWCT/EVAThink") model = AutoModelForMultimodalLM.from_pretrained("MaplesWCT/EVAThink", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -14,7 +14,7 @@ tags:
|
|
| 14 |
|
| 15 |
<!-- TODO(author): one-line expansion of the name, paper title / arXiv link, code repo link -->
|
| 16 |
|
| 17 |
-
**EVAThink-Qwen3** is a
|
| 18 |
|
| 19 |
You don't need to tell it which mode to use. Give it the plain question and options, and it picks the mode itself.
|
| 20 |
|
|
@@ -43,7 +43,7 @@ Each model uses its intended prompt:
|
|
| 43 |
| MVBench (4,000) | 65.8 | **69.0** | 66.5 | 5.1% |
|
| 44 |
| MMVU, multiple-choice (625) | 70.4 | 69.6 | **72.8** | 59.4% |
|
| 45 |
| Video-MMMU (900) | 65.6 | 64.3 | **67.4** | 85.7% |
|
| 46 |
-
| LongVideoBench val (1,337) |
|
| 47 |
| VSI-Bench (5,130) | 58.3 | 54.6 | **59.9** | 0.0% |
|
| 48 |
|
| 49 |
Notes:
|
|
@@ -97,21 +97,6 @@ vllm serve MaplesWCT/EVAThink-Qwen3 --max-model-len 64000 --limit-mm-per-prompt
|
|
| 97 |
|
| 98 |
A simple parser covers both: take the letter at the start of the response. If there isn't one, take the last letter that follows "answer is" (ignoring `**` and `:`). If that also fails, fall back to the last standalone option letter in the response.
|
| 99 |
|
| 100 |
-
## Training
|
| 101 |
-
|
| 102 |
-
<!-- TODO(author): fill in data sources, sizes, and the exact recipe. The outline below reflects our internal setup; please check it before release. -->
|
| 103 |
-
|
| 104 |
-
EVAThink starts from Qwen3-VL-8B-Instruct and is trained in two RL (GRPO) stages on multiple-choice video QA with verifiable answers:
|
| 105 |
-
|
| 106 |
-
1. **Stage 1: learning when to think.** Training uses paired samples of questions whose evidence is dense vs. sparse, together with a regularizer on the first token that shapes the choice between answering directly and reasoning.
|
| 107 |
-
2. **Stage 2: calibrating the mode choice.** Training continues on held-out questions with a reward that slightly favors correct direct answers over correct reasoned answers (1.1 vs. 1.0; wrong answers get 0), which discourages thinking when it isn't needed.
|
| 108 |
-
|
| 109 |
-
## Limitations
|
| 110 |
-
|
| 111 |
-
- Because the model mostly thinks on the questions it's unsure of, questions it thinks on have lower accuracy than questions it answers directly. That is a selection effect: it doesn't mean thinking hurts.
|
| 112 |
-
- The model was evaluated on multiple-choice video QA, plus numerical estimation in VSI-Bench. Behavior on open-ended captioning or grounding tasks has not been studied.
|
| 113 |
-
- Evaluation uses a fixed 64-frame budget. Results under other frame budgets may differ.
|
| 114 |
-
|
| 115 |
## Citation
|
| 116 |
|
| 117 |
```bibtex
|
|
@@ -125,4 +110,4 @@ EVAThink starts from Qwen3-VL-8B-Instruct and is trained in two RL (GRPO) stages
|
|
| 125 |
|
| 126 |
## Acknowledgements
|
| 127 |
|
| 128 |
-
Built on [Qwen3-VL](https://github.com/QwenLM/Qwen3-VL).
|
|
|
|
| 14 |
|
| 15 |
<!-- TODO(author): one-line expansion of the name, paper title / arXiv link, code repo link -->
|
| 16 |
|
| 17 |
+
**EVAThink-Qwen3** is a vision-language model fine-tuned from [Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) with reinforcement learning to **decide per question whether to think**. When the visual evidence plainly supports an answer, it replies directly. When the evidence is sparse, discontinuous, or scattered across frames, it writes a short, evidence-grounded reasoning trace first.
|
| 18 |
|
| 19 |
You don't need to tell it which mode to use. Give it the plain question and options, and it picks the mode itself.
|
| 20 |
|
|
|
|
| 43 |
| MVBench (4,000) | 65.8 | **69.0** | 66.5 | 5.1% |
|
| 44 |
| MMVU, multiple-choice (625) | 70.4 | 69.6 | **72.8** | 59.4% |
|
| 45 |
| Video-MMMU (900) | 65.6 | 64.3 | **67.4** | 85.7% |
|
| 46 |
+
| LongVideoBench val (1,337) | 60.4 | 54.6 | **60.7** | 46.1% |
|
| 47 |
| VSI-Bench (5,130) | 58.3 | 54.6 | **59.9** | 0.0% |
|
| 48 |
|
| 49 |
Notes:
|
|
|
|
| 97 |
|
| 98 |
A simple parser covers both: take the letter at the start of the response. If there isn't one, take the last letter that follows "answer is" (ignoring `**` and `:`). If that also fails, fall back to the last standalone option letter in the response.
|
| 99 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 100 |
## Citation
|
| 101 |
|
| 102 |
```bibtex
|
|
|
|
| 110 |
|
| 111 |
## Acknowledgements
|
| 112 |
|
| 113 |
+
Built on [Qwen3-VL](https://github.com/QwenLM/Qwen3-VL).
|