Blame for README.md

---
license: apache-2.0
base_model: Qwen/Qwen3-VL-8B-Instruct
pipeline_tag: video-text-to-text
library_name: transformers
tags:
  - video-understanding
  - reinforcement-learning
  - adaptive-reasoning
  - qwen3-vl
---

# EVAThink-Stage1

<!-- TODO(author): one-line expansion of the name, paper title / arXiv link, code repo link -->

**EVAThink-Stage1** is the intermediate checkpoint after the first of the two reinforcement learning stages behind
[EVAThink-Qwen3](https://huggingface.co/MaplesWCT/EVAThink-Qwen3). It is fine-tuned from
[Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) and already chooses per question whether to answer
directly or to write an evidence-grounded reasoning trace first, but it still reasons on most questions.

**For general use, prefer [EVAThink-Qwen3](https://huggingface.co/MaplesWCT/EVAThink-Qwen3).** It reaches about the same
accuracy with roughly a third fewer output tokens. This checkpoint is released for studying the training stages.

## Highlights

- **Accuracy is already close to the final model.** Averaged over six benchmarks it scores 65.4, against 65.6 for
  EVAThink-Qwen3 and 64.3 for the base model.
- **It thinks much more often than it needs to.** It reasons on 77% of Video-MME questions and 83% of LongVideoBench
  questions (EVAThink-Qwen3: 34% and 46%). Stage 2 mainly teaches the model to skip reasoning when it isn't needed.
- **It still adapts to the task.** On purely perceptual spatial questions (VSI-Bench) it almost never reasons (0.2%), and on
  MVBench it reasons on about a third of questions.

## Results

Same protocol as EVAThink-Qwen3, all models evaluated by us:
- 64 frames sampled uniformly (first and last frame included)
- each frame resized to a fixed 200,704 pixels (factor 28)
- greedy decoding with presence penalty 1.5, up to 4,096 new tokens
- vLLM

EVAThink models get the bare question and options with no instruction. Qwen3-VL-8B-Instruct gets each benchmark's official prompt.

| Benchmark (split, #Q) | Qwen3-VL-8B-Instruct | **EVAThink-Stage1** | EVAThink-Qwen3 | Stage1 think-rate | Stage1 mean tokens |
|---|---|---|---|---|---|
| Video-MME w/o subs (2,700) | 64.9 | 65.4 | 66.3 | 76.7% | 203 |
| MVBench (4,000) | 65.8 | 65.2 | 66.5 | 31.9% | 90 |
| MMVU, multiple-choice (625) | 70.4 | 73.4 | 72.8 | 96.0% | 413 |
| Video-MMMU (900) | 65.6 | 66.7 | 67.4 | 98.3% | 1,038 |
| LongVideoBench val (1,337) | 60.7 | 61.3 | 60.7 | 83.2% | 237 |
| VSI-Bench (5,130) | 58.3 | 60.3 | 59.9 | 0.2% | 6 |

Notes:
- These numbers come from our own fixed-budget protocol and are not directly comparable with numbers in other papers.
- Mean output length over the six benchmarks is 331 tokens for EVAThink-Stage1 and 213 for EVAThink-Qwen3.
- A response counts as "thinking" (think-rate) when it doesn't start directly with the answer.
- See the [EVAThink-Qwen3 card](https://huggingface.co/MaplesWCT/EVAThink-Qwen3) for per-benchmark scoring details.

## Usage

EVAThink-Stage1 uses the standard Qwen3-VL architecture and chat template, so any Qwen3-VL-compatible stack works.
Give it just the question and options.

### Transformers

```python
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

model_id = "MaplesWCT/EVAThink-Stage1"
model = Qwen3VLForConditionalGeneration.from_pretrained(model_id, dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained(model_id)

question = "What does the person do right after opening the fridge?\nA. Takes out milk\nB. Closes it\nC. Pours water\nD. Leaves the room"
messages = [{
    "role": "user",
    "content": [
        {"type": "video", "video": "path/to/video.mp4"},
        {"type": "text", "text": question},
    ],
}]
inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt",
    fps=2,
).to(model.device)
out = model.generate(**inputs, max_new_tokens=4096, do_sample=False)
print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])
```

### vLLM

```bash
vllm serve MaplesWCT/EVAThink-Stage1 --max-model-len 64000 --limit-mm-per-prompt '{"video": 1}'
```

### Output format

Same as EVAThink-Qwen3: either the answer alone (e.g. `B`), or a short reasoning passage followed by the final answer.
Take the letter at the start of the response; if there isn't one, take the last letter that follows "answer is".

## Training

<!-- TODO(author): check this outline before release. -->

Stage 1 starts from Qwen3-VL-8B-Instruct and runs GRPO on
[EVAThink-Dataset-Stage1](https://huggingface.co/datasets/MaplesWCT/EVAThink-Dataset-Stage1): 1,766 multiple-choice video
questions, each shown in a **dense** view (the evidence fills the clip) and a **sparse** view (the evidence is diluted in a
longer video), 32 frames per view.

- The reward depends on the view. On the dense view a correct direct answer earns the most; on the sparse view a correct
  reasoned answer does. Malformed responses are penalized.
- A regularizer on the first response token keeps the model from collapsing onto a single mode.
- Both views of a question go into the same update: 32 prompts (16 questions × 2 views) × 8 rollouts per step.
- One epoch (110 steps), learning rate 1e-6, no KL penalty.

EVAThink-Qwen3 continues from this checkpoint with a stage-2 reward that no longer looks at the view.

## Limitations

- It reasons on most questions, so it is slower than EVAThink-Qwen3 at about the same accuracy.
- It was evaluated on multiple-choice video QA, plus numerical estimation in VSI-Bench. Behavior on open-ended tasks has not been studied.
- Evaluation uses a fixed 64-frame budget. Results under other frame budgets may differ.

## Citation

```bibtex
@misc{evathink2026,
  title  = {TODO},
  author = {TODO},
  year   = {2026},
  note   = {TODO: arXiv link}
}
```

## Acknowledgements

Built on [Qwen3-VL](https://github.com/QwenLM/Qwen3-VL).