--- license: apache-2.0 base_model: Qwen/Qwen3-VL-8B-Instruct pipeline_tag: video-text-to-text library_name: transformers tags: - video-understanding - reinforcement-learning - adaptive-reasoning - qwen3-vl --- # EVAThink-Stage1 **EVAThink-Stage1** is the intermediate checkpoint after the first of the two reinforcement learning stages behind [EVAThink-Qwen3](https://huggingface.co/MaplesWCT/EVAThink-Qwen3). It is fine-tuned from [Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) and already chooses per question whether to answer directly or to write an evidence-grounded reasoning trace first, but it still reasons on most questions. **For general use, prefer [EVAThink-Qwen3](https://huggingface.co/MaplesWCT/EVAThink-Qwen3).** It reaches about the same accuracy with roughly a third fewer output tokens. This checkpoint is released for studying the training stages. ## Highlights - **Accuracy is already close to the final model.** Averaged over six benchmarks it scores 65.4, against 65.6 for EVAThink-Qwen3 and 64.3 for the base model. - **It thinks much more often than it needs to.** It reasons on 77% of Video-MME questions and 83% of LongVideoBench questions (EVAThink-Qwen3: 34% and 46%). Stage 2 mainly teaches the model to skip reasoning when it isn't needed. - **It still adapts to the task.** On purely perceptual spatial questions (VSI-Bench) it almost never reasons (0.2%), and on MVBench it reasons on about a third of questions. ## Results Same protocol as EVAThink-Qwen3, all models evaluated by us: - 64 frames sampled uniformly (first and last frame included) - each frame resized to a fixed 200,704 pixels (factor 28) - greedy decoding with presence penalty 1.5, up to 4,096 new tokens - vLLM EVAThink models get the bare question and options with no instruction. Qwen3-VL-8B-Instruct gets each benchmark's official prompt. | Benchmark (split, #Q) | Qwen3-VL-8B-Instruct | **EVAThink-Stage1** | EVAThink-Qwen3 | Stage1 think-rate | Stage1 mean tokens | |---|---|---|---|---|---| | Video-MME w/o subs (2,700) | 64.9 | 65.4 | 66.3 | 76.7% | 203 | | MVBench (4,000) | 65.8 | 65.2 | 66.5 | 31.9% | 90 | | MMVU, multiple-choice (625) | 70.4 | 73.4 | 72.8 | 96.0% | 413 | | Video-MMMU (900) | 65.6 | 66.7 | 67.4 | 98.3% | 1,038 | | LongVideoBench val (1,337) | 60.7 | 61.3 | 60.7 | 83.2% | 237 | | VSI-Bench (5,130) | 58.3 | 60.3 | 59.9 | 0.2% | 6 | Notes: - These numbers come from our own fixed-budget protocol and are not directly comparable with numbers in other papers. - Mean output length over the six benchmarks is 331 tokens for EVAThink-Stage1 and 213 for EVAThink-Qwen3. - A response counts as "thinking" (think-rate) when it doesn't start directly with the answer. - See the [EVAThink-Qwen3 card](https://huggingface.co/MaplesWCT/EVAThink-Qwen3) for per-benchmark scoring details. ## Usage EVAThink-Stage1 uses the standard Qwen3-VL architecture and chat template, so any Qwen3-VL-compatible stack works. Give it just the question and options. ### Transformers ```python from transformers import AutoProcessor, Qwen3VLForConditionalGeneration model_id = "MaplesWCT/EVAThink-Stage1" model = Qwen3VLForConditionalGeneration.from_pretrained(model_id, dtype="auto", device_map="auto") processor = AutoProcessor.from_pretrained(model_id) question = "What does the person do right after opening the fridge?\nA. Takes out milk\nB. Closes it\nC. Pours water\nD. Leaves the room" messages = [{ "role": "user", "content": [ {"type": "video", "video": "path/to/video.mp4"}, {"type": "text", "text": question}, ], }] inputs = processor.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt", fps=2, ).to(model.device) out = model.generate(**inputs, max_new_tokens=4096, do_sample=False) print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]) ``` ### vLLM ```bash vllm serve MaplesWCT/EVAThink-Stage1 --max-model-len 64000 --limit-mm-per-prompt '{"video": 1}' ``` ### Output format Same as EVAThink-Qwen3: either the answer alone (e.g. `B`), or a short reasoning passage followed by the final answer. Take the letter at the start of the response; if there isn't one, take the last letter that follows "answer is". ## Training Stage 1 starts from Qwen3-VL-8B-Instruct and runs GRPO on [EVAThink-Dataset-Stage1](https://huggingface.co/datasets/MaplesWCT/EVAThink-Dataset-Stage1): 1,766 multiple-choice video questions, each shown in a **dense** view (the evidence fills the clip) and a **sparse** view (the evidence is diluted in a longer video), 32 frames per view. - The reward depends on the view. On the dense view a correct direct answer earns the most; on the sparse view a correct reasoned answer does. Malformed responses are penalized. - A regularizer on the first response token keeps the model from collapsing onto a single mode. - Both views of a question go into the same update: 32 prompts (16 questions × 2 views) × 8 rollouts per step. - One epoch (110 steps), learning rate 1e-6, no KL penalty. EVAThink-Qwen3 continues from this checkpoint with a stage-2 reward that no longer looks at the view. ## Limitations - It reasons on most questions, so it is slower than EVAThink-Qwen3 at about the same accuracy. - It was evaluated on multiple-choice video QA, plus numerical estimation in VSI-Bench. Behavior on open-ended tasks has not been studied. - Evaluation uses a fixed 64-frame budget. Results under other frame budgets may differ. ## Citation ```bibtex @misc{evathink2026, title = {TODO}, author = {TODO}, year = {2026}, note = {TODO: arXiv link} } ``` ## Acknowledgements Built on [Qwen3-VL](https://github.com/QwenLM/Qwen3-VL).