Update README.md
Browse files
README.md
CHANGED
|
@@ -29,8 +29,7 @@ You don't need to tell it which mode to use. Give it the plain question and opti
|
|
| 29 |
All models were evaluated by us under **one shared protocol**:
|
| 30 |
- 64 frames sampled uniformly (first and last frame included)
|
| 31 |
- each frame resized to a fixed 200,704 pixels (factor 28)
|
| 32 |
-
-
|
| 33 |
-
- vLLM
|
| 34 |
|
| 35 |
Each model uses its intended prompt:
|
| 36 |
- **EVAThink**: the bare question and options, with no instruction.
|
|
@@ -95,8 +94,6 @@ vllm serve MaplesWCT/EVAThink-Qwen3 --max-model-len 64000 --limit-mm-per-prompt
|
|
| 95 |
- **Direct mode:** the answer alone, e.g. `B`.
|
| 96 |
- **Thinking mode:** a brief reasoning passage that ties together the relevant frames, followed by the final answer, most often phrased as `Therefore, the correct answer is: **B**` (the letter is usually bolded).
|
| 97 |
|
| 98 |
-
A simple parser covers both: take the letter at the start of the response. If there isn't one, take the last letter that follows "answer is" (ignoring `**` and `:`). If that also fails, fall back to the last standalone option letter in the response.
|
| 99 |
-
|
| 100 |
## Citation
|
| 101 |
|
| 102 |
```bibtex
|
|
@@ -107,7 +104,3 @@ A simple parser covers both: take the letter at the start of the response. If th
|
|
| 107 |
note = {TODO: arXiv link}
|
| 108 |
}
|
| 109 |
```
|
| 110 |
-
|
| 111 |
-
## Acknowledgements
|
| 112 |
-
|
| 113 |
-
Built on [Qwen3-VL](https://github.com/QwenLM/Qwen3-VL).
|
|
|
|
| 29 |
All models were evaluated by us under **one shared protocol**:
|
| 30 |
- 64 frames sampled uniformly (first and last frame included)
|
| 31 |
- each frame resized to a fixed 200,704 pixels (factor 28)
|
| 32 |
+
- temperature=0.0, greedy decoding
|
|
|
|
| 33 |
|
| 34 |
Each model uses its intended prompt:
|
| 35 |
- **EVAThink**: the bare question and options, with no instruction.
|
|
|
|
| 94 |
- **Direct mode:** the answer alone, e.g. `B`.
|
| 95 |
- **Thinking mode:** a brief reasoning passage that ties together the relevant frames, followed by the final answer, most often phrased as `Therefore, the correct answer is: **B**` (the letter is usually bolded).
|
| 96 |
|
|
|
|
|
|
|
| 97 |
## Citation
|
| 98 |
|
| 99 |
```bibtex
|
|
|
|
| 104 |
note = {TODO: arXiv link}
|
| 105 |
}
|
| 106 |
```
|
|
|
|
|
|
|
|
|
|
|
|