File size: 4,648 Bytes
bc56e63 2682c6e bc56e63 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 | # How this model works
A short explainer for why Qevi-2B answers questions the way it does. Assumes no background.
For usage, see the [model card](README.md).
> **Where the idea came from.** [TypeSafe](https://typesafe.ai) published the case for models that
> return typed, calibrated values instead of generated text, in their System One model **Jev**.
> Jev works on text; this project asked whether the same trick works on images, and began life
> named JEVI (*Jev for images*). It is an independent implementation, not affiliated with
> TypeSafe and not using their method or code.
## What a normal vision model does
Show a model a photo and ask "is there a ladder?" and it *writes* an answer: it picks the most
likely next word, appends that word to its own input, runs the whole model again for the next
word, and repeats. A thirteen-word reply means thirteen-plus passes through 2.1 billion
parameters, to communicate what is really one bit of information.
Ask ten questions about the same photo and it does that ten times over, re-encoding the photo
from scratch each time. The photo is ~1,000 tokens and the question is maybe 20, so almost all
of that work is repeated for nothing.
## What we do instead
**1. Read the answer, don't generate it.**
Instruction-tuned models follow a rigid script. After your question ends, there is a specific
position where the model is about to write the first word of its reply. We call it the *sentinel*.
```
<|im_start|>user
<|vision_start|>[image tokens]<|vision_end|>
Statement: There is a ladder in this image.
Is this statement true of the image? Answer Yes or No.<|im_end|>
<|im_start|>assistant
^ the sentinel: the model's opinion already exists here
```
At that position the model has already computed a score for every word in its vocabulary. The
score for `Yes` and the score for `No` are sitting right there. We take those two numbers and
stop. No generation, no parsing, no chance of it replying in an unexpected format.
**2. Ask everything at once.**
Because nothing is being generated, we can lay the image down once and append every question
after it, then use the attention mask to enforce three rules:
- every question can see the image
- every question can see itself
- **no question can see any other question**
Each question therefore behaves exactly as if it had been asked alone. We verified this: shuffle
the question order and the outputs are bitwise identical. Packing doesn't quietly change answers.
The payoff is that the expensive part (encoding the image, ~500 ms) happens once, while each
extra question costs about 5 ms. Thirty questions run ~24x faster than asking them one at a time.
## What the fine-tune changed
The base model could already do this — reading logits works on a stock checkpoint. We fine-tuned
all 2.13 billion parameters against exactly that objective: cross-entropy over the candidate
answer logits, nothing else. Train-time and test-time behaviour are identical, which is rarer
than it sounds.
One epoch over 85,500 questions on 28,500 images, ~5 GPU-hours on two consumer cards. Results:
| | Base 2B | Qevi-2B |
|---|---|---|
| Accuracy, domains seen in training | 0.855 | **0.977** |
| Accuracy, 12 domains never trained on | 0.745 | **0.889** |
| Calibration error (held-out, lower better) | 0.160 | **0.054** |
The interesting number is the second row. It improved *more* on domains it had never seen
(+14.4) than on ones it trained on (+12.2), which is the evidence it learned a transferable
skill rather than memorising 28,500 images.
## What this costs you
Every question needs a finite answer set declared up front. The model cannot caption an image,
describe it freely, or answer something you didn't anticipate. Free-form generation still works
(we checked) but comes out about 44% shorter than the base model, having been trained on
one-word answers.
For classification, moderation, triage, inspection and tagging, that trade is usually correct.
For open-ended description, use the base model.
## Honest caveats
- Two of 31 domains got **worse**: German traffic signs (0.834 vs 0.879) and heavily pixelated
car models (0.474 vs 0.501, where both models are near the floor).
- One epoch, one seed, no ablations — there are no error bars on any of these numbers.
- The baseline in every comparison is the base model run through the *same* readout, which
isolates the effect of fine-tuning. It is not a comparison against the base model used
conversationally.
- English prompt templates only.
See the [model card](README.md) for the full evaluation, per-domain results and limitations.
|