zandervdm commited on
Commit
a267957
·
verified ·
1 Parent(s): 64649c8

Document the bundled qevi engine; drop the pip install claim

Browse files
Files changed (1) hide show
  1. README.md +58 -7
README.md CHANGED
@@ -37,7 +37,8 @@ restricted to the allowed answer tokens. If you call `.generate()` and parse the
37
  reproduce them, because that is not the path the model was fine-tuned on.
38
 
39
  The readout is ~30 lines of ordinary transformers code and needs no `trust_remote_code`. A
40
- self-contained version is below; the [`qevi`](#the-qevi-package) package adds question packing.
 
41
 
42
  ---
43
 
@@ -77,7 +78,7 @@ probs = torch.softmax(logits[cand].float() / 2.45, dim=-1) # 2.45 = calibratio
77
  print(dict(zip(answers, probs.tolist()))) # {'Yes': 0.97, 'No': 0.03}
78
  ```
79
 
80
- ### With the `qevi` package (many questions, one forward pass)
81
 
82
  ```python
83
  from qevi import build_readouts, forward_logits, softmax, noul_question, choice_question_generic
@@ -267,14 +268,64 @@ documented above (1 epoch, lr 2e-5, ~5 GPU-hours on two consumer cards).
267
  has not been validated for moderation use and should not be deployed for it without its own
268
  evaluation.
269
 
270
- ## The `qevi` package
271
 
272
- Packing, prompt construction and the readout live in a small Apache-2.0 package
273
- (~350 lines, torch only):
 
274
 
275
- ```bash
276
- pip install qevi
277
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
278
 
279
  ## Citation
280
 
 
37
  reproduce them, because that is not the path the model was fine-tuned on.
38
 
39
  The readout is ~30 lines of ordinary transformers code and needs no `trust_remote_code`. A
40
+ self-contained version is below; the [`qevi` engine bundled in this repo](#the-qevi-engine-bundled-in-this-repo)
41
+ adds question packing.
42
 
43
  ---
44
 
 
78
  print(dict(zip(answers, probs.tolist()))) # {'Yes': 0.97, 'No': 0.03}
79
  ```
80
 
81
+ ### Many questions, one forward pass (using the bundled engine)
82
 
83
  ```python
84
  from qevi import build_readouts, forward_logits, softmax, noul_question, choice_question_generic
 
268
  has not been validated for moderation use and should not be deployed for it without its own
269
  evaluation.
270
 
271
+ ## The `qevi` engine (bundled in this repo)
272
 
273
+ Packing, prompt construction and the readout live in `qevi/` inside **this repository** — three
274
+ files, ~350 lines, torch only, Apache-2.0. Nothing to pip install, and no `trust_remote_code`:
275
+ it is ordinary Python you can read before you run it.
276
 
 
 
277
  ```
278
+ qevi/__init__.py public API
279
+ qevi/core.py prompt templates, build_readouts(), forward_logits()
280
+ qevi/pack.py block-diagonal mask + M-RoPE position construction
281
+ ```
282
+
283
+ ### Using it
284
+
285
+ `snapshot_download` fetches the whole repo (weights and engine together), then put it on the path:
286
+
287
+ ```python
288
+ import sys, torch
289
+ from PIL import Image
290
+ from huggingface_hub import snapshot_download
291
+ from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
292
+
293
+ REPO = "MeerDevelopment/Qevi-2B"
294
+ local = snapshot_download(REPO) # weights + qevi/ in one download
295
+ sys.path.insert(0, local) # makes `import qevi` work
296
+
297
+ from qevi import (build_readouts, forward_logits, softmax,
298
+ noul_question, choice_question_generic)
299
+
300
+ model = Qwen3VLForConditionalGeneration.from_pretrained(local, dtype=torch.bfloat16).cuda().eval()
301
+ processor = AutoProcessor.from_pretrained(local)
302
+ tok = processor.tokenizer
303
+ image = Image.open("photo.jpg").convert("RGB")
304
+
305
+ specs = [
306
+ (noul_question("There is a ladder in this image."), ["Yes", "No"]),
307
+ (noul_question("There is a bicycle in this image."), ["Yes", "No"]),
308
+ (choice_question_generic("Which is shown?",
309
+ ["ladder", "bicycle", "car", "person"]), ["A", "B", "C", "D"]),
310
+ ]
311
+
312
+ readouts = build_readouts(tok, specs)
313
+ logit_sets = forward_logits(model, processor, image, readouts) # ONE forward pass
314
+
315
+ for (question, answers), lg in zip(specs, logit_sets):
316
+ probs = softmax(lg, T=2.45) # see temperatures above
317
+ print(dict(zip(answers, [round(p, 4) for p in probs])))
318
+ ```
319
+
320
+ All three questions share a single encoding of the image and are isolated from one another by the
321
+ block-diagonal mask, so the answers are identical to asking them separately — verified bitwise,
322
+ see [Order invariance](#order-invariance). Asking 30 questions instead of 3 costs about 5 ms more
323
+ per question rather than another full forward pass each.
324
+
325
+ ### If you would rather vendor it
326
+
327
+ The three files have no dependency on this repo's layout. Copy `qevi/` into your own project and
328
+ `import qevi` directly; only `torch`, `transformers` and `pillow` are required.
329
 
330
  ## Citation
331