Card: new positioning, image example, trust section

README.md

@@ -6,7 +6,7 @@ library_name: peft
 pipeline_tag: image-text-to-text
 language: [en, zh]
 datasets: [CountingSheep/vev-anchor-v3]
-tags: [judgment, classification, calibration, vision-language, system-one, vev]
+tags: [decision-model, judgment, classification, vision-language, jev, systemone, vev]
 model-index:
   - name: vev-9b-lora
     results:
@@ -35,28 +35,39 @@ model-index:
 
 # vev-9b-lora
 
-Vev is a judgment model with open weights. Given a state (text, JSON, images, or a mix) and typed questions
-(yes/no, multiple choice, graded score), it returns a probability for every allowed answer in one forward pass,
-without generating text. `vev-9b-lora` is the LoRA adapter of
-[CountingSheep/vev-9b](https://huggingface.co/CountingSheep/vev-9b), a fine-tune of
-[Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B). This repository holds the adapter only; `vev serve`
-downloads the base model and applies the adapter at load time. The `vev` server exposes it through an HTTP API with
-the same request and response format as TypeSafe's `/v1/systemone`.
-
-This is v0.1, a research preview, released for non-commercial use under CC BY-NC 4.0. Code:
-[Xiaooolong/vev](https://github.com/Xiaooolong/vev). All models of this release:
-[vev-4b](https://huggingface.co/CountingSheep/vev-4b) ·
-[vev-4b-lora](https://huggingface.co/CountingSheep/vev-4b-lora) ·
-[vev-9b](https://huggingface.co/CountingSheep/vev-9b) ·
-[vev-9b-lora](https://huggingface.co/CountingSheep/vev-9b-lora); training anchor data:
-[CountingSheep/vev-anchor-v3](https://huggingface.co/datasets/CountingSheep/vev-anchor-v3);
-[collection](https://huggingface.co/collections/CountingSheep/vev-v01-6abd3e17303828e10c49dc65).
-
-## Usage
+**Jev-like decision models that can also see images.** Ask yes/no, multiple-choice or graded questions about a
+piece of text, a JSON record or a screenshot, and get a probability for every allowed answer in one forward pass.
+Vev serves the same request format as TypeSafe's
+[Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev) (`/v1/systemone`), and runs on your own GPU.
+Code, API and full results: [Xiaooolong/vev](https://github.com/Xiaooolong/vev).
+
+This repository holds the LoRA adapter of [CountingSheep/vev-9b](https://huggingface.co/CountingSheep/vev-9b) (170
+MB). `vev serve` applies it to [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) at load time, so it is the
+smaller download if you already have the base model.
+
+<img src="https://raw.githubusercontent.com/Xiaooolong/vev/main/docs/checkout.png" width="520" alt="A checkout page with a red banner: Payment failed: your card was declined.">
+
+```python
+questions = {
+    "error": Noul(instructions="Does the screen show an error message?"),
+    "step": Choice(instructions="Which checkout step is the user on?",
+                   criteria={"shipping": None, "payment": None, "review": None}),
+    "next": Choice(instructions="What should the user do next?",
+                   criteria={"retry": "Try another card", "wait": "Wait for the order to ship",
+                             "nothing": "Nothing, the order went through"}),
+}
+# vev-9b-lora:  error 0.984
+#  step  {"shipping": 0.002, "payment": 0.974, "review": 0.024}
+#  next  {"retry": 0.992, "wait": 0.001, "nothing": 0.006}
+```
+
+v0.1 is a research preview, released for non-commercial use under CC BY-NC 4.0.
+
+## Use it
 
 ```bash
 pip install git+https://github.com/Xiaooolong/vev typesafe-sdk
-vev serve --model CountingSheep/vev-9b-lora
+vev serve --model CountingSheep/vev-9b-lora          # add --revision v0.1.0 to pin this release
 ```
 
 ```python
@@ -65,27 +76,48 @@ from typesafe_sdk import Choice, Noul, TypeSafeClient
 client = TypeSafeClient(api_key="local", base_url="http://127.0.0.1:8009")
 resp = client.system_one(
     state="Order #4411 still shows 'label created' after 9 days. I need it before Friday.",
-    questions={
-        "urgent": Noul(instructions="Is the customer asking for something time-sensitive?"),
-        "team": Choice(instructions="Which team should handle this?",
-                       criteria={"shipping": "Delivery and tracking", "billing": "Charges and refunds"}),
-    },
+    questions={"urgent": Noul(instructions="Is the customer asking for something time-sensitive?")},
 )
-print(resp.answers["urgent"].noul, resp.answers["team"].probabilities)
+print(resp.answers["urgent"].noul)
 ```
 
-`Noul` is the API's name for a yes/no question; `.noul` is the probability of "yes". Images go into the state as
-`{"image": {"url": "data:image/png;base64,..."}}`. The API is specified in
-[spec/systemone-api.md](https://github.com/Xiaooolong/vev/blob/main/spec/systemone-api.md).
+`Noul` is the API's name for a yes/no question. Images go into the state as `{"image": {"url":
+"data:image/png;base64,..."}}`. Needs an NVIDIA GPU with about 19 GB of free memory.
 
-To pin this version, download the tagged revision and serve the local copy:
+## What it can do
 
-```bash
-hf download CountingSheep/vev-9b-lora --revision v0.1.0 --local-dir vev-9b-lora-v0.1.0
-vev serve --model vev-9b-lora-v0.1.0
-```
+Accuracy on human-labelled data, by the kind of question:
+
+| Kind of question | Example | vev-9b-lora |
+|---|---|---|
+| Jev-style decisions on text | JevBench public subset | 0.823 |
+| The same, on sources the model has not seen | kev transfer-v4 | 0.781 |
+| Chinese | judgekit | 0.938 |
+| Pairs where a small change flips the answer | nimble | 0.747 |
+| UI state in an app screenshot | "Is there a switch or checkbox that is turned on?" / "Is there a text input field?" | 0.923 / 0.955 |
+| An image against a written safety policy | the LlavaGuard policy categories | 0.724 |
+| A generated image against its prompt | "Does the image show the element 'grass' as the prompt describes?" | 0.703 |
+| General questions about a photo | MMBench-EN / POPE / MMStar | 0.902 / 0.897 / 0.674 |
+| Bugs in game screenshots | glitch / object clipping (VideoGameQA-Bench) | 0.626 / 0.729 |
+
+The text rows are public benchmarks; the image rows are judgment sets converted from labelled public data. Check it
+on a labelled sample of your own questions before relying on it. Significance tests and the comparison with the
+base model are in the [GitHub README](https://github.com/Xiaooolong/vev#detailed-results).
 
-### Loading without the server
+## How far to trust it
+
+- Jev is more accurate on nimble (by 18 points) and kev transfer-v4 (7); JevBench and judgekit are not
+  significantly different.
+- The probabilities rank answers well but are not exact frequencies: expected calibration error is 0.01–0.14
+  depending on the task. Choose any threshold on your own labelled data.
+- Reversing the order of the options changes the top answer on 13% of JevBench and kev transfer-v4 questions (Jev:
+  under 4%).
+- Asked together with other questions, a question's probabilities move by up to a few hundredths (p99 0.025).
+- Judgments that need several steps of reasoning are weaker than the base model's own answer when it may think
+  first. On "is something wrong here?" questions the model answers "no" more often than the labels do.
+- Vev is not affiliated with TypeSafe; it matches Jev's API, not its behaviour.
+
+## Load the weights directly
 
 The adapter was trained on the inner model (`full.model`), and its files are in `adapter/`:
 
@@ -100,79 +132,34 @@ adapter = os.path.join(snapshot_download("CountingSheep/vev-9b-lora"), "adapter"
 full.model = PeftModel.from_pretrained(full.model, adapter).merge_and_unload()
 ```
 
-This gives the model only. Vev reads its answers from the next-token probabilities of the answer tokens (Yes/No,
-option letters, level digits) at the end of a fixed prompt; the prompt template and the readout are described in
-[spec/systemone-api.md, section
+This gives the model only. Vev reads the next-token probabilities of the answer tokens (Yes/No, option letters,
+level digits) at the end of a fixed prompt, described in [spec/systemone-api.md, section
 10](https://github.com/Xiaooolong/vev/blob/main/spec/systemone-api.md#10-how-answers-are-computed).
 
-## Evaluation
-
-Accuracy on full sets with the harness in the code repository. The zero-shot column is the base model read the same
-way without fine-tuning. [Jev](https://typesafe.ai/blog/introducing-system-one-models-and-jev) is TypeSafe's hosted
-judgment model, queried through its API (`jev-latest`, 23–24 September 2026); no Jev outputs were used for
-training. [kev-4B](https://github.com/jaredpalmer/kev) is an open judgment model on a 4B Qwen base, run locally
-with the same harness. The text sets are [judgekit](https://github.com/lexingtonhibiki/judgekit) (Chinese),
-[JevBench](https://github.com/fstandhartinger/jevbench) (public subset),
-[nimble](https://github.com/bespokelabsai/nimble) and kev's own held-out test set, transfer-v4.
-
-| Set | n | Qwen3.5-9B zero-shot | vev-9b-lora | kev-4B | Jev |
-|---|---|---|---|---|---|
-| judgekit (Chinese) | 130 | 0.938 | 0.938 | 0.846 | 0.962 |
-| JevBench (public subset) | 231 | 0.805 | 0.823 | 0.719 | 0.861 |
-| nimble | 324 | 0.710 | 0.747 | 0.735 | 0.923 |
-| kev transfer-v4 | 1528 | 0.764 | 0.781 | 0.802 | 0.854 |
-
-| Set | n | Qwen3.5-9B zero-shot | vev-9b-lora |
-|---|---|---|---|
-| MMBench-EN | 1164 | 0.887 | 0.902 |
-| POPE | 9000 | 0.894 | 0.897 |
-| MMStar | 1498 | 0.608 | 0.674 |
-| policy_mod: violates this safety policy? (LlavaGuard) | 659 | 0.666 | 0.724 |
-| game_glitch: screenshot contains a glitch? (VideoGameQA-Bench) | 1000 | 0.577 | 0.626 |
-| game_clip: object clips into character? (VideoGameQA-Bench) | 686 | 0.736 | 0.729 |
-| ui_toggle: switch on / off? (MobileViews) | 800 | 0.921 | 0.923 |
-| ui_input: text input field present? (MobileViews) | 800 | 0.948 | 0.955 |
-| t2i_elem: generated image shows the prompt element? (EvalMuse) | 1027 | 0.733 | 0.703 |
-
-The last six rows are judgment sets we converted from human-labelled public data; they are not established
-benchmarks. Brier score and calibration error for every set are in `results/` in the code repository; significance
-tests against the base model, kev-4B and Jev are in its README.
-
-Reversing the order of the options changes the top answer on 13% of JevBench and kev transfer-v4 questions (kev-4B:
-10–12%, Jev: under 4%).
-
-## Limitations
-
-- Jev is significantly more accurate on nimble and kev transfer-v4 (by 18 and 7 points).
-- Answers depend on option order more than kev-4B's and Jev's do (rates above).
-- Judgments that need several reasoning steps are weaker than the base model's own answer when it may think first
-  (4–8 points on one internal scenario set).
-- On "is something wrong here?" questions the model flags problems less often than the labels do.
-- The output format matches `/v1/systemone`; the behaviour is not Jev's. Vev is not affiliated with TypeSafe.
-
 ## Training
 
-LoRA rank 16 on all linear layers of the language model, vision tower frozen; 2,500 steps on 100,000 records
-sampled from a mixture of 42 public text and image datasets (English and Chinese), with a KL term that keeps the
-answer distribution close to the base model's on training rows and on the 10,123 unlabeled rows of
-[CountingSheep/vev-anchor-v3](https://huggingface.co/datasets/CountingSheep/vev-anchor-v3). One NVIDIA H800. The
-exact recipe, library versions and the list of sources with their licenses are in
-[TRAINING.md](https://github.com/Xiaooolong/vev/blob/main/TRAINING.md).
+LoRA rank 16 on the language model of Qwen/Qwen3.5-9B, vision tower frozen; 2,500 steps on 100,000 records from 42
+public English and Chinese text and image datasets, with a KL term towards the base model on training rows and on
+the 10,123 unlabeled rows of
+[CountingSheep/vev-anchor-v3](https://huggingface.co/datasets/CountingSheep/vev-anchor-v3). Recipe, library
+versions and data sources: [TRAINING.md](https://github.com/Xiaooolong/vev/blob/main/TRAINING.md).
 
 ## License
 
-CC BY-NC 4.0. Six of the training sources are research-only or non-commercial and eleven have no clear license for
-their data; this license does not replace their terms. The base model, Qwen/Qwen3.5-9B, is licensed under the
-Apache License 2.0, included here as `LICENSE-Qwen`.
+CC BY-NC 4.0, non-commercial use only, because some training data is licensed for research use (sources and terms
+in TRAINING.md). The base model is Apache 2.0; its license is included as `LICENSE-Qwen`.
+
+All Vev models and data:
+[collection](https://huggingface.co/collections/CountingSheep/vev-6abd3e17303828e10c49dc65).
 
 ## Citation
 
 ```bibtex
 @software{vev2026,
-  title   = {Vev: an open-weight judgment model for text and images},
+  title   = {Vev: Jev-like decision models that can also see images},
   author  = {Wang, Xiaolong},
   year    = {2026},
   url     = {https://github.com/Xiaooolong/vev},
-  version = {0.1.0}
+  version = {0.1.1}
 }
 ```