Upload folder using huggingface_hub

README.md

@@ -1,23 +1,85 @@
 ---
+language: en
 base_model: google/gemma-4-E2B
+base_model_relation: adapter
 library_name: peft
 license: apache-2.0
-tags: [gev, jev, lora, pointer-head, decision]
+tags: [gev, decision-model, lora, pointer-head, multiple-choice, calibration]
+datasets: [jaredpalmer/kev-suites]
+metrics: [accuracy, brier_score, expected_calibration_error]
+model-index:
+  - name: gev-e2b
+    results:
+      - task: { type: text-classification, name: typed decision (choice / noul / score) }
+        dataset: { type: jaredpalmer/kev-suites, name: "decision-v7/test (clean questions)" }
+        metrics:
+          - { type: accuracy, value: 0.8292 }
+          - { type: brier_score, value: 0.2336, name: "Brier (as served)" }
+          - { type: expected_calibration_error, value: 0.0170, name: "ECE (as served)" }
+      - task: { type: text-classification, name: typed decision (choice / noul / score) }
+        dataset: { type: jaredpalmer/kev-suites, name: "transfer-v4/test (clean questions)" }
+        metrics:
+          - { type: accuracy, value: 0.6250 }
+          - { type: brier_score, value: 0.4491, name: "Brier (as served)" }
+          - { type: expected_calibration_error, value: 0.0571, name: "ECE (as served)" }
 ---
 
-# gev-e2b
+# gev-e2b — Gemma 4 E2B decision model
 
-A Jev-like Gev decision model: a LoRA adapter on [`google/gemma-4-E2B`](https://huggingface.co/google/gemma-4-E2B) (revision `d29ff6b45f081a49ee2733a859c9c9c2d95d1a6f`) plus a pointer head that scores each option against the question. Load it with the `gev` package: `gev predict <this repo> --input request.json`.
+**State and typed questions in; one answer and a probability distribution per question out.** Gev scores the supplied options rather than generating text. It combines a LoRA adapter on [`google/gemma-4-E2B`](https://huggingface.co/google/gemma-4-E2B) with a separately trained pointer head.
 
-The architecture follows [Jev's Architecture Unmasked](https://archerhume.com/posts/jevs-architecture-unmasked); the data, augmentation, and metrics come from [Kev](https://github.com/jaredpalmer/kev).
+The pointer head (`pointer.safetensors`) is required: loading the PEFT adapter alone does not produce Gev decisions. Run this checkpoint through [Gev](https://github.com/onatm/gev), which loads the base model, adapter, pointer head, tokenizer markers, and serving temperature.
 
-Serving temperature: `1.6245047927124707`.
+- **Trained-source test:** 82.9% accuracy on 1,200 clean questions.
+- **New-source test:** 62.5% accuracy on 656 clean questions.
 
-## Evaluation (clean questions, raw T=1)
+## Use
 
-| Suite/split | n | Accuracy | NLL | Brier | ECE |
-| --- | ---: | ---: | ---: | ---: | ---: |
-| decision-v7/development | 1264 | 0.7975 | 0.5446 | 0.2847 | 0.0628 |
-| transfer-v4/development | 656 | 0.6113 | 0.8726 | 0.4990 | 0.1163 |
-| decision-v7/test | 1200 | 0.8292 | 0.4709 | 0.2457 | 0.0661 |
-| transfer-v4/test | 656 | 0.6250 | 0.8340 | 0.4680 | 0.1136 |
+Install [Gev](https://github.com/onatm/gev) (add `--extra mlx` to `uv sync` on Apple Silicon), then save this as `request.json`:
+
+```json
+{"state":"Order #1 arrived damaged.","questions":{"route":{"type":"choice","instructions":"Which team handles this?","criteria":{"billing":"Payments","support":"Product issues"}}}}
+```
+
+```bash
+uv run gev predict onatm/gev-e2b --input request.json
+```
+
+The response includes `questions.route.answer`, `questions.route.probabilities` (one per option), and the serving temperature. Choice, yes/no (`noul`), and ordinal (`score`) questions are supported. A Hugging Face text-generation or PEFT-only pipeline cannot serve this model.
+
+## Evaluation
+
+Clean-question scores from the saved reports. Accuracy is at raw T=1 (temperature scaling does not change the winning answer); Brier and ECE are lower-is-better. The served columns use the saved temperature **T=1.6245**. A dash means calibrated metrics were not recorded for that split.
+
+| Suite / split | Clean n | Accuracy | Brier raw | Brier served | ECE raw | ECE served |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| decision-v7/development | 1,264 | 0.7975 | 0.2847 | - | 0.0628 | - |
+| transfer-v4/development | 656 | 0.6113 | 0.4990 | 0.4735 | 0.1163 | 0.0522 |
+| decision-v7/test | 1,200 | 0.8292 | 0.2457 | 0.2336 | 0.0661 | 0.0170 |
+| transfer-v4/test | 656 | 0.6250 | 0.4680 | 0.4491 | 0.1136 | 0.0571 |
+
+`decision-v7` contains held-out questions from the training source families; `transfer-v4` contains new sources and held-out policy structures. Development was for model selection; these reports describe seed 0. Temperature was fitted on the separate `decision-v7/calibration` split, not on either test split.
+
+The decision-v7 test report predates the temperature fit. Its served metrics were computed afterward from saved raw logits, without rerunning inference or fitting on test.
+
+## Known limits
+
+This is one seed, not a multi-seed study. The new-source test is substantially harder than the trained-source test; evaluate on your own decisions before use.
+
+- For changed-answer contrastive pairs, both answers were correct in 12/64 pairs.
+- With a none-of-the-above option present, 14/36 questions were correct.
+
+## Model and provenance
+
+- Base: `google/gemma-4-E2B` at revision `d29ff6b45f081a49ee2733a859c9c9c2d95d1a6f`; frozen base weights are not in this repo.
+- Architecture: LoRA rank 16, alpha 32 on the text decoder and a 256-wide pointer head. Each question is scored independently.
+- Recipe: seed 0, 2 epochs, 12,576 training records, 3,144 steps, MLX/BF16. The checkpoint stores the fitted temperature in `gev.json`.
+- Training suite: [jaredpalmer/kev-suites](https://huggingface.co/datasets/jaredpalmer/kev-suites) (`decision-v7`, pinned and verified by hash).
+
+[Source code and inference instructions](https://github.com/onatm/gev) · [Jev architecture](https://archerhume.com/posts/jevs-architecture-unmasked)
+
+[Gemma 4 E2B seed-0 results and methodology](https://github.com/onatm/gev/blob/main/docs/results/gemma4-e2b-s0.md)
+
+## License
+
+The adapter and pointer head are apache-2.0; check the separately loaded base model and dataset licenses as well.

adapter_config.json

@@ -41,11 +41,10 @@
     "down_proj"
   ],
   "target_parameters": null,
-  "task_type": null,
   "trainable_token_indices": null,
   "use_bdlora": null,
   "use_dora": false,
   "use_qalora": false,
   "use_rslora": false,
   "velora_config": null
-}
+}