Multiple Choice
PEFT
Safetensors
English
gev
decision-model
lora
pointer-head
calibration
Eval Results (legacy)
Instructions to use onatm/gev-e2b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use onatm/gev-e2b with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -28,14 +28,14 @@ model-index:
|
|
| 28 |
|
| 29 |
**State and typed questions in; one answer and a probability distribution per question out.** Gev scores the supplied options rather than generating text. It combines a LoRA adapter on [`google/gemma-4-E2B`](https://huggingface.co/google/gemma-4-E2B) with a separately trained pointer head.
|
| 30 |
|
| 31 |
-
The pointer head (`pointer.safetensors`) is required: loading the PEFT adapter alone does not produce Gev decisions.
|
| 32 |
|
| 33 |
- **Trained-source test:** 82.9% accuracy on 1,200 clean questions.
|
| 34 |
- **New-source test:** 62.5% accuracy on 656 clean questions.
|
| 35 |
|
| 36 |
## Use
|
| 37 |
|
| 38 |
-
|
| 39 |
|
| 40 |
```json
|
| 41 |
{"state":"Order #1 arrived damaged.","questions":{"route":{"type":"choice","instructions":"Which team handles this?","criteria":{"billing":"Payments","support":"Product issues"}}}}
|
|
@@ -74,11 +74,10 @@ This is one seed, not a multi-seed study. The new-source test is substantially h
|
|
| 74 |
- Base: `google/gemma-4-E2B` at revision `d29ff6b45f081a49ee2733a859c9c9c2d95d1a6f`; frozen base weights are not in this repo.
|
| 75 |
- Architecture: LoRA rank 16, alpha 32 on the text decoder and a 256-wide pointer head. Each question is scored independently.
|
| 76 |
- Recipe: seed 0, 2 epochs, 12,576 training records, 3,144 steps, MLX/BF16. The checkpoint stores the fitted temperature in `gev.json`.
|
|
|
|
| 77 |
- Training suite: [jaredpalmer/kev-suites](https://huggingface.co/datasets/jaredpalmer/kev-suites) (`decision-v7`, pinned and verified by hash).
|
| 78 |
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
[Gemma 4 E2B seed-0 results and methodology](https://github.com/onatm/gev/blob/main/docs/results/gemma4-e2b-s0.md)
|
| 82 |
|
| 83 |
## License
|
| 84 |
|
|
|
|
| 28 |
|
| 29 |
**State and typed questions in; one answer and a probability distribution per question out.** Gev scores the supplied options rather than generating text. It combines a LoRA adapter on [`google/gemma-4-E2B`](https://huggingface.co/google/gemma-4-E2B) with a separately trained pointer head.
|
| 30 |
|
| 31 |
+
The pointer head (`pointer.safetensors`) is required: loading the PEFT adapter alone does not produce Gev decisions. The Gev package loads the base model, adapter, pointer head, tokenizer markers, and serving temperature. Its [source repository](https://github.com/onatm/gev) is currently private; access to it is needed to run inference.
|
| 32 |
|
| 33 |
- **Trained-source test:** 82.9% accuracy on 1,200 clean questions.
|
| 34 |
- **New-source test:** 62.5% accuracy on 656 clean questions.
|
| 35 |
|
| 36 |
## Use
|
| 37 |
|
| 38 |
+
With access to the Gev source, run `uv sync --locked` in its checkout (add `--extra mlx` on Apple Silicon). Save this as `request.json`:
|
| 39 |
|
| 40 |
```json
|
| 41 |
{"state":"Order #1 arrived damaged.","questions":{"route":{"type":"choice","instructions":"Which team handles this?","criteria":{"billing":"Payments","support":"Product issues"}}}}
|
|
|
|
| 74 |
- Base: `google/gemma-4-E2B` at revision `d29ff6b45f081a49ee2733a859c9c9c2d95d1a6f`; frozen base weights are not in this repo.
|
| 75 |
- Architecture: LoRA rank 16, alpha 32 on the text decoder and a 256-wide pointer head. Each question is scored independently.
|
| 76 |
- Recipe: seed 0, 2 epochs, 12,576 training records, 3,144 steps, MLX/BF16. The checkpoint stores the fitted temperature in `gev.json`.
|
| 77 |
+
- Training data SHA-256: `7ed5254b5cb5291baefaceb09edf7e13110258211518c8038f4a12c11bd628ad`; each saved evaluation report also records its split's SHA-256.
|
| 78 |
- Training suite: [jaredpalmer/kev-suites](https://huggingface.co/datasets/jaredpalmer/kev-suites) (`decision-v7`, pinned and verified by hash).
|
| 79 |
|
| 80 |
+
Training uses option permutation, none-of-the-above and distractor augmentation, and contrastive pairs. The architecture follows [Jev's Architecture Unmasked](https://archerhume.com/posts/jevs-architecture-unmasked); the data and evaluation protocol are adapted from [Kev](https://github.com/jaredpalmer/kev). The detailed run reports and code are in the private Gev repository.
|
|
|
|
|
|
|
| 81 |
|
| 82 |
## License
|
| 83 |
|