AnkitAI commited on
Commit
34243e1
·
verified ·
1 Parent(s): ed1ee89

update model card

Browse files
Files changed (1) hide show
  1. README.md +93 -24
README.md CHANGED
@@ -8,41 +8,110 @@ language: [en]
8
  datasets: [jaredpalmer/kev-suites]
9
  ---
10
 
11
- # tinyjev-0.6b
12
 
13
- A tiny typed-decision model: a state plus typed questions (`choice`, `noul` yes/no, `score`) in, calibrated
14
- probabilities out, one forward pass, no text generation. Runs on MLX (Apple Silicon) or PyTorch.
15
- Code, every benchmark file and the research trail: **https://github.com/ankit-aglawe/tinyjev**
 
16
 
17
- | | transfer-v4 dev | transfer-v4 test (read once) | ECE (test) |
18
- |---|---|---|---|
19
- | **tinyjev-0.6b** | **0.625** | **0.663** | 0.082 |
20
- | Kev-0.6B published (same base) | 0.620 | 0.642 | — |
21
- | Kev-4B published | 0.797 | 0.837 | — |
22
 
23
- Scored on Kev's frozen held-out suite with Kev's own harness. This is the same-size public anchor's level,
24
- slightly ahead on the locked test with lower ECE; it is not 4B-class. Full-fine-tuning, distillation from
25
- Kev-4B and a 149M encoder all failed to beat it — the numbers are in the GitHub README.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
 
27
  ```bash
28
- pip install 'tinyjev[mlx]' # Apple Silicon | pip install 'tinyjev[torch]' # elsewhere
 
29
  ```
30
 
31
  ```python
32
  import tinyjev
33
  agent = tinyjev.load("tinyjev-0.6b")
34
- agent.predict({"state": "I was charged twice. Please fix this ASAP.",
35
- "questions": {"billing": {"type": "noul", "instructions": "Is this ticket about billing?"},
36
- "tone": {"type": "choice", "instructions": "What is the customer's tone?",
37
- "criteria": {"calm": None, "frustrated": None, "angry": None}}}})
 
 
 
 
 
 
 
 
38
  ```
39
 
40
- `AutoModel.from_pretrained("AnkitAI/tinyjev-0.6b")` loads the backbone (a standard `Qwen3Model`, fp16); the
41
- decision head is in `head.safetensors` and needs `tinyjev` to turn hidden states into decisions.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
42
 
43
- Architecture: Qwen3-0.6B-Base + a pointer head (Kev's design), LoRA r16 at lr 5e-5 merged into the base,
44
- trained on `jaredpalmer/kev-suites` decision-v7 (public sources + generated policy examples; no transfer-v4/v9
45
- holdout source used). Fitted temperature 1.46 is applied at inference. On a base M1 via MLX: 65 ms per
46
- request fp16, INT8 free, 4-bit −2.1 points.
47
 
48
- MIT. Base Qwen3-0.6B-Base (Apache-2.0). Training data, harness and head design from Kev (Apache-2.0).
 
 
 
 
8
  datasets: [jaredpalmer/kev-suites]
9
  ---
10
 
11
+ <div align="center">
12
 
13
+ <picture>
14
+ <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/ankit-aglawe/tinyjev/main/assets/tinyjev_header_dark.png">
15
+ <img alt="TinyJev" src="https://raw.githubusercontent.com/ankit-aglawe/tinyjev/main/assets/tinyjev_header.png" width="620">
16
+ </picture>
17
 
18
+ <p>Typed decisions, on your laptop, in one forward pass.</p>
 
 
 
 
19
 
20
+ <p>
21
+ <a href="https://github.com/ankit-aglawe/tinyjev">GitHub</a> ·
22
+ <a href="https://pypi.org/project/tinyjev/">PyPI</a> ·
23
+ <a href="https://github.com/ankit-aglawe/tinyjev/tree/main/examples">Examples</a>
24
+ </p>
25
+
26
+ </div>
27
+
28
+ Send this model some state, a ticket or a record or a log line, plus questions with the answers you
29
+ will accept. It returns a probability for every option you offered. It cannot answer with anything
30
+ else, because it never generates text; it scores the options you gave it and stops.
31
+
32
+ - `Choice` picks one option from a list, with a probability for each.
33
+ - `Noul` measures whether a statement is true.
34
+ - `Score` places state on an ordered scale.
35
+ - Confidence is calibrated, so a threshold means something.
36
+
37
+ 596M parameters, about 1.2 GB. MLX on Apple Silicon, PyTorch everywhere else, fully offline.
38
+
39
+ ## Watch it decide
40
+
41
+ <div align="center">
42
+ <img alt="TinyJev triaging support tickets" src="https://raw.githubusercontent.com/ankit-aglawe/tinyjev/main/assets/demo_triage.gif" width="860">
43
+ </div>
44
+
45
+ Eight real support tickets, three questions each in a single forward pass, about 110 ms per ticket on
46
+ a base M1. Half route themselves. The other half do not: ticket six is late, the wrong size and double
47
+ charged all at once, so probability spreads across all three teams and confidence falls to 0.14. Below
48
+ the gate it goes to a person rather than being guessed at.
49
+
50
+ ## Use it
51
 
52
  ```bash
53
+ pip install 'tinyjev[mlx]' # Apple Silicon
54
+ pip install 'tinyjev[torch]' # everything else
55
  ```
56
 
57
  ```python
58
  import tinyjev
59
  agent = tinyjev.load("tinyjev-0.6b")
60
+
61
+ agent.predict({
62
+ "state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.",
63
+ "questions": {
64
+ "team": {"type": "choice", "instructions": "Which team should handle this?",
65
+ "criteria": {"returns": "Exchanges, refunds, wrong or damaged items",
66
+ "shipping": "Delivery status, delays, lost packages",
67
+ "billing": "Charges, invoices, payment problems"}},
68
+ "escalate": {"type": "noul", "instructions": "Does this need urgent human attention?"},
69
+ "anger": {"type": "score", "instructions": "How angry is the customer?",
70
+ "criteria": ["calm", "frustrated", "very angry"]},
71
+ }})
72
  ```
73
 
74
+ On Apple Silicon you can quantize as it loads. Eight bits is free: half the memory, slightly faster,
75
+ and it scored identically to full precision on our held-out set.
76
+
77
+ ```python
78
+ agent = tinyjev.load("tinyjev-0.6b", quantize=8)
79
+ ```
80
+
81
+ Serve it over HTTP, speaking the System One request shape:
82
+
83
+ ```bash
84
+ tinyjev serve tinyjev-0.6b # POST /v1/systemone on 127.0.0.1:8077
85
+ ```
86
+
87
+ ## What is in this repo
88
+
89
+ `AutoModel.from_pretrained("AnkitAI/tinyjev-0.6b")` loads the backbone on its own, a standard
90
+ `Qwen3Model` in fp16. The decision head lives in `head.safetensors`, and `tinyjev` is what turns hidden
91
+ states into calibrated answers.
92
+
93
+ ## How it was built, and how it scores
94
+
95
+ Qwen3-0.6B-Base with a pointer head, LoRA r16 at lr 5e-5 merged back into the base, trained on the
96
+ public `jaredpalmer/kev-suites` decision-v7 split. No held-out transfer source was used in training.
97
+ A fitted temperature of 1.46 is applied at inference.
98
+
99
+ | | transfer-v4 dev | transfer-v4 test, read once | ECE on test |
100
+ | --- | --- | --- | --- |
101
+ | tinyjev-0.6b | 0.625 | 0.663 | 0.082 |
102
+ | Same-size public anchor | 0.620 | 0.642 | — |
103
+
104
+ Scored with the upstream harness on its frozen held-out suite. This matches the same-size public anchor
105
+ and edges ahead on the locked test with lower calibration error. It is not 4B-class, and it is not
106
+ meant to be. Full fine-tuning, distillation from a 4B teacher, and a 149M encoder were all tried and
107
+ all lost to the configuration above.
108
+
109
+ It will not play games. Given state that is already structured and numeric, it stops discriminating;
110
+ its competence is judgment about language. Use the confidence gate on anything far from what it saw.
111
 
112
+ ## Credits
 
 
 
113
 
114
+ Built on [Qwen3-0.6B-Base](https://huggingface.co/Qwen/Qwen3-0.6B-Base) (Apache-2.0). The training data,
115
+ evaluation suites and the pointer-head design come from [Kev](https://github.com/jaredpalmer/kev) by
116
+ Jared Palmer (Apache-2.0). The typed-decision interface follows
117
+ [TypeSafe's Jev](https://docs.typesafe.ai/introduction). MIT licensed.