Text Classification
Transformers
Safetensors
MLX
English
qwen3
feature-extraction
tinyjev
jev
decision-model
system-one
typed-decisions
text-embeddings-inference
Instructions to use AnkitAI/TinyJev-0.6B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AnkitAI/TinyJev-0.6B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="AnkitAI/TinyJev-0.6B")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("AnkitAI/TinyJev-0.6B") model = AutoModel.from_pretrained("AnkitAI/TinyJev-0.6B", device_map="auto") - MLX
How to use AnkitAI/TinyJev-0.6B with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download AnkitAI/TinyJev-0.6B --local-dir TinyJev-0.6B
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
update readme
Browse files
README.md
CHANGED
|
@@ -50,19 +50,36 @@ else, because it never generates text; it scores the options you gave it and sto
|
|
| 50 |
## Watch it decide
|
| 51 |
|
| 52 |
<div align="center">
|
| 53 |
-
<img alt="TinyJev answering
|
| 54 |
</div>
|
| 55 |
|
| 56 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
|
| 58 |
<details>
|
| 59 |
-
<summary><b>
|
| 60 |
|
| 61 |
<div align="center">
|
| 62 |
-
<img alt="TinyJev
|
| 63 |
</div>
|
| 64 |
|
| 65 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 66 |
|
| 67 |
</details>
|
| 68 |
|
|
@@ -73,7 +90,12 @@ Six decisions from six domains it never trained on, the same question to both at
|
|
| 73 |
<img alt="TinyJev triaging support tickets" src="https://raw.githubusercontent.com/ankit-aglawe/tinyjev/main/assets/demo_triage.gif" width="860">
|
| 74 |
</div>
|
| 75 |
|
| 76 |
-
Eight real support tickets,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
|
| 78 |
</details>
|
| 79 |
|
|
@@ -95,6 +117,27 @@ This is a demo, not a benchmark: on structured numeric state the answer barely m
|
|
| 95 |
|
| 96 |
</details>
|
| 97 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 98 |
## Measured
|
| 99 |
|
| 100 |
On OpenDecision's Original Choice 500, a suite of 25 domains that was not in the training data, with the same 500 inputs for every model:
|
|
|
|
| 50 |
## Watch it decide
|
| 51 |
|
| 52 |
<div align="center">
|
| 53 |
+
<img alt="TinyJev and GPT-6 Sol answering the same six never-seen questions from a shared start; TinyJev finishes each in under 130 ms, GPT-6 Sol writes JSON for about two seconds" src="https://raw.githubusercontent.com/ankit-aglawe/tinyjev/main/assets/demo_race.gif" width="860">
|
| 54 |
</div>
|
| 55 |
|
| 56 |
+
Six decisions from six domains it never trained on, the same question to both models at the same
|
| 57 |
+
instant. TinyJev answers in one forward pass: 6 of 6 right, 86 ms a question on a base M1. GPT-6 Sol
|
| 58 |
+
writes the answer as JSON, token by token: 6 of 6 right, 2,042 ms a question. Both lanes are real
|
| 59 |
+
runs replayed at real speed; the API lane is a recording with wall-clock timestamps. The questions,
|
| 60 |
+
the answers and both timings are in [`assets/recordings`](assets/recordings).
|
| 61 |
+
|
| 62 |
+
```bash
|
| 63 |
+
pip install 'tinyjev[mlx,demo]'
|
| 64 |
+
python demos/race.py --recording assets/recordings/gpt-6-sol.jsonl --llm-name "GPT-6 Sol" --gif demo.gif
|
| 65 |
+
```
|
| 66 |
|
| 67 |
<details>
|
| 68 |
+
<summary><b>Twelve decisions about one ticket, in one pass</b> — click to expand</summary>
|
| 69 |
|
| 70 |
<div align="center">
|
| 71 |
+
<img alt="TinyJev answering 12 typed decisions about one support ticket in one forward pass, next to GPT-6 Sol writing the same answers as JSON token by token" src="https://raw.githubusercontent.com/ankit-aglawe/tinyjev/main/assets/demo_batch.gif" width="860">
|
| 72 |
</div>
|
| 73 |
|
| 74 |
+
One support ticket, twelve typed decisions: which team, what priority, what the customer wants.
|
| 75 |
+
TinyJev scores every option of every question in a single forward pass and returns all twelve
|
| 76 |
+
together, 596 ms, 10 of 12 right. GPT-6 Sol writes the same twelve as JSON: 2,198 ms, 12 of 12 right.
|
| 77 |
+
The ticket, the questions, the expected answers and both measurements are in
|
| 78 |
+
[`demos/cases`](demos/cases) and [`assets/recordings`](assets/recordings).
|
| 79 |
+
|
| 80 |
+
```bash
|
| 81 |
+
python demos/batch_race.py --data assets/recordings/batch-support-ticket-gpt-6-sol.json --gif demo.gif
|
| 82 |
+
```
|
| 83 |
|
| 84 |
</details>
|
| 85 |
|
|
|
|
| 90 |
<img alt="TinyJev triaging support tickets" src="https://raw.githubusercontent.com/ankit-aglawe/tinyjev/main/assets/demo_triage.gif" width="860">
|
| 91 |
</div>
|
| 92 |
|
| 93 |
+
Eight real support tickets, one after another, on a base M1. Three questions per ticket in a
|
| 94 |
+
single forward pass, about 110 ms each. Every number in that recording came from a live run.
|
| 95 |
+
|
| 96 |
+
```bash
|
| 97 |
+
python demos/triage_desk.py --gif demo.gif
|
| 98 |
+
```
|
| 99 |
|
| 100 |
</details>
|
| 101 |
|
|
|
|
| 117 |
|
| 118 |
</details>
|
| 119 |
|
| 120 |
+
## Models
|
| 121 |
+
|
| 122 |
+
Two models so far, same head, same training data, scored on the same 500 never-seen cases from 25
|
| 123 |
+
domains ([`benchmarks/opendecision`](https://github.com/ankit-aglawe/tinyjev/tree/main/benchmarks/opendecision), every case and probability logged).
|
| 124 |
+
Latency is a base M1 (16 GB) via MLX, one forward pass per case.
|
| 125 |
+
|
| 126 |
+
| Model | Params | OD-500 | ECE | Gate 0.85 | Cov. @ 2% err | transfer-v4 | ms / case | Size | Weights |
|
| 127 |
+
|---|---:|---:|---:|---|---:|---:|---:|---:|---|
|
| 128 |
+
| <img src="https://raw.githubusercontent.com/ankit-aglawe/tinyjev/main/assets/logos/tinyjev.png" width="18"> **TinyJev 0.6B** | 596M | 440 (88.0%) | 0.071 | 59% @ 98.0% | 63% | 0.625 | 85 | 1.2 GB | 🤗 [AnkitAI/tinyjev-0.6b](https://huggingface.co/AnkitAI/tinyjev-0.6b) |
|
| 129 |
+
| <img src="https://raw.githubusercontent.com/ankit-aglawe/tinyjev/main/assets/logos/tinyjev.png" width="18"> **TinyJev 4B** | 4.0B | 474 (94.8%) | 0.022 | 87% @ 99.1% | 92% | 0.762 | 628 | 8.0 GB | 🤗 release pending |
|
| 130 |
+
|
| 131 |
+
OD-500 is correct answers out of 500. Gate 0.85 is the share of decisions answered on its own at
|
| 132 |
+
confidence ≥ 0.85, and how often those were right. Load either with `tinyjev.load("tinyjev-0.6b")`
|
| 133 |
+
or `tinyjev.load("tinyjev-4b")` once released.
|
| 134 |
+
|
| 135 |
+
Both rows are fp16. Loading with `quantize=8` keeps the same weights in half the memory and changes
|
| 136 |
+
almost nothing: the 0.6B scores 440 at 90 ms, the 4B 473 at 845 ms, one answer in 500 different from
|
| 137 |
+
fp16. The gate is the number that matters in production; the rest of the queue goes to a person or a
|
| 138 |
+
bigger model. On the same 500: Kev-0.8B 463, Claude Opus 5.5 496, the same Qwen3-0.6B weights read
|
| 139 |
+
through letter logits with no head 354.
|
| 140 |
+
|
| 141 |
## Measured
|
| 142 |
|
| 143 |
On OpenDecision's Original Choice 500, a suite of 25 domains that was not in the training data, with the same 500 inputs for every model:
|