devankit7873 commited on
Commit
f43096a
Β·
verified Β·
1 Parent(s): 1be400b

docs: sync polished README with Phase 3 benchmarks and multi-engine serving

Browse files
Files changed (1) hide show
  1. README.md +331 -349
README.md CHANGED
@@ -1,349 +1,331 @@
1
- ---
2
- license: apache-2.0
3
- base_model: Qwen/Qwen2.5-1.5B-Instruct
4
- library_name: transformers
5
- tags:
6
- - transformers
7
- - pytorch
8
- - causal-lm
9
- - qwen
10
- - qwen2
11
- - decision-making
12
- - system-one
13
- - reasoning
14
- - jev
15
- - jevbench
16
- - joint-expected-value
17
- - classification
18
- - safetensors
19
- - qlora
20
- - agent
21
- - decision-model
22
- - text-classification
23
- - fast-inference
24
- datasets:
25
- - SargeDev/jev-distill-corpus-v3
26
- metrics:
27
- - accuracy
28
- - tvd
29
- - kl_divergence
30
- pipeline_tag: text-classification
31
- ---
32
-
33
- <div align="center">
34
-
35
- # ⚑ QJev-1.5B: Qwen2.5-1.5B JEV Decision Model
36
- ### High-Speed System 1 Real-Time Decision Engine (~31 ms Forward Pass)
37
-
38
- [![Hugging Face](https://img.shields.io/badge/Hugging%20Face-QJev--1.5B-FFD21E?style=for-the-badge&logo=huggingface&logoColor=black)](https://huggingface.co/devankit7873/QJev-1.5B)
39
- [![GitHub](https://img.shields.io/badge/GitHub-QJev--1.5B-181717?style=for-the-badge&logo=github)](https://github.com/AnkitNayak-dev/QJev-1.5B)
40
- [![JevBench](https://img.shields.io/badge/JevBench-62.53%25%20Official-8A2BE2?style=for-the-badge)](https://jevbench.github.io/)
41
- [![License](https://img.shields.io/badge/License-Apache%202.0-2EA44F?style=for-the-badge)](LICENSE)
42
-
43
- ---
44
-
45
- ### πŸ“¦ QJev Model Family (Choose Your Format):
46
- - **Full Fused Standalone:** [devankit7873/QJev-1.5B](https://huggingface.co/devankit7873/QJev-1.5B) (3.08 GB β€’ Zero PEFT dependency)
47
- - **Lightweight LoRA Adapter:** [devankit7873/QJev-1.5B-LoRA](https://huggingface.co/devankit7873/QJev-1.5B-LoRA) (20 MB β€’ Modular edge loading)
48
- - **4-Bit NF4 Quantized:** [devankit7873/QJev-1.5B-4bit](https://huggingface.co/devankit7873/QJev-1.5B-4bit) (1.15 GB β€’ Ultra-low 1.07 GB VRAM)
49
-
50
- ---
51
-
52
- [News](#-news) β€’ [Quickstart](#-quickstart--usage) β€’ [Model Download](#-how-to-download-the-model) β€’ [Benchmarks](#-official-jevbench-metamorphic-results) β€’ [Architecture](#-model-architecture--system-1-paradigm) β€’ [Citation](#-citation)
53
-
54
- </div>
55
-
56
- ---
57
-
58
- ## πŸ“’ News
59
- * **[2026-11-16]** πŸš€ **QJev-1.5B is officially released!** Trained on all 655,806 samples of `SargeDev/jev-distill-corpus-v3`. Achieves **71.5% calibrated JevBench score** (within 2.2% of 12B leaders) while running in **3.4 GB VRAM at ~31 ms per decision**.
60
-
61
- ---
62
-
63
- ## πŸ’‘ Overview: System 1 Decision Paradigm
64
-
65
- Traditional LLMs make decisions through slow, non-deterministic token-by-token autoregressive generation (e.g. generating `"Option A"` over 300–600 ms).
66
-
67
- **`QJev-1.5B`** introduces a **System 1 Decision Architecture**:
68
- 1. Attaches a calibrated **Multi-Layer Perceptron (MLP) Decision Head** to the final hidden state of `Qwen/Qwen2.5-1.5B-Instruct`.
69
- 2. Jointly trains attention projection layers (`q_proj`, `k_proj`, `v_proj`, `o_proj`) via **QLoRA** and the decision head across all 655,806 distillation samples.
70
- 3. Outputs a **mathematically calibrated probability distribution** over up to 10 choices in a **single forward pass (~31 ms)**.
71
-
72
- ---
73
-
74
- ## βš–οΈ JEV Ecosystem Comparison
75
-
76
- | Model | Parameters | Target Hardware | Decision Latency | Choice Flexibility | JevBench Score | OOD Accuracy | Memory Required |
77
- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
78
- | **`SargeDev/jev-gate-student-b`** | 0.49B | Edge / CPU | ~23 ms | **Binary Only (2 choices)** | ~50.2% | ~64.5% | ~1.5 GB |
79
- | **`QJev-1.5B` (Ours)** | **1.54B** | **Any 8GB Laptop / GPU** | **31.49 ms (~32 dec/s)** | **Arbitrary Choices ($\le 10$)** | **71.50%** | **80.07%** | **~3.4 GB** |
80
- | **`autotrust/JEV-9B`** | 9.0B | 24GB Server GPU | ~45 – 90 ms | Arbitrary Choices ($\le 16$) | ~72.0% | ~83.5% | ~18.5 GB |
81
- | **`autotrust/JEV-27B`** | 27.0B | Datacenter (A100) | ~150 – 350 ms | Arbitrary Choices ($\le 16$) | ~76.5% | ~86.2% | ~55.0 GB |
82
- | **`TypeSafe Jev 1.13` (Teacher)** | ~30B+ | Cloud API | ~200 ms+ | Arbitrary Choices | ~72.1% | ~88.0% | Closed API |
83
-
84
- ### Why Choose QJev-1.5B?
85
- - **The "Goldilocks" Sweet Spot:** Captures **~95% of the decision capability of 9B models** while being **6x smaller** and **30%–65% faster**.
86
- - **Dynamic Multi-Choice Support:** Unlike 0.5B students that only do binary yes/no, QJev-1.5B dynamically handles 2, 3, 5, or up to 10 choices.
87
- - **Consumer Hardware Friendly:** Pegged at **3.4 GB VRAM**, deployable on consumer laptops, edge robotics, and $5/month cloud VPS instances.
88
-
89
- ---
90
-
91
- ## πŸ† Official JevBench Metamorphic Results
92
-
93
- Evaluated against the official **[JevBench](https://jevbench.github.io/)** test harness (`jevbench-mini`, 240 real-world decision cases, 50 scored metamorphic relations):
94
-
95
- | JevBench Metamorphic Dimension | Baseline (Pre-Consistency) | QJev-1.5B (Phase 3 Consistency) | Delta / Gain | Practical Meaning |
96
- | :--- | :---: | :---: | :---: | :--- |
97
- | **`BAT.isolation`** | **100.0%** | **100.0%** | **0.0%** | Predictions are completely independent of batching (Zero cross-talk) |
98
- | **`REP.type`** | 77.1% | **83.3%** | **+6.2%** | Format coherence across multi-choice, boolean, and ordinal score |
99
- | **`REP.invariance`** | 73.2% | **74.4%** | **+1.2%** | Immune to choice shuffling, key renames, and prompt perturbation |
100
- | **`LOG.bounds`** | 62.5% | **67.9%** | **+5.4%** | Strict adherence to probabilistic upper/lower bounds |
101
- | **`CHO.conditioning`** | 41.7% | **56.9%** | **+15.2%** | Removing irrelevant choices preserves odds ratios (Independence) |
102
- | **`REP.equivariance`** | 48.6% | **55.6%** | **+7.0%** | Symmetrical consistency under prompt transformations |
103
- | **`MEA.partition`** | 30.7% | **41.1%** | **+10.4%** | Probability mass is preserved across coarsened option partitions |
104
- | **OVERALL JEVBENCH OFFICIAL (Strict 0.05)** | **58.67%** `[56–61]` | **62.53%** `[60–65]` | **+3.86%** | **Official Benchmark Standard (Places in 4B-model tier)** |
105
- | **OVERALL JEVBENCH (Relaxed 0.10)** | 60.2% | **71.5%** | **+11.3%** | Lenient consistency threshold |
106
-
107
- Machine-readable evaluation reports are archived in `reports/jevbench/`.
108
-
109
- ---
110
-
111
- ## πŸ“Š Dataset Benchmark Trajectory (655,806 Samples)
112
-
113
- Evaluated on fixed 1,500-sample test (in-distribution) and Out-of-Distribution (OOD) benchmark splits from `SargeDev/jev-distill-corpus-v3`:
114
-
115
- | Metric | Phase 1 (Frozen Backbone Baseline) | Phase 2 (QLoRA Joint Training) | Delta / Improvement |
116
- | :--- | :---: | :---: | :---: |
117
- | **OOD Accuracy (`ood`)** | 57.53% | **80.07%** | **+22.54%** |
118
- | **OOD Total Variation Distance (TVD)** | 0.4607 | **0.2389** | **-48.1% (Calibrated)** |
119
- | **OOD KL Loss** | 0.7884 | **0.6972** | **-11.6%** |
120
- | **In-Distribution TVD (`test`)** | 0.1664 | **0.0921** | **-44.6% (Sub-9% deviation)** |
121
- | **In-Distribution Loss** | 0.1757 | **0.1033** | **-41.2%** |
122
- | **Inference Latency** | 42.94 ms | **31.49 ms** | **26.7% faster** |
123
-
124
- ---
125
-
126
- ## πŸ“₯ How to Download the Model
127
-
128
- We host full fused weights and modular LoRA adapters on the [Hugging Face Hub](https://huggingface.co/devankit7873/QJev-1.5B).
129
-
130
- ### Method 1: Hugging Face CLI (Fastest)
131
- ```bash
132
- huggingface-cli download devankit7873/QJev-1.5B --local-dir ./QJev-1.5B
133
- ```
134
-
135
- ### Method 2: Python `snapshot_download`
136
- ```python
137
- from huggingface_hub import snapshot_download
138
-
139
- snapshot_download(repo_id="devankit7873/QJev-1.5B", local_dir="./QJev-1.5B")
140
- ```
141
-
142
- ### Method 3: Git LFS Clone
143
- ```bash
144
- git lfs install
145
- git clone https://huggingface.co/devankit7873/QJev-1.5B
146
- ```
147
-
148
- ---
149
-
150
- ## πŸš€ Quickstart & Usage
151
-
152
- ### 1. Installation
153
- ```bash
154
- git clone https://github.com/AnkitNayak-dev/QJev-1.5B.git
155
- cd QJev-1.5B
156
- pip install -r requirements.txt
157
- ```
158
-
159
- ### 2. 1-Command CLI Decision
160
- ```bash
161
- python qjev.py --context "Patient has severe chest pain and elevated troponin" --choices "Emergency PCI" "Aspirin" "Wait"
162
- ```
163
-
164
- ### 3. Super Simple Python API (3 Lines of Code)
165
- ```python
166
- from qjev import QJev
167
-
168
- # Load ANY edition (Full Standalone, 4-bit Quantized, or LoRA Adapter) in 1 line:
169
- model = QJev.from_pretrained("devankit7873/QJev-1.5B")
170
- # Or 4-bit (1.07 GB VRAM): model = QJev.from_pretrained("devankit7873/QJev-1.5B-4bit")
171
- # Or LoRA (20 MB download): model = QJev.from_pretrained("devankit7873/QJev-1.5B-LoRA")
172
-
173
- # 1-Line Decision (<35 ms)
174
- result = model.decide(
175
- context="Production database primary replica CPU spiked to 100% and connection pool is exhausted.",
176
- choices=[
177
- "Trigger immediate automated failover to standby replica",
178
- "Restart the primary database daemon immediately",
179
- "Double connection pool limits without scaling compute",
180
- "Silence alerts and monitor for next 15 minutes"
181
- ]
182
- )
183
-
184
- print(f"πŸ‘‰ Recommended Action: {result['best_choice']} ({result['confidence']}%)")
185
- for choice, prob in result['probabilities'].items():
186
- print(f" [{prob:>5.2f}%] {choice}")
187
- ```
188
-
189
- **Expected Output:**
190
- ```text
191
- πŸ‘‰ Recommended Action: Trigger immediate automated failover to standby replica (88.28%)
192
- [88.28%] Trigger immediate automated failover to standby replica
193
- [ 6.54%] Restart the primary database daemon immediately
194
- [ 4.79%] Double connection pool limits without scaling compute
195
- [ 0.45%] Silence alerts and monitor for next 15 minutes
196
- ```
197
-
198
- ---
199
-
200
- ## 🌐 Multi-Engine Inference Deployment
201
-
202
- QJev-1.5B is designed to deploy across **all major LLM serving runtimes**:
203
-
204
- ### 1. vLLM (High-Throughput Production)
205
- ```bash
206
- # Serve QJev with continuous batching and PagedAttention:
207
- vllm serve devankit7873/QJev-1.5B --task classify --port 8000
208
- ```
209
- Query from Python via `qjev.py`:
210
- ```python
211
- from qjev import QJev
212
- model = QJev.from_remote("http://localhost:8000")
213
- result = model.decide(context="Kubernetes node 3 unreachable", choices=["Drain", "Reboot", "Wait"])
214
- ```
215
-
216
- ### 2. Ollama (Local Mac / PC / CPU)
217
- Create and run locally using the included `Modelfile`:
218
- ```bash
219
- ollama create qjev-1.5b -f Modelfile
220
- ollama run qjev-1.5b
221
- ```
222
- Or query from Python:
223
- ```python
224
- from qjev import QJev
225
- model = QJev.from_ollama("http://localhost:11434", model="qjev-1.5b")
226
- ```
227
-
228
- ### 3. llama.cpp / GGUF (Edge Devices & LM Studio)
229
- Convert to GGUF format using the export tool:
230
- ```bash
231
- python scripts/export_gguf.py --model-dir export/QJev-1.5B-Phase3 --out-file qjev-1.5b-f16.gguf
232
- ./llama-quantize qjev-1.5b-f16.gguf qjev-1.5b-q4_k_m.gguf Q4_K_M
233
- ```
234
-
235
- ### 4. Unified Microservice Server (`serve.py`)
236
- Run the all-in-one FastAPI server exposing REST, System One, and OpenAI protocols:
237
- ```bash
238
- python serve.py --port 8000
239
- ```
240
- * **REST API:** `POST /v1/decide`
241
- * **Official Jev Protocol:** `POST /v1/systemone`
242
- * **OpenAI Drop-In:** `POST /v1/chat/completions`
243
-
244
- ---
245
-
246
- ## πŸ—οΈ Model Architecture & System 1 Paradigm
247
-
248
- ```
249
- [User Context + Formatted Choices]
250
- β”‚
251
- β–Ό
252
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
253
- β”‚ Qwen2.5-1.5B Backbone β”‚ (QLoRA adapted on q, k, v, o projections)
254
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
255
- β”‚ (Last Hidden State: 1536 dim)
256
- β–Ό
257
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
258
- β”‚ LayerNorm(1536) β”‚
259
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
260
- β”‚
261
- β–Ό
262
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
263
- β”‚ Linear(1536 -> 512) β”‚ (GELU + Dropout 0.1)
264
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
265
- β”‚
266
- β–Ό
267
- β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
268
- β”‚ Linear(512 -> 10) β”‚ (Dynamic Choice Masking)
269
- β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
270
- β”‚
271
- β–Ό
272
- [Calibrated Softmax Probabilities]
273
- ```
274
-
275
- ---
276
-
277
- ## πŸ’» Hardware & Training Infrastructure
278
-
279
- Trained from scratch locally on consumer silicon:
280
- - **Host System:** HP OMEN Gaming Laptop (Windows 11 Pro 64-bit)
281
- - **GPU:** NVIDIA GeForce RTX 5060 Laptop GPU (8GB VRAM)
282
- - **CUDA / Driver:** CUDA 12.8 / Driver 621.24 (CUDA UMD 13.5)
283
- - **Framework:** PyTorch `2.12.0.dev20260408+cu128`, `transformers 5.22.0`, `peft 0.21.2`
284
- - **Precision:** Float32 Matmul Precision set to `high` with Tensor Core **TF32** acceleration
285
- - **VRAM Footprint:** Strictly pegged at **5.90 GB / 8.0 GB VRAM** (0 OOM errors over 655k steps)
286
- - **Effective Batch Size:** 8 per device $\times$ 4 gradient accumulation = 32
287
- - **Throughput:** ~11.5–12.0 samples/sec (total ~18.5 GPU compute hours)
288
-
289
- ---
290
-
291
- ## πŸ“ Repository Structure
292
-
293
- ```text
294
- β”œβ”€β”€ README.md # Project showcase, benchmarks, and usage
295
- β”œβ”€β”€ LICENSE # Apache 2.0 open-source license
296
- β”œβ”€β”€ requirements.txt # Python dependencies
297
- β”œβ”€β”€ predict.py # 1-click single-pass inference demo
298
- β”‚
299
- β”œβ”€β”€ src/ # Core modeling and training pipeline
300
- β”‚ β”œβ”€β”€ model.py # MLP Decision Head + Qwen integration
301
- β”‚ β”œβ”€β”€ dataset.py # ChatML streaming dataset loader
302
- β”‚ β”œβ”€β”€ train.py # Phase 1 baseline training script
303
- β”‚ β”œβ”€β”€ train_phase2.py # Phase 2 QLoRA joint training script (655k steps)
304
- β”‚ └── evaluate.py # Evaluation metrics (TVD, Accuracy, KL loss)
305
- β”‚
306
- β”œβ”€β”€ benchmark/ # Evaluation suites
307
- β”‚ └── run_jevbench.py # Official JevBench metamorphic evaluation runner
308
- β”‚
309
- β”œβ”€β”€ reports/ # Complete empirical benchmark records
310
- β”‚ β”œβ”€β”€ jevbench_aligned/ # Official JevBench scores (71.5% calibrated)
311
- β”‚ β”œβ”€β”€ phase_1/ # Phase 1 baseline evaluation records
312
- β”‚ └── phase_2/ # Phase 2 evaluations across all 50k checkpoints
313
- β”‚
314
- └── scripts/ # Workflow utilities
315
- β”œβ”€β”€ merge_and_export.py # LoRA fusing and safetensors export
316
- β”œβ”€β”€ download_data.py # Dataset loader from Hugging Face
317
- β”œβ”€β”€ download_model.py # Base model downloader
318
- └── upload_to_hf.py # Automated Hugging Face publisher
319
- ```
320
-
321
- ---
322
-
323
- ## πŸ“ˆ Telemetry & Tracking
324
- All training trajectories, step metrics, and evaluations were recorded in real time:
325
- - **Weights & Biases Dashboard:** [`phase-2-lora-joint-run`](https://forge.coreweave.com/wandb/devankit/qwen-decision-head/runs/phase-2-lora-joint-run)
326
- - **Source Code & Pipelines:** [github.com/AnkitNayak-dev/QJev-1.5B](https://github.com/AnkitNayak-dev/QJev-1.5B)
327
- - **Model Weights & Hub:** [huggingface.co/devankit7873/QJev-1.5B](https://huggingface.co/devankit7873/QJev-1.5B)
328
-
329
- ---
330
-
331
- ## πŸ“š Citation
332
-
333
- If you use **QJev-1.5B** or its benchmark methodology in your research, please cite:
334
-
335
- ```bibtex
336
- @misc{nayak2026qjev15b,
337
- author = {Ankit Nayak},
338
- title = {QJev-1.5B: High-Speed System 1 JEV Decision Model based on Qwen2.5},
339
- year = {2026},
340
- publisher = {GitHub and Hugging Face},
341
- howpublished = {\url{https://github.com/AnkitNayak-dev/QJev-1.5B}}
342
- }
343
- ```
344
-
345
- ---
346
-
347
- ## πŸ“„ License
348
- This model and codebase are released under the **Apache 2.0 License**.
349
- Base weights are governed by the Alibaba Cloud Qwen License. Distillation corpus is provided by SargeDev (`jev-distill-corpus-v3`). Metamorphic benchmarking powered by JevBench.
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Qwen/Qwen2.5-1.5B-Instruct
4
+ library_name: transformers
5
+ tags:
6
+ - transformers
7
+ - pytorch
8
+ - causal-lm
9
+ - qwen
10
+ - qwen2
11
+ - decision-making
12
+ - system-one
13
+ - reasoning
14
+ - jev
15
+ - jevbench
16
+ - joint-expected-value
17
+ - classification
18
+ - safetensors
19
+ - qlora
20
+ - agent
21
+ - decision-model
22
+ - text-classification
23
+ - fast-inference
24
+ datasets:
25
+ - SargeDev/jev-distill-corpus-v3
26
+ metrics:
27
+ - accuracy
28
+ - tvd
29
+ - kl_divergence
30
+ pipeline_tag: text-classification
31
+ ---
32
+
33
+ <div align="center">
34
+
35
+ # ⚑ QJev-1.5B: High-Speed System 1 JEV Decision Model
36
+ ### Single-Pass Real-Time Decision Engine (~31 ms Forward Pass β€’ Ranked #1 Sub-2B Model)
37
+
38
+ [![Hugging Face](https://img.shields.io/badge/Hugging%20Face-devankit7873%2FQJev--1.5B-FFD21E?style=for-the-badge&logo=huggingface&logoColor=black)](https://huggingface.co/devankit7873/QJev-1.5B)
39
+ [![GitHub](https://img.shields.io/badge/GitHub-QJev--1.5B-181717?style=for-the-badge&logo=github)](https://github.com/AnkitNayak-dev/QJev-1.5B)
40
+ [![JevBench](https://img.shields.io/badge/JevBench%20Official-62.53%25%20(tol=0.05)-8A2BE2?style=for-the-badge)](https://benchmarkheaven.com/jev-models)
41
+ [![JevBench Relaxed](https://img.shields.io/badge/JevBench%20Relaxed-71.50%25%20(tol=0.10)-blue?style=for-the-badge)](https://benchmarkheaven.com/jev-models)
42
+ [![License](https://img.shields.io/badge/License-Apache%202.0-2EA44F?style=for-the-badge)](LICENSE)
43
+
44
+ ---
45
+
46
+ ### 🌳 Hugging Face Model Tree & Flavors:
47
+ ```
48
+ Qwen/Qwen2.5-1.5B-Instruct (Root Base)
49
+ └── ⚑ devankit7873/QJev-1.5B (Flagship Fused 16-bit β€’ 3.08 GB β€’ Zero PEFT dependency)
50
+ β”œβ”€β”€ πŸ”Œ devankit7873/QJev-1.5B-LoRA (PEFT Adapter β€’ 20 MB β€’ Modular edge loading)
51
+ └── πŸ—œοΈ devankit7873/QJev-1.5B-4bit (NF4 Quantized β€’ 1.15 GB β€’ 1.07 GB VRAM)
52
+ ```
53
+
54
+ [News](#-news) β€’ [Overview](#-overview-system-1-decision-paradigm) β€’ [Leaderboard & Benchmarks](#-official-benchmarkheaven--jevbench-results) β€’ [Quickstart](#-quickstart--usage) β€’ [Multi-Engine Serving](#-multi-engine-inference-deployment) β€’ [Architecture](#-model-architecture) β€’ [Repo Structure](#-repository-structure) β€’ [Citation](#-citation)
55
+
56
+ </div>
57
+
58
+ ---
59
+
60
+ ## πŸ“’ News
61
+ * **[2026-11-17] 🎯 Phase 3 Consistency Release & Multi-Engine Serving:**
62
+ - Trained with cross-format symmetric consistency loss, boosting official **JevBench strict tolerance (0.05) score from 58.67% to 62.53%** (+3.86% overall, +15.2% on choice conditioning).
63
+ - Officially ranks as the **#1 decision model in the sub-2B tier**, outperforming multiple 4B baselines on [BenchmarkHeaven](https://benchmarkheaven.com/jev-models).
64
+ - Shipped native deployment support for **vLLM**, **Ollama (`Modelfile`)**, **llama.cpp (GGUF)**, and **production FastAPI (`serve.py`)**.
65
+ * **[2026-11-16] πŸš€ Initial Release:**
66
+ - Full QLoRA joint training across all 655,806 distillation samples of `SargeDev/jev-distill-corpus-v3`.
67
+ - Achieves **71.50% relaxed metamorphic score** in **3.4 GB VRAM at ~31 ms per decision**.
68
+
69
+ ---
70
+
71
+ ## πŸ’‘ Overview: System 1 Decision Paradigm
72
+
73
+ Traditional LLMs make decisions through slow, non-deterministic token-by-token autoregressive generation (e.g. generating `"Option A"` over 300–600 ms).
74
+
75
+ **`QJev-1.5B`** introduces a **System 1 Decision Architecture**:
76
+ 1. Attaches a calibrated **Multi-Layer Perceptron (MLP) Decision Head** to the final hidden state of `Qwen/Qwen2.5-1.5B-Instruct`.
77
+ 2. Jointly trains attention projection layers (`q_proj`, `k_proj`, `v_proj`, `o_proj`) via **QLoRA** and the decision head across all 655,806 distillation samples.
78
+ 3. In Phase 3, fine-tunes with soft target KL-divergence consistency loss across multiple-choice, boolean probes, and permutation groups.
79
+ 4. Outputs a **mathematically calibrated probability distribution** over up to 10 choices in a **single forward pass (~31 ms)**.
80
+
81
+ ---
82
+
83
+ ## πŸ† Official BenchmarkHeaven & JevBench Results
84
+
85
+ Evaluated against the official **JevBench** test harness (`jevbench-mini`, 240 cases, 50 scored metamorphic relations, 1,468 trials):
86
+
87
+ ### 1. Leaderboard Placement (BenchmarkHeaven JEV Leaderboard)
88
+
89
+ Under the official BenchmarkHeaven standard (**strict tolerance = 0.05**):
90
+
91
+ | Model | Parameters | Benchmark Score (tol=0.05) | Rank / Category | Latency | VRAM Footprint |
92
+ | :--- | :---: | :---: | :---: | :---: | :---: |
93
+ | `EXAONE-1.2B` | 1.2B | 47.7% | Sub-2B | ~25 ms | ~2.8 GB |
94
+ | `SargeDev/jev-gate-student-b` | 0.49B | 50.2% | Sub-2B (Binary Only) | ~23 ms | ~1.5 GB |
95
+ | `decision-2b` | 2.0B | 54.8% | 2B Tier | ~40 ms | ~4.5 GB |
96
+ | `pjev-2b` | 2.0B | 56.5% | 2B Tier | ~38 ms | ~4.4 GB |
97
+ | **`QJev-1.5B` (Ours, Phase 3)** | **1.54B** | **62.53%** | **πŸ‘‘ #1 Sub-2B (Beats 4B)** | **~31 ms** | **~3.4 GB (1.07 GB in 4-bit)** |
98
+ | `jiwo-4b` | 4.0B | 60.2% | 4B Tier | ~65 ms | ~8.8 GB |
99
+ | `decision-4b-v12` | 4.0B | 60.4% | 4B Tier | ~62 ms | ~8.5 GB |
100
+ | `autotrust/JEV-9B` | 9.0B | 72.0% | 9B Tier | ~85 ms | ~18.5 GB |
101
+ | `autotrust/JEV-27B` | 27.0B | 76.5% | Server Tier | ~220 ms | ~55.0 GB |
102
+
103
+ > **Note on Evaluation Tolerances:**
104
+ > - **Official Strict (`tol = 0.05`):** **62.53%** (Official score reported on BenchmarkHeaven public leaderboard).
105
+ > - **Relaxed Exploration (`tol = 0.10`):** **71.50%** (Lenient consistency threshold).
106
+
107
+ ### 2. Breakdown Across Metamorphic Dimensions
108
+
109
+ | Dimension | Baseline (Pre-Consistency) | QJev-1.5B (Phase 3 Consistency) | Gain | Practical Meaning |
110
+ | :--- | :---: | :---: | :---: | :--- |
111
+ | **`BAT.isolation`** | **100.0%** | **100.0%** | **0.0%** | Batching independence (Zero cross-talk between batch items) |
112
+ | **`REP.type`** | 77.1% | **83.3%** | **+6.2%** | Format coherence across multi-choice, boolean, and ordinal prompts |
113
+ | **`REP.invariance`** | 73.2% | **74.4%** | **+1.2%** | Immune to choice reordering and prompt syntax perturbations |
114
+ | **`LOG.bounds`** | 62.5% | **67.9%** | **+5.4%** | Strict adherence to probabilistic upper and lower bounds |
115
+ | **`CHO.conditioning`** | 41.7% | **56.9%** | **+15.2%** | Removing irrelevant choices preserves odds ratios (Independence of Irrelevant Alternatives) |
116
+ | **`REP.equivariance`** | 48.6% | **55.6%** | **+7.0%** | Symmetrical probability shifts under negation transformations |
117
+ | **`MEA.partition`** | 30.7% | **41.1%** | **+10.4%** | Probability mass preservation across option clustering |
118
+ | **OVERALL (Strict 0.05)** | **58.67%** | **62.53%** | **+3.86%** | **Official Benchmark Standard** |
119
+
120
+ Machine-readable evaluation reports are archived in `reports/jevbench/`.
121
+
122
+ ---
123
+
124
+ ## πŸš€ Quickstart & Usage
125
+
126
+ ### 1. Installation
127
+ ```bash
128
+ git clone https://github.com/AnkitNayak-dev/QJev-1.5B.git
129
+ cd QJev-1.5B
130
+ pip install -r requirements.txt
131
+ ```
132
+
133
+ ### 2. Fast Python API (3 Lines of Code)
134
+ ```python
135
+ from qjev import QJev
136
+
137
+ # Load ANY edition (Full Standalone, 4-bit Quantized, or LoRA Adapter) in 1 line:
138
+ model = QJev.from_pretrained("devankit7873/QJev-1.5B")
139
+ # Or 4-bit (1.07 GB VRAM): model = QJev.from_pretrained("devankit7873/QJev-1.5B-4bit")
140
+ # Or LoRA (20 MB download): model = QJev.from_pretrained("devankit7873/QJev-1.5B-LoRA")
141
+
142
+ # 1-Line Decision (~31 ms)
143
+ result = model.decide(
144
+ context="Production database primary replica CPU spiked to 100% and connection pool is exhausted.",
145
+ choices=[
146
+ "Trigger immediate automated failover to standby replica",
147
+ "Restart the primary database daemon immediately",
148
+ "Double connection pool limits without scaling compute",
149
+ "Silence alerts and monitor for next 15 minutes"
150
+ ]
151
+ )
152
+
153
+ print(f"πŸ‘‰ Recommended Action: {result['best_choice']} ({result['confidence']}%)")
154
+ for choice, prob in result['probabilities'].items():
155
+ print(f" [{prob:>5.2f}%] {choice}")
156
+ ```
157
+
158
+ **Expected Output:**
159
+ ```text
160
+ πŸ‘‰ Recommended Action: Trigger immediate automated failover to standby replica (88.28%)
161
+ [88.28%] Trigger immediate automated failover to standby replica
162
+ [ 6.54%] Restart the primary database daemon immediately
163
+ [ 4.79%] Double connection pool limits without scaling compute
164
+ [ 0.45%] Silence alerts and monitor for next 15 minutes
165
+ ```
166
+
167
+ ### 3. CLI Command-Line Decision
168
+ ```bash
169
+ python qjev.py --context "Patient has severe chest pain and elevated troponin" --choices "Emergency PCI" "Aspirin" "Wait"
170
+ ```
171
+
172
+ ---
173
+
174
+ ## 🌐 Multi-Engine Inference Deployment
175
+
176
+ QJev-1.5B natively supports **all standard LLM serving runtimes**:
177
+
178
+ ### 1. High-Performance Microservice Server (`serve.py`)
179
+ Run the unified FastAPI server exposing REST, System One, and OpenAI compatible endpoints:
180
+ ```bash
181
+ python serve.py --model devankit7873/QJev-1.5B --port 8000
182
+ ```
183
+ Endpoints provided:
184
+ * **REST Decision API:** `POST /v1/decide`
185
+ * **Official Jev Protocol (BenchmarkHeaven):** `POST /v1/systemone`
186
+ * **OpenAI Drop-In Compatible:** `POST /v1/chat/completions`
187
+
188
+ Query via cURL:
189
+ ```bash
190
+ curl -X POST http://localhost:8000/v1/decide \
191
+ -H "Content-Type: application/json" \
192
+ -d '{
193
+ "context": "Customer requests immediate refund after 45 days.",
194
+ "options": ["Approve refund", "Offer store credit", "Deny request"]
195
+ }'
196
+ ```
197
+
198
+ ### 2. Ollama (Local Mac / PC / Linux)
199
+ Package and run locally with Ollama using the included `Modelfile`:
200
+ ```bash
201
+ ollama create qjev-1.5b -f Modelfile
202
+ ollama run qjev-1.5b
203
+ ```
204
+ Or query from Python via `QJev`:
205
+ ```python
206
+ from qjev import QJev
207
+ model = QJev.from_ollama("http://localhost:11434", model="qjev-1.5b")
208
+ result = model.decide("Route packet to cluster A or B?", ["Cluster A", "Cluster B"])
209
+ ```
210
+
211
+ ### 3. llama.cpp / GGUF (LM Studio & Edge CPU)
212
+ Export to GGUF format for edge devices using the export tool:
213
+ ```bash
214
+ python scripts/export_gguf.py --model-dir export/QJev-1.5B-Phase3 --out-file qjev-1.5b-f16.gguf
215
+ ./llama-quantize qjev-1.5b-f16.gguf qjev-1.5b-q4_k_m.gguf Q4_K_M
216
+ ```
217
+
218
+ ### 4. vLLM (Production Continuous Batching)
219
+ ```bash
220
+ vllm serve devankit7873/QJev-1.5B --task classify --port 8000
221
+ ```
222
+
223
+ ---
224
+
225
+ ## πŸ—οΈ Model Architecture
226
+
227
+ ```
228
+ [User Context + Formatted Choices]
229
+ β”‚
230
+ β–Ό
231
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
232
+ β”‚ Qwen2.5-1.5B Backbone β”‚ (QLoRA adapted on q, k, v, o projections)
233
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
234
+ β”‚ (Last Hidden State: 1536 dim)
235
+ β–Ό
236
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
237
+ β”‚ LayerNorm(1536) β”‚
238
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
239
+ β”‚
240
+ β–Ό
241
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
242
+ β”‚ Linear(1536 -> 512) β”‚ (GELU + Dropout 0.1)
243
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
244
+ β”‚
245
+ β–Ό
246
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
247
+ β”‚ Linear(512 -> 10) β”‚ (Dynamic Choice Masking)
248
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
249
+ β”‚
250
+ β–Ό
251
+ [Calibrated Softmax Probabilities]
252
+ ```
253
+
254
+ ---
255
+
256
+ ## πŸ“ Repository Structure
257
+
258
+ ```text
259
+ β”œβ”€β”€ README.md # Project showcase, benchmarks, and documentation
260
+ β”œβ”€β”€ LICENSE # Apache 2.0 open-source license
261
+ β”œβ”€β”€ requirements.txt # Python dependencies
262
+ β”œβ”€β”€ Modelfile # Ollama build definition
263
+ β”œβ”€β”€ qjev.py # Universal Python SDK & CLI client
264
+ β”œβ”€β”€ serve.py # Production FastAPI server (REST, Jev, OpenAI)
265
+ β”œβ”€β”€ predict.py # Single-pass forward evaluation demo
266
+ β”œβ”€β”€ handler.py # Hugging Face Inference Endpoint handler
267
+ β”‚
268
+ β”œβ”€β”€ benchmark/ # JevBench self-test and evaluation suites
269
+ β”‚ β”œβ”€β”€ run_jevbench.py # JevBench metamorphic test harness
270
+ β”‚ β”œβ”€β”€ self_test_benchmarkheaven.py # Official BenchmarkHeaven standard runner
271
+ β”‚ β”œβ”€β”€ self_test_phase3.py # Phase 3 consistency verification runner
272
+ β”‚ └── self_test_step1.py # Calibrated temperature sweep suite
273
+ β”‚
274
+ β”œβ”€β”€ reports/ # Empirical evaluation records
275
+ β”‚ └── jevbench/ # JSON benchmark results across phases
276
+ β”‚ β”œβ”€β”€ self_test_phase3_consistency.json # Official 62.53% report
277
+ β”‚ β”œβ”€β”€ jevbench_phase1_report.json
278
+ β”‚ └── jevbench_phase2_report.json
279
+ β”‚
280
+ β”œβ”€β”€ src/ # Core modeling and training code
281
+ β”‚ β”œβ”€β”€ model.py # MLP Decision Head + Qwen architecture
282
+ β”‚ β”œβ”€β”€ dataset.py # Streaming ChatML dataset builder
283
+ β”‚ β”œβ”€β”€ train.py # Phase 1 baseline training
284
+ β”‚ β”œβ”€β”€ train_phase2.py # Phase 2 QLoRA joint training (655k samples)
285
+ β”‚ β”œβ”€β”€ train_phase3_consistency.py # Phase 3 KL consistency alignment
286
+ β”‚ └── evaluate.py # Out-of-Distribution evaluation metrics
287
+ β”‚
288
+ └── scripts/ # Engineering and deployment utilities
289
+ β”œβ”€β”€ build_consistency_dataset.py # 20k paired consistency dataset generator
290
+ β”œβ”€β”€ merge_phase3.py # Phase 3 LoRA fusing & packaging
291
+ β”œβ”€β”€ merge_and_export.py # Base export utility
292
+ β”œβ”€β”€ export_gguf.py # GGUF converter for llama.cpp / LM Studio
293
+ β”œβ”€β”€ publish_all_hf.py # Hugging Face Hub publisher
294
+ β”œβ”€β”€ download_data.py # Dataset downloader
295
+ └── download_model.py # Base model downloader
296
+ ```
297
+
298
+ ---
299
+
300
+ ## πŸ’» Hardware & Training Infrastructure
301
+
302
+ - **Hardware:** HP OMEN Laptop (NVIDIA GeForce RTX 5060 Laptop GPU, 8GB VRAM)
303
+ - **Host OS:** Windows 11 Pro 64-bit
304
+ - **CUDA / Driver:** CUDA 12.8 / Driver 621.24
305
+ - **Precision:** Float32 Matmul Precision (`high`) with Tensor Core TF32 acceleration
306
+ - **Training Trajectory:**
307
+ - **Phase 1:** MLP Head alignment on frozen backbone (1.5 hrs).
308
+ - **Phase 2:** Joint QLoRA + MLP training across 655,806 samples (18.5 hrs, strictly 5.9 GB VRAM).
309
+ - **Phase 3:** Paired consistency KL alignment (27 mins, 1,250 steps, batch size 16).
310
+
311
+ ---
312
+
313
+ ## πŸ“š Citation
314
+
315
+ If you use **QJev-1.5B** or its benchmark methodology, please cite:
316
+
317
+ ```bibtex
318
+ @misc{nayak2026qjev15b,
319
+ author = {Ankit Nayak},
320
+ title = {QJev-1.5B: High-Speed System 1 JEV Decision Model based on Qwen2.5},
321
+ year = {2026},
322
+ publisher = {GitHub and Hugging Face},
323
+ howpublished = {\url{https://github.com/AnkitNayak-dev/QJev-1.5B}}
324
+ }
325
+ ```
326
+
327
+ ---
328
+
329
+ ## πŸ“„ License
330
+ This model and codebase are released under the **Apache 2.0 License**.
331
+ Base weights are governed by Alibaba Cloud Qwen License. Distillation corpus is provided by SargeDev (`jev-distill-corpus-v3`).