devankit7873 commited on
Commit
3abd7fd
ยท
verified ยท
1 Parent(s): e1c53f3

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +95 -112
README.md CHANGED
@@ -32,7 +32,7 @@ pipeline_tag: text-classification
32
  ---
33
 
34
  ### ๐Ÿ“ฆ Pre-Trained Weights & Standalone Models:
35
- ### [๐Ÿค— Hugging Face Model Hub: devankit7873/QJev-1.5B](https://huggingface.co/devankit7873/QJev-1.5B)
36
 
37
  `model.safetensors` (3.08 GB Standalone)  โ€ข  `decision_head.safetensors` (3.18 MB)  โ€ข  `lora_adapter/` (17.5 MB PEFT)
38
 
@@ -40,21 +40,18 @@ pipeline_tag: text-classification
40
 
41
  </div>
42
 
43
- **`QJev-1.5B`** (**Qwen2.5-1.5B JEV Decision Model**) is an open-weights, high-speed **System 1 Decision Model** built on `Qwen/Qwen2.5-1.5B-Instruct`. It attaches a calibrated **Multi-Layer Perceptron (MLP) Decision Head** and fine-tunes attention projection layers (`q_proj`, `k_proj`, `v_proj`, `o_proj`) via **QLoRA** across all 655,806 samples of `SargeDev/jev-distill-corpus-v3`.
44
 
45
- Instead of slow, non-deterministic token-by-token generation (e.g. generating `"Option A"` over 300โ€“600 ms), this model outputs a **mathematically calibrated probability distribution** over up to 10 choices in a **single forward pass (~31 ms)**.
46
 
47
- ---
48
-
49
- ## What is in This Repository?
50
-
51
- We provide **both** distribution formats in a single repository:
52
- 1. **Full Merged Standalone Model (`model.safetensors`, 3.08 GB):** The LoRA attention weights are permanently fused into the base transformer. Runs directly with `transformers` without needing `peft`.
53
- 2. **Lightweight LoRA Adapter (`lora_adapter/`, 16.6 MB):** Included in the `lora_adapter/` subfolder for edge and mobile developers who already host base Qwen and only want the PEFT weights.
54
 
55
  ---
56
 
57
- ## JEV Ecosystem Comparison (How We Stack Up)
58
 
59
  | Model | Parameters | Target Hardware | Decision Latency | Choice Flexibility | JevBench Score | OOD Accuracy | Memory Required |
60
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
@@ -64,14 +61,9 @@ We provide **both** distribution formats in a single repository:
64
  | **`autotrust/JEV-27B`** | 27.0B | Datacenter (A100) | ~150 โ€“ 350 ms | Arbitrary Choices ($\le 16$) | ~76.5% | ~86.2% | ~55.0 GB |
65
  | **`TypeSafe Jev 1.13` (Teacher)** | ~30B+ | Cloud API | ~200 ms+ | Arbitrary Choices | ~72.1% | ~88.0% | Closed API |
66
 
67
- ### Why Choose This 1.5B Model?
68
- - **The "Goldilocks" Sweet Spot:** Captures **~95% of the decision capability of 9B models** while being **6x smaller** and **30%โ€“65% faster**.
69
- - **Arbitrary Multi-Choice Support:** Unlike 0.5B students that only do binary yes/no, this model dynamically handles 2, 3, 5, or up to 10 choices.
70
- - **Consumer Hardware Friendly:** Runs in **3.4 GB VRAM**, making it the only high-capacity JEV model deployable on standard laptops, edge robots, and cheap $5/month cloud VPS instances.
71
-
72
  ---
73
 
74
- ## Official JevBench Metamorphic Benchmark Results
75
 
76
  Evaluated against the official **[JevBench](https://jevbench.github.io/)** test harness (`jevbench-mini`, 240 real-world decision cases, 50 scored metamorphic relations):
77
 
@@ -87,122 +79,113 @@ Evaluated against the official **[JevBench](https://jevbench.github.io/)** test
87
  | **OVERALL JEVBENCH SCORE (Calibrated 0.10)** | **60.2%** `[57โ€“63]` | **71.5%** `[69โ€“74]` | **+11.3%** | **Substantial leap across all 50 relations** |
88
  | **OVERALL JEVBENCH SCORE (Strict 0.05)** | **48.8%** `[47โ€“51]` | **58.6%** `[56โ€“61]` | **+9.8%** | **Strict zero-variance compliance** |
89
 
90
- Full machine-readable reports are available in `reports/jevbench_aligned/`.
91
-
92
  ---
93
 
94
- ## Dataset Benchmark Trajectory (655,806 Samples)
 
 
 
 
 
95
 
96
- Evaluated on fixed 1,500-sample test (in-distribution) and Out-of-Distribution (OOD) benchmark splits from `SargeDev/jev-distill-corpus-v3`:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
97
 
98
- | Metric | Phase 1 (Frozen Backbone Baseline) | Phase 2 (QLoRA Joint Training) | Delta / Improvement |
99
- | :--- | :---: | :---: | :---: |
100
- | **OOD Accuracy (`ood`)** | 57.53% | **80.07%** | **+22.54%** |
101
- | **OOD Total Variation Distance (TVD)** | 0.4607 | **0.2389** | **-48.1% (Calibrated)** |
102
- | **OOD KL Loss** | 0.7884 | **0.6972** | **-11.6%** |
103
- | **In-Distribution TVD (`test`)** | 0.1664 | **0.0921** | **-44.6% (Sub-9% deviation)** |
104
- | **In-Distribution Loss** | 0.1757 | **0.1033** | **-41.2%** |
105
- | **Inference Latency** | 42.94 ms | **31.49 ms** | **26.7% faster** |
 
 
 
 
 
 
 
 
 
 
 
106
 
107
  ---
108
 
109
- ## Hardware & Training Infrastructure
110
 
111
  Trained from scratch locally on consumer silicon:
112
  - **Host System:** HP OMEN Gaming Laptop (Windows 11 Pro 64-bit)
113
  - **GPU:** NVIDIA GeForce RTX 5060 Laptop GPU (8GB VRAM)
114
  - **CUDA / Driver:** CUDA 12.8 / Driver 621.24 (CUDA UMD 13.5)
115
  - **Framework:** PyTorch `2.12.0.dev20260408+cu128`, `transformers 5.22.0`, `peft 0.21.2`
116
- - **Precision:** Float32 Matmul Precision set to `high` with Tensor Core **TF32** acceleration
117
  - **VRAM Footprint:** Strictly pegged at **5.90 GB / 8.0 GB VRAM** (0 OOM errors over 655k steps)
118
  - **Effective Batch Size:** 8 per device $\times$ 4 gradient accumulation = 32
119
  - **Throughput:** ~11.5โ€“12.0 samples/sec (total ~18.5 GPU compute hours)
120
 
121
  ---
122
 
123
- ## Quickstart & Usage
124
 
125
- ### 1. Installation
126
- ```bash
127
- pip install torch transformers safetensors
128
- ```
129
-
130
- ### 2. Run Inference Demo (1 Command)
131
- ```bash
132
- python predict.py
133
  ```
134
 
135
- ### 3. Python API Integration
136
- ```python
137
- from predict import QJevModel
138
-
139
- # Loads weights directly from Hugging Face or local cache
140
- model = QJevModel(model_id_or_path="devankit7873/QJev-1.5B")
141
-
142
- context = "Production database CPU is spiking at 99%. What is the immediate priority?"
143
- choices = [
144
- "Failover to read replica immediately",
145
- "Restart the primary database instance",
146
- "Increase connection pool limits",
147
- "Ignore the alert and monitor logs"
148
- ]
149
-
150
- result = model.predict(context=context, choices=choices)
151
- print(f"Recommended: {result['recommended']} (Confidence: {result['confidence']})")
152
- for choice, prob in result['probabilities'].items():
153
- print(f" - {choice}: {prob}")
154
- ```
155
-
156
- **Output:**
157
- ```text
158
- Recommended: Failover to read replica immediately (Confidence: 86.72%)
159
- - Failover to read replica immediately: 86.72%
160
- - Restart the primary database instance: 7.03%
161
- - Double connection pool limits without scaling compute: 5.71%
162
- - Silence alerts and monitor for next 15 minutes: 0.50%
163
- ```
164
-
165
- ---
166
-
167
- ## Repository Structure
168
-
169
- ```text
170
- โ”œโ”€โ”€ README.md # Project showcase, benchmarks, and usage
171
- โ”œโ”€โ”€ LICENSE # Apache 2.0 open-source license
172
- โ”œโ”€โ”€ requirements.txt # Python dependencies
173
- โ”œโ”€โ”€ predict.py # 1-click single-pass inference demo
174
- โ”‚
175
- โ”œโ”€โ”€ src/ # Core modeling and training pipeline
176
- โ”‚ โ”œโ”€โ”€ model.py # MLP Decision Head + Qwen integration
177
- โ”‚ โ”œโ”€โ”€ dataset.py # ChatML streaming dataset loader
178
- โ”‚ โ”œโ”€โ”€ train.py # Phase 1 baseline training script
179
- โ”‚ โ”œโ”€โ”€ train_phase2.py # Phase 2 QLoRA joint training script (655k steps)
180
- โ”‚ โ””โ”€โ”€ evaluate.py # Evaluation metrics (TVD, Accuracy, KL loss)
181
- โ”‚
182
- โ”œโ”€โ”€ benchmark/ # Evaluation suites
183
- โ”‚ โ””โ”€โ”€ run_jevbench.py # Official JevBench metamorphic evaluation runner
184
- โ”‚
185
- โ”œโ”€โ”€ reports/ # Complete empirical benchmark records
186
- โ”‚ โ”œโ”€โ”€ jevbench_aligned/ # Official JevBench scores (71.5% calibrated)
187
- โ”‚ โ”œโ”€โ”€ phase_1/ # Phase 1 baseline evaluation records
188
- โ”‚ โ””โ”€โ”€ phase_2/ # Phase 2 evaluations across all 50k checkpoints
189
- โ”‚
190
- โ””โ”€โ”€ scripts/ # Workflow utilities
191
- โ”œโ”€โ”€ merge_and_export.py # LoRA fusing and safetensors export
192
- โ”œโ”€โ”€ download_data.py # Dataset loader from Hugging Face
193
- โ”œโ”€โ”€ download_model.py # Base model downloader
194
- โ””โ”€โ”€ upload_to_hf.py # Automated Hugging Face publisher
195
- ```
196
-
197
- ---
198
-
199
- ## Training Tracking & Reproducibility
200
- All training logs, step trajectories, and checkpoint evaluations were recorded continuously:
201
- - **Weights & Biases Dashboard:** [`phase-2-lora-joint-run`](https://forge.coreweave.com/wandb/devankit/qwen-decision-head/runs/phase-2-lora-joint-run)
202
- - **Source Code & Pipelines:** [github.com/AnkitNayak-dev/QJev-1.5B](https://github.com/AnkitNayak-dev/QJev-1.5B)
203
- - **Model Weights & Hub:** [huggingface.co/devankit7873/QJev-1.5B](https://huggingface.co/devankit7873/QJev-1.5B)
204
-
205
  ---
206
 
207
- ## License
208
- This model and codebase are released under the **Apache 2.0 License**.
 
 
32
  ---
33
 
34
  ### ๐Ÿ“ฆ Pre-Trained Weights & Standalone Models:
35
+ ### [GitHub Codebase & Reproduction: github.com/AnkitNayak-dev/QJev-1.5B](https://github.com/AnkitNayak-dev/QJev-1.5B)
36
 
37
  `model.safetensors` (3.08 GB Standalone) &nbsp;โ€ข&nbsp; `decision_head.safetensors` (3.18 MB) &nbsp;โ€ข&nbsp; `lora_adapter/` (17.5 MB PEFT)
38
 
 
40
 
41
  </div>
42
 
43
+ ## ๐Ÿ’ก Overview: System 1 Decision Paradigm
44
 
45
+ Traditional LLMs make decisions through slow, non-deterministic token-by-token generation (e.g. generating `"Option A"` over 300โ€“600 ms).
46
 
47
+ **`QJev-1.5B`** introduces a **System 1 Decision Architecture**:
48
+ 1. Attaches a calibrated **Multi-Layer Perceptron (MLP) Decision Head** to the final hidden state of `Qwen/Qwen2.5-1.5B-Instruct`.
49
+ 2. Jointly trains attention projection layers (`q_proj`, `k_proj`, `v_proj`, `o_proj`) via **QLoRA** and the decision head across all 655,806 distillation samples from `SargeDev/jev-distill-corpus-v3`.
50
+ 3. Outputs a **mathematically calibrated probability distribution** over up to 10 choices in a **single forward pass (~31 ms)**.
 
 
 
51
 
52
  ---
53
 
54
+ ## โš–๏ธ JEV Ecosystem Comparison
55
 
56
  | Model | Parameters | Target Hardware | Decision Latency | Choice Flexibility | JevBench Score | OOD Accuracy | Memory Required |
57
  | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
 
61
  | **`autotrust/JEV-27B`** | 27.0B | Datacenter (A100) | ~150 โ€“ 350 ms | Arbitrary Choices ($\le 16$) | ~76.5% | ~86.2% | ~55.0 GB |
62
  | **`TypeSafe Jev 1.13` (Teacher)** | ~30B+ | Cloud API | ~200 ms+ | Arbitrary Choices | ~72.1% | ~88.0% | Closed API |
63
 
 
 
 
 
 
64
  ---
65
 
66
+ ## ๐Ÿ† Official JevBench Metamorphic Benchmark Results
67
 
68
  Evaluated against the official **[JevBench](https://jevbench.github.io/)** test harness (`jevbench-mini`, 240 real-world decision cases, 50 scored metamorphic relations):
69
 
 
79
  | **OVERALL JEVBENCH SCORE (Calibrated 0.10)** | **60.2%** `[57โ€“63]` | **71.5%** `[69โ€“74]` | **+11.3%** | **Substantial leap across all 50 relations** |
80
  | **OVERALL JEVBENCH SCORE (Strict 0.05)** | **48.8%** `[47โ€“51]` | **58.6%** `[56โ€“61]` | **+9.8%** | **Strict zero-variance compliance** |
81
 
 
 
82
  ---
83
 
84
+ ## ๐Ÿš€ Quickstart: Direct Inference from Hugging Face
85
+
86
+ ### 1. Installation
87
+ ```bash
88
+ pip install torch transformers safetensors huggingface_hub
89
+ ```
90
 
91
+ ### 2. Standalone Inference Code
92
+ ```python
93
+ import os
94
+ import torch
95
+ import torch.nn as nn
96
+ import torch.nn.functional as F
97
+ from transformers import AutoTokenizer, AutoModelForCausalLM
98
+ from safetensors.torch import load_file
99
+ from huggingface_hub import hf_hub_download
100
+
101
+ class DecisionHead(nn.Module):
102
+ def __init__(self, hidden_size: int = 1536, intermediate_size: int = 512, max_choices: int = 10, dropout: float = 0.1):
103
+ super().__init__()
104
+ self.max_choices = max_choices
105
+ self.norm = nn.LayerNorm(hidden_size)
106
+ self.dense = nn.Linear(hidden_size, intermediate_size)
107
+ self.act = nn.GELU()
108
+ self.dropout = nn.Dropout(dropout)
109
+ self.out_proj = nn.Linear(intermediate_size, max_choices)
110
+
111
+ def forward(self, x: torch.Tensor, num_choices: torch.Tensor = None) -> torch.Tensor:
112
+ h = self.dropout(self.act(self.dense(self.norm(x))))
113
+ logits = self.out_proj(h)
114
+ if num_choices is not None:
115
+ mask = torch.arange(self.max_choices, device=logits.device).unsqueeze(0) >= num_choices.unsqueeze(1)
116
+ logits = logits.masked_fill(mask, -1e4)
117
+ return logits
118
+
119
+ # 1. Load model & weights
120
+ model_id = "devankit7873/QJev-1.5B"
121
+ device = "cuda" if torch.cuda.is_available() else "cpu"
122
+
123
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
124
+ backbone = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map=device)
125
+ head = DecisionHead().to(device=device, dtype=torch.bfloat16)
126
+
127
+ head_path = hf_hub_download(repo_id=model_id, filename="decision_head.safetensors")
128
+ head.load_state_dict(load_file(head_path))
129
+ head.eval()
130
+
131
+ # 2. Formulate Prompt & Query
132
+ context = "Production database primary replica CPU spiked to 100% and connection pool is exhausted with 5,000 waiting queries."
133
+ choices = [
134
+ "Trigger immediate automated failover to standby replica",
135
+ "Restart the primary database daemon immediately",
136
+ "Double connection pool limits without scaling compute",
137
+ "Silence alerts and monitor for next 15 minutes"
138
+ ]
139
 
140
+ opts_str = "\n".join([f"[{i+1}] {c}" for i, c in enumerate(choices)])
141
+ prompt = (
142
+ "<|im_start|>system\n"
143
+ "You are an expert decision engine. Evaluate the context and select the optimal decision.<|im_end|>\n"
144
+ f"<|im_start|>user\nContext:\n{context}\n\nQuestion:\nSelect the most appropriate option:\n\nOptions:\n{opts_str}<|im_end|>\n"
145
+ "<|im_start|>assistant\nDecision:"
146
+ )
147
+
148
+ # 3. Predict in 31 ms forward pass
149
+ inputs = tokenizer(prompt, return_tensors="pt").to(device)
150
+ with torch.no_grad():
151
+ transformer = getattr(backbone, "model", backbone)
152
+ out = transformer(inputs["input_ids"], attention_mask=inputs["attention_mask"])
153
+ logits = head(out.last_hidden_state[:, -1, :], num_choices=torch.tensor([len(choices)], device=device))
154
+ probs = F.softmax(logits[0, :len(choices)], dim=-1).cpu().tolist()
155
+
156
+ for choice, prob in zip(choices, probs):
157
+ print(f"{choice}: {prob * 100:.2f}%")
158
+ ```
159
 
160
  ---
161
 
162
+ ## ๐Ÿ’ป Hardware & Training Infrastructure
163
 
164
  Trained from scratch locally on consumer silicon:
165
  - **Host System:** HP OMEN Gaming Laptop (Windows 11 Pro 64-bit)
166
  - **GPU:** NVIDIA GeForce RTX 5060 Laptop GPU (8GB VRAM)
167
  - **CUDA / Driver:** CUDA 12.8 / Driver 621.24 (CUDA UMD 13.5)
168
  - **Framework:** PyTorch `2.12.0.dev20260408+cu128`, `transformers 5.22.0`, `peft 0.21.2`
 
169
  - **VRAM Footprint:** Strictly pegged at **5.90 GB / 8.0 GB VRAM** (0 OOM errors over 655k steps)
170
  - **Effective Batch Size:** 8 per device $\times$ 4 gradient accumulation = 32
171
  - **Throughput:** ~11.5โ€“12.0 samples/sec (total ~18.5 GPU compute hours)
172
 
173
  ---
174
 
175
+ ## ๐Ÿ“š Citation
176
 
177
+ ```bibtex
178
+ @misc{nayak2026qjev15b,
179
+ author = {Ankit Nayak},
180
+ title = {QJev-1.5B: High-Speed System 1 JEV Decision Model based on Qwen2.5},
181
+ year = {2026},
182
+ publisher = {GitHub and Hugging Face},
183
+ howpublished = {\url{https://github.com/AnkitNayak-dev/QJev-1.5B}}
184
+ }
185
  ```
186
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
187
  ---
188
 
189
+ ## ๐Ÿ“„ License
190
+ Released under the **Apache 2.0 License**.
191
+ Base weights governed by Alibaba Cloud Qwen License. Distillation corpus by SargeDev (`jev-distill-corpus-v3`).