Text Classification
Transformers
Safetensors
PyTorch
qwen2
text-generation
causal-lm
qwen
decision-making
system-one
reasoning
jev
jevbench
joint-expected-value
classification
qlora
agent
decision-model
fast-inference
text-embeddings-inference
Instructions to use devankit7873/QJev-1.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use devankit7873/QJev-1.5B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="devankit7873/QJev-1.5B")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("devankit7873/QJev-1.5B") model = AutoModelForCausalLM.from_pretrained("devankit7873/QJev-1.5B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -32,7 +32,7 @@ pipeline_tag: text-classification
|
|
| 32 |
---
|
| 33 |
|
| 34 |
### ๐ฆ Pre-Trained Weights & Standalone Models:
|
| 35 |
-
### [
|
| 36 |
|
| 37 |
`model.safetensors` (3.08 GB Standalone) โข `decision_head.safetensors` (3.18 MB) โข `lora_adapter/` (17.5 MB PEFT)
|
| 38 |
|
|
@@ -40,21 +40,18 @@ pipeline_tag: text-classification
|
|
| 40 |
|
| 41 |
</div>
|
| 42 |
|
| 43 |
-
|
| 44 |
|
| 45 |
-
|
| 46 |
|
| 47 |
-
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
We provide **both** distribution formats in a single repository:
|
| 52 |
-
1. **Full Merged Standalone Model (`model.safetensors`, 3.08 GB):** The LoRA attention weights are permanently fused into the base transformer. Runs directly with `transformers` without needing `peft`.
|
| 53 |
-
2. **Lightweight LoRA Adapter (`lora_adapter/`, 16.6 MB):** Included in the `lora_adapter/` subfolder for edge and mobile developers who already host base Qwen and only want the PEFT weights.
|
| 54 |
|
| 55 |
---
|
| 56 |
|
| 57 |
-
## JEV Ecosystem Comparison
|
| 58 |
|
| 59 |
| Model | Parameters | Target Hardware | Decision Latency | Choice Flexibility | JevBench Score | OOD Accuracy | Memory Required |
|
| 60 |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
|
@@ -64,14 +61,9 @@ We provide **both** distribution formats in a single repository:
|
|
| 64 |
| **`autotrust/JEV-27B`** | 27.0B | Datacenter (A100) | ~150 โ 350 ms | Arbitrary Choices ($\le 16$) | ~76.5% | ~86.2% | ~55.0 GB |
|
| 65 |
| **`TypeSafe Jev 1.13` (Teacher)** | ~30B+ | Cloud API | ~200 ms+ | Arbitrary Choices | ~72.1% | ~88.0% | Closed API |
|
| 66 |
|
| 67 |
-
### Why Choose This 1.5B Model?
|
| 68 |
-
- **The "Goldilocks" Sweet Spot:** Captures **~95% of the decision capability of 9B models** while being **6x smaller** and **30%โ65% faster**.
|
| 69 |
-
- **Arbitrary Multi-Choice Support:** Unlike 0.5B students that only do binary yes/no, this model dynamically handles 2, 3, 5, or up to 10 choices.
|
| 70 |
-
- **Consumer Hardware Friendly:** Runs in **3.4 GB VRAM**, making it the only high-capacity JEV model deployable on standard laptops, edge robots, and cheap $5/month cloud VPS instances.
|
| 71 |
-
|
| 72 |
---
|
| 73 |
|
| 74 |
-
## Official JevBench Metamorphic Benchmark Results
|
| 75 |
|
| 76 |
Evaluated against the official **[JevBench](https://jevbench.github.io/)** test harness (`jevbench-mini`, 240 real-world decision cases, 50 scored metamorphic relations):
|
| 77 |
|
|
@@ -87,122 +79,113 @@ Evaluated against the official **[JevBench](https://jevbench.github.io/)** test
|
|
| 87 |
| **OVERALL JEVBENCH SCORE (Calibrated 0.10)** | **60.2%** `[57โ63]` | **71.5%** `[69โ74]` | **+11.3%** | **Substantial leap across all 50 relations** |
|
| 88 |
| **OVERALL JEVBENCH SCORE (Strict 0.05)** | **48.8%** `[47โ51]` | **58.6%** `[56โ61]` | **+9.8%** | **Strict zero-variance compliance** |
|
| 89 |
|
| 90 |
-
Full machine-readable reports are available in `reports/jevbench_aligned/`.
|
| 91 |
-
|
| 92 |
---
|
| 93 |
|
| 94 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 95 |
|
| 96 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 97 |
|
| 98 |
-
|
| 99 |
-
|
| 100 |
-
|
|
| 101 |
-
|
| 102 |
-
|
|
| 103 |
-
|
|
| 104 |
-
|
| 105 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 106 |
|
| 107 |
---
|
| 108 |
|
| 109 |
-
## Hardware & Training Infrastructure
|
| 110 |
|
| 111 |
Trained from scratch locally on consumer silicon:
|
| 112 |
- **Host System:** HP OMEN Gaming Laptop (Windows 11 Pro 64-bit)
|
| 113 |
- **GPU:** NVIDIA GeForce RTX 5060 Laptop GPU (8GB VRAM)
|
| 114 |
- **CUDA / Driver:** CUDA 12.8 / Driver 621.24 (CUDA UMD 13.5)
|
| 115 |
- **Framework:** PyTorch `2.12.0.dev20260408+cu128`, `transformers 5.22.0`, `peft 0.21.2`
|
| 116 |
-
- **Precision:** Float32 Matmul Precision set to `high` with Tensor Core **TF32** acceleration
|
| 117 |
- **VRAM Footprint:** Strictly pegged at **5.90 GB / 8.0 GB VRAM** (0 OOM errors over 655k steps)
|
| 118 |
- **Effective Batch Size:** 8 per device $\times$ 4 gradient accumulation = 32
|
| 119 |
- **Throughput:** ~11.5โ12.0 samples/sec (total ~18.5 GPU compute hours)
|
| 120 |
|
| 121 |
---
|
| 122 |
|
| 123 |
-
##
|
| 124 |
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
| 129 |
-
|
| 130 |
-
|
| 131 |
-
|
| 132 |
-
|
| 133 |
```
|
| 134 |
|
| 135 |
-
### 3. Python API Integration
|
| 136 |
-
```python
|
| 137 |
-
from predict import QJevModel
|
| 138 |
-
|
| 139 |
-
# Loads weights directly from Hugging Face or local cache
|
| 140 |
-
model = QJevModel(model_id_or_path="devankit7873/QJev-1.5B")
|
| 141 |
-
|
| 142 |
-
context = "Production database CPU is spiking at 99%. What is the immediate priority?"
|
| 143 |
-
choices = [
|
| 144 |
-
"Failover to read replica immediately",
|
| 145 |
-
"Restart the primary database instance",
|
| 146 |
-
"Increase connection pool limits",
|
| 147 |
-
"Ignore the alert and monitor logs"
|
| 148 |
-
]
|
| 149 |
-
|
| 150 |
-
result = model.predict(context=context, choices=choices)
|
| 151 |
-
print(f"Recommended: {result['recommended']} (Confidence: {result['confidence']})")
|
| 152 |
-
for choice, prob in result['probabilities'].items():
|
| 153 |
-
print(f" - {choice}: {prob}")
|
| 154 |
-
```
|
| 155 |
-
|
| 156 |
-
**Output:**
|
| 157 |
-
```text
|
| 158 |
-
Recommended: Failover to read replica immediately (Confidence: 86.72%)
|
| 159 |
-
- Failover to read replica immediately: 86.72%
|
| 160 |
-
- Restart the primary database instance: 7.03%
|
| 161 |
-
- Double connection pool limits without scaling compute: 5.71%
|
| 162 |
-
- Silence alerts and monitor for next 15 minutes: 0.50%
|
| 163 |
-
```
|
| 164 |
-
|
| 165 |
-
---
|
| 166 |
-
|
| 167 |
-
## Repository Structure
|
| 168 |
-
|
| 169 |
-
```text
|
| 170 |
-
โโโ README.md # Project showcase, benchmarks, and usage
|
| 171 |
-
โโโ LICENSE # Apache 2.0 open-source license
|
| 172 |
-
โโโ requirements.txt # Python dependencies
|
| 173 |
-
โโโ predict.py # 1-click single-pass inference demo
|
| 174 |
-
โ
|
| 175 |
-
โโโ src/ # Core modeling and training pipeline
|
| 176 |
-
โ โโโ model.py # MLP Decision Head + Qwen integration
|
| 177 |
-
โ โโโ dataset.py # ChatML streaming dataset loader
|
| 178 |
-
โ โโโ train.py # Phase 1 baseline training script
|
| 179 |
-
โ โโโ train_phase2.py # Phase 2 QLoRA joint training script (655k steps)
|
| 180 |
-
โ โโโ evaluate.py # Evaluation metrics (TVD, Accuracy, KL loss)
|
| 181 |
-
โ
|
| 182 |
-
โโโ benchmark/ # Evaluation suites
|
| 183 |
-
โ โโโ run_jevbench.py # Official JevBench metamorphic evaluation runner
|
| 184 |
-
โ
|
| 185 |
-
โโโ reports/ # Complete empirical benchmark records
|
| 186 |
-
โ โโโ jevbench_aligned/ # Official JevBench scores (71.5% calibrated)
|
| 187 |
-
โ โโโ phase_1/ # Phase 1 baseline evaluation records
|
| 188 |
-
โ โโโ phase_2/ # Phase 2 evaluations across all 50k checkpoints
|
| 189 |
-
โ
|
| 190 |
-
โโโ scripts/ # Workflow utilities
|
| 191 |
-
โโโ merge_and_export.py # LoRA fusing and safetensors export
|
| 192 |
-
โโโ download_data.py # Dataset loader from Hugging Face
|
| 193 |
-
โโโ download_model.py # Base model downloader
|
| 194 |
-
โโโ upload_to_hf.py # Automated Hugging Face publisher
|
| 195 |
-
```
|
| 196 |
-
|
| 197 |
-
---
|
| 198 |
-
|
| 199 |
-
## Training Tracking & Reproducibility
|
| 200 |
-
All training logs, step trajectories, and checkpoint evaluations were recorded continuously:
|
| 201 |
-
- **Weights & Biases Dashboard:** [`phase-2-lora-joint-run`](https://forge.coreweave.com/wandb/devankit/qwen-decision-head/runs/phase-2-lora-joint-run)
|
| 202 |
-
- **Source Code & Pipelines:** [github.com/AnkitNayak-dev/QJev-1.5B](https://github.com/AnkitNayak-dev/QJev-1.5B)
|
| 203 |
-
- **Model Weights & Hub:** [huggingface.co/devankit7873/QJev-1.5B](https://huggingface.co/devankit7873/QJev-1.5B)
|
| 204 |
-
|
| 205 |
---
|
| 206 |
|
| 207 |
-
## License
|
| 208 |
-
|
|
|
|
|
|
| 32 |
---
|
| 33 |
|
| 34 |
### ๐ฆ Pre-Trained Weights & Standalone Models:
|
| 35 |
+
### [GitHub Codebase & Reproduction: github.com/AnkitNayak-dev/QJev-1.5B](https://github.com/AnkitNayak-dev/QJev-1.5B)
|
| 36 |
|
| 37 |
`model.safetensors` (3.08 GB Standalone) โข `decision_head.safetensors` (3.18 MB) โข `lora_adapter/` (17.5 MB PEFT)
|
| 38 |
|
|
|
|
| 40 |
|
| 41 |
</div>
|
| 42 |
|
| 43 |
+
## ๐ก Overview: System 1 Decision Paradigm
|
| 44 |
|
| 45 |
+
Traditional LLMs make decisions through slow, non-deterministic token-by-token generation (e.g. generating `"Option A"` over 300โ600 ms).
|
| 46 |
|
| 47 |
+
**`QJev-1.5B`** introduces a **System 1 Decision Architecture**:
|
| 48 |
+
1. Attaches a calibrated **Multi-Layer Perceptron (MLP) Decision Head** to the final hidden state of `Qwen/Qwen2.5-1.5B-Instruct`.
|
| 49 |
+
2. Jointly trains attention projection layers (`q_proj`, `k_proj`, `v_proj`, `o_proj`) via **QLoRA** and the decision head across all 655,806 distillation samples from `SargeDev/jev-distill-corpus-v3`.
|
| 50 |
+
3. Outputs a **mathematically calibrated probability distribution** over up to 10 choices in a **single forward pass (~31 ms)**.
|
|
|
|
|
|
|
|
|
|
| 51 |
|
| 52 |
---
|
| 53 |
|
| 54 |
+
## โ๏ธ JEV Ecosystem Comparison
|
| 55 |
|
| 56 |
| Model | Parameters | Target Hardware | Decision Latency | Choice Flexibility | JevBench Score | OOD Accuracy | Memory Required |
|
| 57 |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
|
|
|
| 61 |
| **`autotrust/JEV-27B`** | 27.0B | Datacenter (A100) | ~150 โ 350 ms | Arbitrary Choices ($\le 16$) | ~76.5% | ~86.2% | ~55.0 GB |
|
| 62 |
| **`TypeSafe Jev 1.13` (Teacher)** | ~30B+ | Cloud API | ~200 ms+ | Arbitrary Choices | ~72.1% | ~88.0% | Closed API |
|
| 63 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
---
|
| 65 |
|
| 66 |
+
## ๐ Official JevBench Metamorphic Benchmark Results
|
| 67 |
|
| 68 |
Evaluated against the official **[JevBench](https://jevbench.github.io/)** test harness (`jevbench-mini`, 240 real-world decision cases, 50 scored metamorphic relations):
|
| 69 |
|
|
|
|
| 79 |
| **OVERALL JEVBENCH SCORE (Calibrated 0.10)** | **60.2%** `[57โ63]` | **71.5%** `[69โ74]` | **+11.3%** | **Substantial leap across all 50 relations** |
|
| 80 |
| **OVERALL JEVBENCH SCORE (Strict 0.05)** | **48.8%** `[47โ51]` | **58.6%** `[56โ61]` | **+9.8%** | **Strict zero-variance compliance** |
|
| 81 |
|
|
|
|
|
|
|
| 82 |
---
|
| 83 |
|
| 84 |
+
## ๐ Quickstart: Direct Inference from Hugging Face
|
| 85 |
+
|
| 86 |
+
### 1. Installation
|
| 87 |
+
```bash
|
| 88 |
+
pip install torch transformers safetensors huggingface_hub
|
| 89 |
+
```
|
| 90 |
|
| 91 |
+
### 2. Standalone Inference Code
|
| 92 |
+
```python
|
| 93 |
+
import os
|
| 94 |
+
import torch
|
| 95 |
+
import torch.nn as nn
|
| 96 |
+
import torch.nn.functional as F
|
| 97 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 98 |
+
from safetensors.torch import load_file
|
| 99 |
+
from huggingface_hub import hf_hub_download
|
| 100 |
+
|
| 101 |
+
class DecisionHead(nn.Module):
|
| 102 |
+
def __init__(self, hidden_size: int = 1536, intermediate_size: int = 512, max_choices: int = 10, dropout: float = 0.1):
|
| 103 |
+
super().__init__()
|
| 104 |
+
self.max_choices = max_choices
|
| 105 |
+
self.norm = nn.LayerNorm(hidden_size)
|
| 106 |
+
self.dense = nn.Linear(hidden_size, intermediate_size)
|
| 107 |
+
self.act = nn.GELU()
|
| 108 |
+
self.dropout = nn.Dropout(dropout)
|
| 109 |
+
self.out_proj = nn.Linear(intermediate_size, max_choices)
|
| 110 |
+
|
| 111 |
+
def forward(self, x: torch.Tensor, num_choices: torch.Tensor = None) -> torch.Tensor:
|
| 112 |
+
h = self.dropout(self.act(self.dense(self.norm(x))))
|
| 113 |
+
logits = self.out_proj(h)
|
| 114 |
+
if num_choices is not None:
|
| 115 |
+
mask = torch.arange(self.max_choices, device=logits.device).unsqueeze(0) >= num_choices.unsqueeze(1)
|
| 116 |
+
logits = logits.masked_fill(mask, -1e4)
|
| 117 |
+
return logits
|
| 118 |
+
|
| 119 |
+
# 1. Load model & weights
|
| 120 |
+
model_id = "devankit7873/QJev-1.5B"
|
| 121 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 122 |
+
|
| 123 |
+
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
| 124 |
+
backbone = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map=device)
|
| 125 |
+
head = DecisionHead().to(device=device, dtype=torch.bfloat16)
|
| 126 |
+
|
| 127 |
+
head_path = hf_hub_download(repo_id=model_id, filename="decision_head.safetensors")
|
| 128 |
+
head.load_state_dict(load_file(head_path))
|
| 129 |
+
head.eval()
|
| 130 |
+
|
| 131 |
+
# 2. Formulate Prompt & Query
|
| 132 |
+
context = "Production database primary replica CPU spiked to 100% and connection pool is exhausted with 5,000 waiting queries."
|
| 133 |
+
choices = [
|
| 134 |
+
"Trigger immediate automated failover to standby replica",
|
| 135 |
+
"Restart the primary database daemon immediately",
|
| 136 |
+
"Double connection pool limits without scaling compute",
|
| 137 |
+
"Silence alerts and monitor for next 15 minutes"
|
| 138 |
+
]
|
| 139 |
|
| 140 |
+
opts_str = "\n".join([f"[{i+1}] {c}" for i, c in enumerate(choices)])
|
| 141 |
+
prompt = (
|
| 142 |
+
"<|im_start|>system\n"
|
| 143 |
+
"You are an expert decision engine. Evaluate the context and select the optimal decision.<|im_end|>\n"
|
| 144 |
+
f"<|im_start|>user\nContext:\n{context}\n\nQuestion:\nSelect the most appropriate option:\n\nOptions:\n{opts_str}<|im_end|>\n"
|
| 145 |
+
"<|im_start|>assistant\nDecision:"
|
| 146 |
+
)
|
| 147 |
+
|
| 148 |
+
# 3. Predict in 31 ms forward pass
|
| 149 |
+
inputs = tokenizer(prompt, return_tensors="pt").to(device)
|
| 150 |
+
with torch.no_grad():
|
| 151 |
+
transformer = getattr(backbone, "model", backbone)
|
| 152 |
+
out = transformer(inputs["input_ids"], attention_mask=inputs["attention_mask"])
|
| 153 |
+
logits = head(out.last_hidden_state[:, -1, :], num_choices=torch.tensor([len(choices)], device=device))
|
| 154 |
+
probs = F.softmax(logits[0, :len(choices)], dim=-1).cpu().tolist()
|
| 155 |
+
|
| 156 |
+
for choice, prob in zip(choices, probs):
|
| 157 |
+
print(f"{choice}: {prob * 100:.2f}%")
|
| 158 |
+
```
|
| 159 |
|
| 160 |
---
|
| 161 |
|
| 162 |
+
## ๐ป Hardware & Training Infrastructure
|
| 163 |
|
| 164 |
Trained from scratch locally on consumer silicon:
|
| 165 |
- **Host System:** HP OMEN Gaming Laptop (Windows 11 Pro 64-bit)
|
| 166 |
- **GPU:** NVIDIA GeForce RTX 5060 Laptop GPU (8GB VRAM)
|
| 167 |
- **CUDA / Driver:** CUDA 12.8 / Driver 621.24 (CUDA UMD 13.5)
|
| 168 |
- **Framework:** PyTorch `2.12.0.dev20260408+cu128`, `transformers 5.22.0`, `peft 0.21.2`
|
|
|
|
| 169 |
- **VRAM Footprint:** Strictly pegged at **5.90 GB / 8.0 GB VRAM** (0 OOM errors over 655k steps)
|
| 170 |
- **Effective Batch Size:** 8 per device $\times$ 4 gradient accumulation = 32
|
| 171 |
- **Throughput:** ~11.5โ12.0 samples/sec (total ~18.5 GPU compute hours)
|
| 172 |
|
| 173 |
---
|
| 174 |
|
| 175 |
+
## ๐ Citation
|
| 176 |
|
| 177 |
+
```bibtex
|
| 178 |
+
@misc{nayak2026qjev15b,
|
| 179 |
+
author = {Ankit Nayak},
|
| 180 |
+
title = {QJev-1.5B: High-Speed System 1 JEV Decision Model based on Qwen2.5},
|
| 181 |
+
year = {2026},
|
| 182 |
+
publisher = {GitHub and Hugging Face},
|
| 183 |
+
howpublished = {\url{https://github.com/AnkitNayak-dev/QJev-1.5B}}
|
| 184 |
+
}
|
| 185 |
```
|
| 186 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 187 |
---
|
| 188 |
|
| 189 |
+
## ๐ License
|
| 190 |
+
Released under the **Apache 2.0 License**.
|
| 191 |
+
Base weights governed by Alibaba Cloud Qwen License. Distillation corpus by SargeDev (`jev-distill-corpus-v3`).
|