QJev-1.5B / README.md
devankit7873's picture
Upload README.md with huggingface_hub
0c4b9e8 verified
|
Raw History Blame Contribute Delete
6.61 kB
---
license: apache-2.0
base_model: Qwen/Qwen2.5-1.5B-Instruct
library_name: transformers
tags:
- transformers
- pytorch
- causal-lm
- qwen
- qwen2
- decision-making
- system-one
- reasoning
- jev
- jevbench
- joint-expected-value
- classification
- safetensors
- qlora
- agent
- decision-model
- text-classification
- fast-inference
datasets:
- SargeDev/jev-distill-corpus-v3
metrics:
- accuracy
- tvd
- kl_divergence
pipeline_tag: text-classification
---
<div align="center">
# ⚑ QJev-1.5B: Qwen2.5-1.5B JEV Decision Model
### High-Speed System 1 Real-Time Decision Engine (~31 ms Forward Pass)
[![Hugging Face](https://img.shields.io/badge/Hugging%20Face-QJev--1.5B-FFD21E?style=for-the-badge&logo=huggingface&logoColor=black)](https://huggingface.co/devankit7873/QJev-1.5B)
[![GitHub](https://img.shields.io/badge/GitHub-QJev--1.5B-181717?style=for-the-badge&logo=github)](https://github.com/AnkitNayak-dev/QJev-1.5B)
[![JevBench](https://img.shields.io/badge/JevBench-71.5%25%20Score-8A2BE2?style=for-the-badge)](https://jevbench.github.io/)
[![License](https://img.shields.io/badge/License-Apache%202.0-2EA44F?style=for-the-badge)](LICENSE)
---
### πŸ“¦ QJev Model Family (Choose Your Format):
- **Full Fused Standalone:** [devankit7873/QJev-1.5B](https://huggingface.co/devankit7873/QJev-1.5B) (3.08 GB β€’ Zero PEFT dependency)
- **Lightweight LoRA Adapter:** [devankit7873/QJev-1.5B-LoRA](https://huggingface.co/devankit7873/QJev-1.5B-LoRA) (20 MB β€’ Modular edge loading)
- **4-Bit NF4 Quantized:** [devankit7873/QJev-1.5B-4bit](https://huggingface.co/devankit7873/QJev-1.5B-4bit) (1.15 GB β€’ Ultra-low 1.07 GB VRAM)
---
</div>
## πŸ’‘ Overview: System 1 Decision Paradigm
Traditional LLMs make decisions through slow, non-deterministic token-by-token generation (e.g. generating `"Option A"` over 300–600 ms).
**`QJev-1.5B`** introduces a **System 1 Decision Architecture**:
1. Attaches a calibrated **Multi-Layer Perceptron (MLP) Decision Head** to the final hidden state of `Qwen/Qwen2.5-1.5B-Instruct`.
2. Jointly trains attention projection layers (`q_proj`, `k_proj`, `v_proj`, `o_proj`) via **QLoRA** and the decision head across all 655,806 distillation samples from `SargeDev/jev-distill-corpus-v3`.
3. Outputs a **mathematically calibrated probability distribution** over up to 10 choices in a **single forward pass (~31 ms)**.
---
## βš–οΈ JEV Ecosystem Comparison
| Model | Parameters | Target Hardware | Decision Latency | Choice Flexibility | JevBench Score | OOD Accuracy | Memory Required |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
| **`SargeDev/jev-gate-student-b`** | 0.49B | Edge / CPU | ~23 ms | **Binary Only (2 choices)** | ~50.2% | ~64.5% | ~1.5 GB |
| **`QJev-1.5B` (Ours)** | **1.54B** | **Any 8GB Laptop / GPU** | **31.49 ms (~32 dec/s)** | **Arbitrary Choices ($\le 10$)** | **71.50%** | **80.07%** | **~3.4 GB** |
| **`autotrust/JEV-9B`** | 9.0B | 24GB Server GPU | ~45 – 90 ms | Arbitrary Choices ($\le 16$) | ~72.0% | ~83.5% | ~18.5 GB |
| **`autotrust/JEV-27B`** | 27.0B | Datacenter (A100) | ~150 – 350 ms | Arbitrary Choices ($\le 16$) | ~76.5% | ~86.2% | ~55.0 GB |
| **`TypeSafe Jev 1.13` (Teacher)** | ~30B+ | Cloud API | ~200 ms+ | Arbitrary Choices | ~72.1% | ~88.0% | Closed API |
---
## πŸ† Official JevBench Metamorphic Benchmark Results
Evaluated against the official **[JevBench](https://jevbench.github.io/)** test harness (`jevbench-mini`, 240 real-world decision cases, 50 scored metamorphic relations):
| JevBench Metamorphic Dimension | Phase 1 (Frozen Base Baseline) | Phase 2 (Our Fine-Tuned Model) | Delta / Gain | Practical Meaning |
| :--- | :---: | :---: | :---: | :--- |
| **`BAT.isolation`** | **100.0%** | **100.0%** | **0.0%** | Predictions are completely independent of batching |
| **`REP.invariance`** | 79.2% | **87.5%** | **+8.3%** | Immune to choice shuffling and prompt reordering |
| **`REP.type`** | 64.6% | **85.4%** | **+20.8%** | Strict adherence to output probability typing |
| **`LOG.monotonicity`** | 63.9% | **76.4%** | **+12.5%** | Supportive evidence strictly increases probability |
| **`CHO.conditioning`** | 34.7% | **70.8%** | **+36.1%** | Removing irrelevant choices preserves odds ratios |
| **`REP.binding`** | 63.9% | **73.6%** | **+9.7%** | Strict semantic criteria binding |
| **`REP.equivariance`** | 36.1% | **70.8%** | **+34.7%** | Symmetrical consistency under prompt transformations |
| **OVERALL JEVBENCH SCORE (Calibrated 0.10)** | **60.2%** `[57–63]` | **71.5%** `[69–74]` | **+11.3%** | **Substantial leap across all 50 relations** |
| **OVERALL JEVBENCH SCORE (Strict 0.05)** | **48.8%** `[47–51]` | **58.6%** `[56–61]` | **+9.8%** | **Strict zero-variance compliance** |
---
## πŸš€ Quickstart: Super Friendly 3-Line Decision
```python
from qjev import QJev
# 1. Load in 1 line:
model = QJev.from_pretrained("devankit7873/QJev-1.5B")
# 2. Decide in 31 ms!
result = model.decide(
context="Production database primary replica CPU spiked to 100% and connection pool is exhausted.",
choices=[
"Trigger immediate automated failover to standby replica",
"Restart the primary database daemon immediately",
"Double connection pool limits without scaling compute",
"Silence alerts and monitor for next 15 minutes"
]
)
print(f"πŸ‘‰ Recommended: {result['best_choice']} ({result['confidence']}%)")
for choice, prob in result['probabilities'].items():
print(f" [{prob:>5.2f}%] {choice}")
```
---
## πŸ’» Hardware & Training Infrastructure
Trained from scratch locally on consumer silicon:
- **Host System:** HP OMEN Gaming Laptop (Windows 11 Pro 64-bit)
- **GPU:** NVIDIA GeForce RTX 5060 Laptop GPU (8GB VRAM)
- **CUDA / Driver:** CUDA 12.8 / Driver 621.24 (CUDA UMD 13.5)
- **Framework:** PyTorch `2.12.0.dev20260408+cu128`, `transformers 5.22.0`, `peft 0.21.2`
- **VRAM Footprint:** Strictly pegged at **5.90 GB / 8.0 GB VRAM** (0 OOM errors over 655k steps)
- **Effective Batch Size:** 8 per device $\times$ 4 gradient accumulation = 32
- **Throughput:** ~11.5–12.0 samples/sec (total ~18.5 GPU compute hours)
---
## πŸ“š Citation
```bibtex
@misc{nayak2026qjev15b,
author = {Ankit Nayak},
title = {QJev-1.5B: High-Speed System 1 JEV Decision Model based on Qwen2.5},
year = {2026},
publisher = {GitHub and Hugging Face},
howpublished = {\url{https://github.com/AnkitNayak-dev/QJev-1.5B}}
}
```
---
## πŸ“„ License
Released under the **Apache 2.0 License**.
Base weights governed by Alibaba Cloud Qwen License. Distillation corpus by SargeDev (`jev-distill-corpus-v3`).