QJev-1.5B-4bit / README.md
devankit7873's picture
fix: establish correct model tree hierarchy in README metadata
640726e verified
|
Raw History Blame Contribute Delete
3.1 kB
---
license: apache-2.0
base_model: devankit7873/QJev-1.5B
library_name: transformers
tags:
- transformers
- bitsandbytes
- 4-bit
- quantized
- pytorch
- qwen
- qwen2
- decision-making
- system-one
- reasoning
- jev
- jevbench
- joint-expected-value
- classification
- safetensors
- agent
- fast-inference
datasets:
- SargeDev/jev-distill-corpus-v3
metrics:
- accuracy
- tvd
- kl_divergence
pipeline_tag: text-classification
---
<div align="center">
# โšก QJev-1.5B-4bit: 4-Bit NF4 Quantized Decision Model
### High-Speed System 1 Decision Engine (1.15 GB Weights โ€ข 1.07 GB VRAM)
[![Hugging Face](https://img.shields.io/badge/Hugging%20Face-QJev--1.5B--4bit-FFD21E?style=for-the-badge&logo=huggingface&logoColor=black)](https://huggingface.co/devankit7873/QJev-1.5B-4bit)
[![GitHub](https://img.shields.io/badge/GitHub-QJev--1.5B-181717?style=for-the-badge&logo=github)](https://github.com/AnkitNayak-dev/QJev-1.5B)
[![JevBench](https://img.shields.io/badge/JevBench-71.5%25%20Score-8A2BE2?style=for-the-badge)](https://jevbench.github.io/)
[![License](https://img.shields.io/badge/License-Apache%202.0-2EA44F?style=for-the-badge)](LICENSE)
---
### ๐Ÿ“ฆ Part of the QJev Model Family:
- **Flagship Standalone (Fused):** [devankit7873/QJev-1.5B](https://huggingface.co/devankit7873/QJev-1.5B) (3.08 GB)
- **Lightweight LoRA Adapter:** [devankit7873/QJev-1.5B-LoRA](https://huggingface.co/devankit7873/QJev-1.5B-LoRA) (20 MB)
- **4-Bit Quantized:** [devankit7873/QJev-1.5B-4bit](https://huggingface.co/devankit7873/QJev-1.5B-4bit) (1.15 GB, 1.07 GB VRAM)
---
</div>
## ๐Ÿ’ก Overview
**`QJev-1.5B-4bit`** is the **ultra-efficient 4-bit NormalFloat (NF4) quantized release** of `QJev-1.5B`.
By compressing the 1.54B parameter transformer into 4-bit precision via `bitsandbytes`, the memory footprint drops from 3.4 GB down to **1.07 GB VRAM**, enabling real-time System 1 decisions on ultra-low-cost cloud instances, edge devices, and consumer laptops while preserving calibrated decision accuracy.
- **Storage Size:** **1.15 GB** (`model.safetensors`)
- **Active VRAM Footprint:** **1.07 GB**
- **Decision Speed:** ~35 ms forward pass
- **JevBench Metamorphic Accuracy:** Maintained within margin of full precision.
---
## ๐Ÿš€ Quickstart: Super Friendly 3-Line Decision
```python
from qjev import QJev
# 1. Load the 4-bit model in 1 line (only 1.07 GB VRAM):
model = QJev.from_pretrained("devankit7873/QJev-1.5B-4bit")
# 2. Decide in 35 ms!
result = model.decide(
context="Production database primary replica CPU spiked to 100% and connection pool is exhausted.",
choices=[
"Trigger immediate automated failover to standby replica",
"Restart the primary database daemon immediately",
"Double connection pool limits without scaling compute",
"Silence alerts and monitor for next 15 minutes"
]
)
print(f"๐Ÿ‘‰ Recommended: {result['best_choice']} ({result['confidence']}%)")
for choice, prob in result['probabilities'].items():
print(f" [{prob:>5.2f}%] {choice}")
```
---
## ๐Ÿ“„ License
Released under the **Apache 2.0 License**.