SargeDev/jev-distill-corpus-v3
Viewer • Updated • 741k • 1.96k • 50
How to use devankit7873/QJev-1.5B-4bit with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("text-classification", model="devankit7873/QJev-1.5B-4bit") # pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("devankit7873/QJev-1.5B-4bit")
model = AutoModelForCausalLM.from_pretrained("devankit7873/QJev-1.5B-4bit", device_map="auto")# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("devankit7873/QJev-1.5B-4bit")
model = AutoModelForCausalLM.from_pretrained("devankit7873/QJev-1.5B-4bit", device_map="auto")QJev-1.5B-4bit is the ultra-efficient 4-bit NormalFloat (NF4) quantized release of QJev-1.5B.
By compressing the 1.54B parameter transformer into 4-bit precision via bitsandbytes, the memory footprint drops from 3.4 GB down to 1.07 GB VRAM, enabling real-time System 1 decisions on ultra-low-cost cloud instances, edge devices, and consumer laptops while preserving calibrated decision accuracy.
model.safetensors)from qjev import QJev
# 1. Load the 4-bit model in 1 line (only 1.07 GB VRAM):
model = QJev.from_pretrained("devankit7873/QJev-1.5B-4bit")
# 2. Decide in 35 ms!
result = model.decide(
context="Production database primary replica CPU spiked to 100% and connection pool is exhausted.",
choices=[
"Trigger immediate automated failover to standby replica",
"Restart the primary database daemon immediately",
"Double connection pool limits without scaling compute",
"Silence alerts and monitor for next 15 minutes"
]
)
print(f"👉 Recommended: {result['best_choice']} ({result['confidence']}%)")
for choice, prob in result['probabilities'].items():
print(f" [{prob:>5.2f}%] {choice}")
Released under the Apache 2.0 License.
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="devankit7873/QJev-1.5B-4bit")