QJev-1.5B-4bit / README.md
devankit7873's picture
fix: establish correct model tree hierarchy in README metadata
640726e verified
|
Raw History Blame Contribute Delete
3.1 kB
metadata
license: apache-2.0
base_model: devankit7873/QJev-1.5B
library_name: transformers
tags:
  - transformers
  - bitsandbytes
  - 4-bit
  - quantized
  - pytorch
  - qwen
  - qwen2
  - decision-making
  - system-one
  - reasoning
  - jev
  - jevbench
  - joint-expected-value
  - classification
  - safetensors
  - agent
  - fast-inference
datasets:
  - SargeDev/jev-distill-corpus-v3
metrics:
  - accuracy
  - tvd
  - kl_divergence
pipeline_tag: text-classification

⚑ QJev-1.5B-4bit: 4-Bit NF4 Quantized Decision Model

High-Speed System 1 Decision Engine (1.15 GB Weights β€’ 1.07 GB VRAM)

Hugging Face GitHub JevBench License


πŸ“¦ Part of the QJev Model Family:


πŸ’‘ Overview

QJev-1.5B-4bit is the ultra-efficient 4-bit NormalFloat (NF4) quantized release of QJev-1.5B.

By compressing the 1.54B parameter transformer into 4-bit precision via bitsandbytes, the memory footprint drops from 3.4 GB down to 1.07 GB VRAM, enabling real-time System 1 decisions on ultra-low-cost cloud instances, edge devices, and consumer laptops while preserving calibrated decision accuracy.

  • Storage Size: 1.15 GB (model.safetensors)
  • Active VRAM Footprint: 1.07 GB
  • Decision Speed: ~35 ms forward pass
  • JevBench Metamorphic Accuracy: Maintained within margin of full precision.

πŸš€ Quickstart: Super Friendly 3-Line Decision

from qjev import QJev

# 1. Load the 4-bit model in 1 line (only 1.07 GB VRAM):
model = QJev.from_pretrained("devankit7873/QJev-1.5B-4bit")

# 2. Decide in 35 ms!
result = model.decide(
    context="Production database primary replica CPU spiked to 100% and connection pool is exhausted.",
    choices=[
        "Trigger immediate automated failover to standby replica",
        "Restart the primary database daemon immediately",
        "Double connection pool limits without scaling compute",
        "Silence alerts and monitor for next 15 minutes"
    ]
)

print(f"πŸ‘‰ Recommended: {result['best_choice']} ({result['confidence']}%)")
for choice, prob in result['probabilities'].items():
    print(f"  [{prob:>5.2f}%]  {choice}")

πŸ“„ License

Released under the Apache 2.0 License.