Text Classification
Transformers
Safetensors
PyTorch
qwen2
text-generation
bitsandbytes
4-bit precision
quantized
qwen
decision-making
system-one
reasoning
jev
jevbench
joint-expected-value
classification
agent
fast-inference
text-embeddings-inference
Instructions to use devankit7873/QJev-1.5B-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use devankit7873/QJev-1.5B-4bit with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="devankit7873/QJev-1.5B-4bit")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("devankit7873/QJev-1.5B-4bit") model = AutoModelForCausalLM.from_pretrained("devankit7873/QJev-1.5B-4bit", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from devankit7873/QJev-1.5B-4bit: direct link, hf CLI and curl.
- Browser
- Download file 3.1 kB
-
https://huggingface.co/devankit7873/QJev-1.5B-4bit/resolve/main/README.md
- Command line
-
hf download hf://devankit7873/QJev-1.5B-4bit/README.md
-
curl -L -o README.md https://huggingface.co/devankit7873/QJev-1.5B-4bit/resolve/main/README.md
3.1 kB
| license: apache-2.0 | |
| base_model: devankit7873/QJev-1.5B | |
| library_name: transformers | |
| tags: | |
| - transformers | |
| - bitsandbytes | |
| - 4-bit | |
| - quantized | |
| - pytorch | |
| - qwen | |
| - qwen2 | |
| - decision-making | |
| - system-one | |
| - reasoning | |
| - jev | |
| - jevbench | |
| - joint-expected-value | |
| - classification | |
| - safetensors | |
| - agent | |
| - fast-inference | |
| datasets: | |
| - SargeDev/jev-distill-corpus-v3 | |
| metrics: | |
| - accuracy | |
| - tvd | |
| - kl_divergence | |
| pipeline_tag: text-classification | |
| <div align="center"> | |
| # โก QJev-1.5B-4bit: 4-Bit NF4 Quantized Decision Model | |
| ### High-Speed System 1 Decision Engine (1.15 GB Weights โข 1.07 GB VRAM) | |
| [](https://huggingface.co/devankit7873/QJev-1.5B-4bit) | |
| [](https://github.com/AnkitNayak-dev/QJev-1.5B) | |
| [](https://jevbench.github.io/) | |
| [](LICENSE) | |
| --- | |
| ### ๐ฆ Part of the QJev Model Family: | |
| - **Flagship Standalone (Fused):** [devankit7873/QJev-1.5B](https://huggingface.co/devankit7873/QJev-1.5B) (3.08 GB) | |
| - **Lightweight LoRA Adapter:** [devankit7873/QJev-1.5B-LoRA](https://huggingface.co/devankit7873/QJev-1.5B-LoRA) (20 MB) | |
| - **4-Bit Quantized:** [devankit7873/QJev-1.5B-4bit](https://huggingface.co/devankit7873/QJev-1.5B-4bit) (1.15 GB, 1.07 GB VRAM) | |
| --- | |
| </div> | |
| ## ๐ก Overview | |
| **`QJev-1.5B-4bit`** is the **ultra-efficient 4-bit NormalFloat (NF4) quantized release** of `QJev-1.5B`. | |
| By compressing the 1.54B parameter transformer into 4-bit precision via `bitsandbytes`, the memory footprint drops from 3.4 GB down to **1.07 GB VRAM**, enabling real-time System 1 decisions on ultra-low-cost cloud instances, edge devices, and consumer laptops while preserving calibrated decision accuracy. | |
| - **Storage Size:** **1.15 GB** (`model.safetensors`) | |
| - **Active VRAM Footprint:** **1.07 GB** | |
| - **Decision Speed:** ~35 ms forward pass | |
| - **JevBench Metamorphic Accuracy:** Maintained within margin of full precision. | |
| --- | |
| ## ๐ Quickstart: Super Friendly 3-Line Decision | |
| ```python | |
| from qjev import QJev | |
| # 1. Load the 4-bit model in 1 line (only 1.07 GB VRAM): | |
| model = QJev.from_pretrained("devankit7873/QJev-1.5B-4bit") | |
| # 2. Decide in 35 ms! | |
| result = model.decide( | |
| context="Production database primary replica CPU spiked to 100% and connection pool is exhausted.", | |
| choices=[ | |
| "Trigger immediate automated failover to standby replica", | |
| "Restart the primary database daemon immediately", | |
| "Double connection pool limits without scaling compute", | |
| "Silence alerts and monitor for next 15 minutes" | |
| ] | |
| ) | |
| print(f"๐ Recommended: {result['best_choice']} ({result['confidence']}%)") | |
| for choice, prob in result['probabilities'].items(): | |
| print(f" [{prob:>5.2f}%] {choice}") | |
| ``` | |
| --- | |
| ## ๐ License | |
| Released under the **Apache 2.0 License**. | |