QJev-1.5B: High-Speed System 1 JEV Decision Model

Single-Pass Joint Expected Value Decision Engine

Hugging Face GitHub JevBench Strict JevBench Relaxed License


Qwen/Qwen2.5-1.5B-Instruct (Base Model)
 โ””โ”€โ”€ devankit7873/QJev-1.5B (Flagship Fused 16-bit, 3.08 GB)
      โ”œโ”€โ”€ devankit7873/QJev-1.5B-LoRA (PEFT Adapter, 20 MB)
      โ””โ”€โ”€ devankit7873/QJev-1.5B-4bit (NF4 Quantized, 1.15 GB)

Overview โ€ข Benchmark Evaluation โ€ข Model Family โ€ข Quickstart โ€ข Serving and Deployment โ€ข Architecture โ€ข Repository Structure โ€ข Citation


Overview

Traditional large language models perform decision-making through sequential, autoregressive token generation. This process incurs substantial latency (often 300 to 600 ms) and introduces non-deterministic sampling variance when parsing formatted outputs.

QJev-1.5B implements a direct System 1 decision mechanism by coupling a calibrated Multi-Layer Perceptron (MLP) decision head to the final hidden representation of Qwen/Qwen2.5-1.5B-Instruct. Rather than generating text tokens, the model computes a calibrated probability distribution across up to 10 candidate actions in a single forward pass (~31 ms on consumer hardware).

Key Characteristics

  • Single Forward Pass Execution: Decision latency of 31 ms (32 decisions/second per device).
  • Calibrated Probability Distribution: Direct outputs represent normalized decision probabilities rather than heuristic sequence log-probabilities.
  • Dynamic Choice Support: Accommodates arbitrary choice cardinalities from 2 up to 10 options with dynamic logit masking.
  • Consistency Regularization: Aligned via Phase 3 soft-target symmetric cross-format consistency loss to ensure stability across option permutations, boolean probes, and coarsened subsets.

Benchmark Evaluation

QJev-1.5B is evaluated against the official JevBench metamorphic test harness (jevbench-mini, 240 evaluation cases, 50 scored relations, 1,468 trials) following standard BenchmarkHeaven submission criteria.

BenchmarkHeaven Leaderboard Comparison

All models evaluated under official strict calibration criteria (tolerance = 0.05):

Model Parameters Official Score (tol=0.05) Category Inference Latency Memory Footprint
EXAONE-1.2B 1.2B 47.7% Sub-2B ~25 ms ~2.8 GB
SargeDev/jev-gate-student-b 0.49B 50.2% Sub-2B (Binary Only) ~23 ms ~1.5 GB
decision-2b 2.0B 54.8% 2B Tier ~40 ms ~4.5 GB
pjev-2b 2.0B 56.5% 2B Tier ~38 ms ~4.4 GB
QJev-1.5B (Ours, Phase 3) 1.54B 62.53% Rank 1 Sub-2B ~31 ms 3.4 GB (1.07 GB in 4-bit)
jiwo-4b 4.0B 60.2% 4B Tier ~65 ms ~8.8 GB
decision-4b-v12 4.0B 60.4% 4B Tier ~62 ms ~8.5 GB
autotrust/JEV-9B 9.0B 72.0% 9B Tier ~85 ms ~18.5 GB
autotrust/JEV-27B 27.0B 76.5% Datacenter Tier ~220 ms ~55.0 GB

Tolerance specification: The official BenchmarkHeaven evaluation suite strictly uses tolerance = 0.05 for headline Capability scores. Under exploratory relaxed evaluation (tolerance = 0.10), QJev-1.5B achieves 71.50%.

Metamorphic Consistency Breakdown

Dimension Baseline (Pre-Consistency) QJev-1.5B (Phase 3 Consistency) Delta Formal Property Evaluated
BAT.isolation 100.0% 100.0% 0.0% Invariance to batch composition and sample ordering
REP.type 77.1% 83.3% +6.2% Invariance across prompt format representations (MCQ, boolean, ordinal)
REP.invariance 73.2% 74.4% +1.2% Stability under option key renaming and syntax perturbations
LOG.bounds 62.5% 67.9% +5.4% Adherence to probabilistic upper and lower bounds
CHO.conditioning 41.7% 56.9% +15.2% Preservation of relative odds upon elimination of irrelevant options
REP.equivariance 48.6% 55.6% +7.0% Consistent probability shifts under negation transformations
MEA.partition 30.7% 41.1% +10.4% Probability mass conservation under option clustering
Overall (Official Strict 0.05) 58.67% 62.53% +3.86% Standard JevBench Capability Benchmark

Machine-readable evaluation artifacts are archived under reports/jevbench/.


Model Family

QJev-1.5B is available in three deployment formats:

Format Hugging Face Repository Disk Size VRAM Footprint Recommended Use Case
Full Fused (FP16) devankit7873/QJev-1.5B 3.08 GB 3.4 GB Production servers, vLLM, standalone microservices
PEFT Adapter devankit7873/QJev-1.5B-LoRA 20.7 MB Shared Multi-tenant deployments caching Qwen2.5-1.5B-Instruct
NF4 Quantized devankit7873/QJev-1.5B-4bit 1.15 GB 1.07 GB Resource-constrained edge instances, local workstations

Quickstart

Installation

git clone https://github.com/AnkitNayak-dev/QJev-1.5B.git
cd QJev-1.5B
pip install -r requirements.txt

Python API

from qjev import QJev

# Load any model variant directly from Hugging Face Hub
model = QJev.from_pretrained("devankit7873/QJev-1.5B")
# Or 4-bit quantized: model = QJev.from_pretrained("devankit7873/QJev-1.5B-4bit")
# Or PEFT adapter:    model = QJev.from_pretrained("devankit7873/QJev-1.5B-LoRA")

# Execute decision inference
result = model.decide(
    context="Production database primary replica CPU utilization reached 100% and connection pool is exhausted.",
    choices=[
        "Trigger immediate automated failover to standby replica",
        "Restart the primary database daemon immediately",
        "Double connection pool limits without scaling compute",
        "Silence alerts and monitor for next 15 minutes"
    ]
)

print(f"Recommended Action: {result['best_choice']} ({result['confidence']}%)")
for choice, prob in result['probabilities'].items():
    print(f"  [{prob:>5.2f}%]  {choice}")

Output

Recommended Action: Trigger immediate automated failover to standby replica (88.28%)
  [88.28%]  Trigger immediate automated failover to standby replica
  [ 6.54%]  Restart the primary database daemon immediately
  [ 4.79%]  Double connection pool limits without scaling compute
  [ 0.45%]  Silence alerts and monitor for next 15 minutes

Command-Line Interface

python qjev.py --context "Patient presents with acute chest pain and elevated troponin levels." --choices "Emergency PCI" "Aspirin" "Observation"

Serving and Deployment

1. Unified Microservice Server (serve.py)

A production FastAPI service providing REST, Jev protocol, and OpenAI-compatible endpoints:

python serve.py --model devankit7873/QJev-1.5B --port 8000

Endpoints

  • POST /v1/decide - Direct decision endpoint returning structured probabilities.
  • POST /v1/systemone - Official BenchmarkHeaven JEV evaluation protocol.
  • POST /v1/chat/completions - Drop-in OpenAI-compatible wrapper.

Query via cURL

curl -X POST http://localhost:8000/v1/decide \
  -H "Content-Type: application/json" \
  -d '{
    "context": "Customer requests immediate refund after policy window has expired.",
    "options": ["Approve refund", "Issue store credit", "Deny request"]
  }'

2. Ollama Integration

A preconfigured Modelfile is provided in the repository root:

ollama create qjev-1.5b -f Modelfile
ollama run qjev-1.5b

Querying via the Python SDK:

from qjev import QJev

client = QJev.from_ollama("http://localhost:11434", model="qjev-1.5b")
result = client.decide("Route network traffic to cluster Alpha or Beta?", ["Alpha", "Beta"])

3. llama.cpp and GGUF Export

Convert weights to GGUF format for edge CPU and local deployment:

python scripts/export_gguf.py --model-dir export/QJev-1.5B-Phase3 --out-file qjev-1.5b-f16.gguf
./llama-quantize qjev-1.5b-f16.gguf qjev-1.5b-q4_k_m.gguf Q4_K_M

4. vLLM Serving

vllm serve devankit7873/QJev-1.5B --task classify --port 8000

Architecture

[Prompt Context + Formatted Candidate Choices]
                       โ”‚
                       โ–ผ
           โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
           โ”‚ Qwen2.5-1.5B Backbone  โ”‚  (QLoRA adapted: q, k, v, o projections)
           โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚ (Final Hidden State: 1536 dim)
                       โ–ผ
           โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
           โ”‚    LayerNorm(1536)     โ”‚
           โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚
                       โ–ผ
           โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
           โ”‚  Linear(1536 -> 512)   โ”‚  (GELU Activation, Dropout 0.1)
           โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚
                       โ–ผ
           โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
           โ”‚   Linear(512 -> 10)    โ”‚  (Dynamic Logit Masking for N Choices)
           โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
                       โ”‚
                       โ–ผ
         [Calibrated Softmax Probabilities]

Training and Implementation Details

  • Base Architecture: Qwen/Qwen2.5-1.5B-Instruct
  • Training Corpus: SargeDev/jev-distill-corpus-v3 (655,806 distillation examples)
  • Training Stages:
    • Phase 1 (MLP Head Initialization): 50,000 steps with frozen backbone weights.
    • Phase 2 (QLoRA Joint Optimization): Full-corpus training across 655,806 samples with rank-16 LoRA adapters targeting all attention projections.
    • Phase 3 (Cross-Format Consistency Alignment): 1,250 steps of symmetric KL-divergence minimization across paired multiple-choice, boolean, and subset queries.
  • Hardware Configuration: NVIDIA GeForce RTX 5060 Laptop GPU (8GB VRAM), CUDA 12.8, PyTorch 2.12 with TF32 precision.

Repository Structure

โ”œโ”€โ”€ README.md                           # Project documentation and benchmarks
โ”œโ”€โ”€ LICENSE                             # Apache 2.0 open-source license
โ”œโ”€โ”€ requirements.txt                    # Python dependencies
โ”œโ”€โ”€ Modelfile                           # Ollama configuration
โ”œโ”€โ”€ qjev.py                             # Unified Python SDK and CLI client
โ”œโ”€โ”€ serve.py                            # Production FastAPI service
โ”œโ”€โ”€ predict.py                          # Forward pass evaluation script
โ”œโ”€โ”€ handler.py                          # Hugging Face inference handler
โ”‚
โ”œโ”€โ”€ benchmark/                          # Benchmark evaluation suites
โ”‚   โ”œโ”€โ”€ run_jevbench.py                 # JevBench test harness runner
โ”‚   โ”œโ”€โ”€ self_test_benchmarkheaven.py    # Official BenchmarkHeaven evaluation runner
โ”‚   โ”œโ”€โ”€ self_test_phase3.py             # Phase 3 verification runner
โ”‚   โ””โ”€โ”€ self_test_step1.py              # Calibration sweep script
โ”‚
โ”œโ”€โ”€ reports/                            # Empirical evaluation logs
โ”‚   โ””โ”€โ”€ jevbench/                       # Benchmark records
โ”‚       โ”œโ”€โ”€ self_test_phase3_consistency.json # Official 62.53% report
โ”‚       โ”œโ”€โ”€ jevbench_phase1_report.json
โ”‚       โ””โ”€โ”€ jevbench_phase2_report.json
โ”‚
โ”œโ”€โ”€ src/                                # Model definitions and training routines
โ”‚   โ”œโ”€โ”€ model.py                        # MLP Decision Head and Qwen integration
โ”‚   โ”œโ”€โ”€ dataset.py                      # Streaming ChatML dataset loader
โ”‚   โ”œโ”€โ”€ train.py                        # Phase 1 initialization
โ”‚   โ”œโ”€โ”€ train_phase2.py                 # Phase 2 QLoRA training
โ”‚   โ”œโ”€โ”€ train_phase3_consistency.py     # Phase 3 consistency alignment
โ”‚   โ””โ”€โ”€ evaluate.py                     # Metric evaluation routines
โ”‚
โ””โ”€โ”€ scripts/                            # Utilities and deployment tools
    โ”œโ”€โ”€ build_consistency_dataset.py    # Consistency dataset generator
    โ”œโ”€โ”€ merge_phase3.py                 # Weight fusing and export
    โ”œโ”€โ”€ merge_and_export.py             # Model export routine
    โ”œโ”€โ”€ export_gguf.py                  # GGUF conversion utility
    โ”œโ”€โ”€ publish_all_hf.py               # Hugging Face hub publishing script
    โ”œโ”€โ”€ download_data.py                # Dataset download helper
    โ””โ”€โ”€ download_model.py               # Base model download helper

Citation

@misc{nayak2026qjev15b,
  author = {Ankit Nayak},
  title = {QJev-1.5B: High-Speed System 1 JEV Decision Model based on Qwen2.5},
  year = {2026},
  publisher = {GitHub and Hugging Face},
  howpublished = {\url{https://github.com/AnkitNayak-dev/QJev-1.5B}}
}

License

This project is licensed under the Apache 2.0 License. Base model weights are governed by the Alibaba Cloud Qwen License. Training data is provided by SargeDev (jev-distill-corpus-v3).

Downloads last month
94
Safetensors
Model size
2B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for AnkitNayak-dev/QJev-1.5B

Finetuned
(2,158)
this model
Adapters
1 model
Quantizations
1 model

Dataset used to train AnkitNayak-dev/QJev-1.5B