QJev-1.5B-LoRA: PEFT Decision Adapter

Joint Attention Adapter and MLP Decision Head for Qwen2.5-1.5B-Instruct

Hugging Face GitHub JevBench Strict License


QJev Model Family:

Qwen/Qwen2.5-1.5B-Instruct (Base Model)
 โ””โ”€โ”€ devankit7873/QJev-1.5B (Flagship Fused 16-bit, 3.08 GB)
      โ”œโ”€โ”€ devankit7873/QJev-1.5B-LoRA (PEFT Adapter, 20 MB)
      โ””โ”€โ”€ devankit7873/QJev-1.5B-4bit (NF4 Quantized, 1.15 GB)

Overview

QJev-1.5B-LoRA is the parameter-efficient adapter release of QJev-1.5B. It is designed for multi-tenant, edge, and modular environments where the base model (Qwen/Qwen2.5-1.5B-Instruct) is already locally cached.

By downloading only 20.7 MB of weights (17.5 MB LoRA adapter + 3.18 MB MLP decision head), users can attach direct expected value decision inference onto an existing base model instance without duplicating base model storage.

Specifications

  • Adapter Parameters: Rank 16, Alpha 32 targeting attention projection layers (q_proj, k_proj, v_proj, o_proj).
  • Adapter File Size: 17.5 MB (adapter_model.safetensors).
  • Decision Head File Size: 3.18 MB (decision_head.safetensors, 1536 -> 512 -> 10).
  • Benchmark Evaluation: Evaluates identically to the merged model (62.53% strict tolerance 0.05 / 71.50% relaxed tolerance 0.10 on JevBench).

Quickstart

Python API

from qjev import QJev

# Load the PEFT adapter on top of cached Qwen base weights
model = QJev.from_pretrained("devankit7873/QJev-1.5B-LoRA")

# Execute decision inference
result = model.decide(
    context="Production database primary replica CPU utilization reached 100% and connection pool is exhausted.",
    choices=[
        "Trigger immediate automated failover to standby replica",
        "Restart the primary database daemon immediately",
        "Double connection pool limits without scaling compute",
        "Silence alerts and monitor for next 15 minutes"
    ]
)

print(f"Recommended Action: {result['best_choice']} ({result['confidence']}%)")
for choice, prob in result['probabilities'].items():
    print(f"  [{prob:>5.2f}%]  {choice}")

Output

Recommended Action: Trigger immediate automated failover to standby replica (88.28%)
  [88.28%]  Trigger immediate automated failover to standby replica
  [ 6.54%]  Restart the primary database daemon immediately
  [ 4.79%]  Double connection pool limits without scaling compute
  [ 0.45%]  Silence alerts and monitor for next 15 minutes

Architecture and Integration

When loaded, the adapter hooks into the transformer backbone via PEFT, and the final hidden state of the input sequence is routed through the LayerNorm and two-layer MLP projection head to produce normalized decision logits across the candidate options.


License

Released under the Apache 2.0 License. Base model weights are governed by the Alibaba Cloud Qwen License.

Downloads last month
38
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for AnkitNayak-dev/QJev-1.5B-LoRA

Adapter
(1)
this model

Dataset used to train AnkitNayak-dev/QJev-1.5B-LoRA