metadata
language: en
license: apache-2.0
library_name: unsloth
base_model: unsloth/Qwen3.5-0.8B
tags:
- tool-calling
- distillation
- calibration
- research
- devops
Mini-Jev: Calibrated DevOps Tool-Calling Agent
Mini-Jev is a research project. It is a distilled version of the Beam-1.0-501B model. It is not officially affiliated with or endorsed by the creators of the original Beam model.
The Problem: Overconfidence
Most small models are "overconfident"—they will guess a tool even when they are unsure. In a production DevOps environment, a wrong tool call (like triggering a rollback by mistake) can be catastrophic.
The Solution: RLCD
This model was trained using Reinforcement Learning for Calibrated Decisions (RLCD). Instead of just training for accuracy, we optimized for:
- Cross-Entropy: Getting the right answer.
- Brier Score: Ensuring the probability matches the empirical truth.
- KL Divergence: Distilling the epistemic uncertainty of a 501B parameter teacher.
Calibration
The model is calibrated using post-hoc Temperature Scaling ($T^*$). This ensures that when the model says it is 70% confident, it is actually correct approximately 70% of the time.
Usage (System 1 / System 2 Cascade)
This model is designed to act as System 1.
- If confidence $\hat{p} \ge 0.7$, execute the tool immediately.
- If confidence $\hat{p} < 0.7$, escalate to a larger expert model (System 2).
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained("khide/mini-jev-rlcd")
FastLanguageModel.for_inference(model)
# Use the calibrated logits to route your requests!