Text Generation
Transformers
Safetensors
PEFT
English
unsloth
qwen2
trl
lora
missionctrl
ai-oversight
conversational
Instructions to use Jit-fnc/missionctrl_env with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Jit-fnc/missionctrl_env with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Jit-fnc/missionctrl_env") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Jit-fnc/missionctrl_env", device_map="auto") - PEFT
How to use Jit-fnc/missionctrl_env with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Jit-fnc/missionctrl_env with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Jit-fnc/missionctrl_env" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Jit-fnc/missionctrl_env", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Jit-fnc/missionctrl_env
- SGLang
How to use Jit-fnc/missionctrl_env with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Jit-fnc/missionctrl_env" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Jit-fnc/missionctrl_env", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Jit-fnc/missionctrl_env" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Jit-fnc/missionctrl_env", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Unsloth Desktop
- Docker Model Runner
How to use Jit-fnc/missionctrl_env with Docker Model Runner:
docker model run hf.co/Jit-fnc/missionctrl_env
File size: 4,781 Bytes
1b18f9d 4c460b2 1b18f9d 4c460b2 1b18f9d 4c460b2 1b18f9d 4c460b2 1b18f9d 4c460b2 1b18f9d 4c460b2 1b18f9d 4c460b2 8723187 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 | ---
base_model: unsloth/qwen2.5-0.5b-instruct-unsloth-bnb-4bit
tags:
- text-generation
- transformers
- unsloth
- qwen2
- trl
- peft
- lora
- missionctrl
- ai-oversight
license: apache-2.0
language:
- en
pipeline_tag: text-generation
---
# MissionCtrl Overseer (Qwen2.5-0.5B LoRA)
- **Developed by:** Jit-fnc
- **Model type:** LoRA adapter (PEFT) for Qwen2.5-0.5B-Instruct (4-bit Unsloth base)
- **License:** Apache-2.0
- **Finetuned from:** `unsloth/qwen2.5-0.5b-instruct-unsloth-bnb-4bit`
- **Training framework:** Unsloth + TRL (GRPO)
This model is a task-specific overseer policy trained for the **MissionCtrl** environment: a multi-agent software simulation where the overseer must detect and manage hallucination risk (approve/reject/flag/redelegate/escalate/synthesize actions).
---
## Model description
This adapter is optimized for **structured oversight actions** in MissionCtrl episodes, not generic chat quality.
Training emphasizes:
- Hallucination detection under curriculum difficulty (easy -> medium -> hard)
- Low false-positive behavior
- Action validity and episode-level reward optimization
---
## Training details
### Base + method
- Base: `unsloth/qwen2.5-0.5b-instruct-unsloth-bnb-4bit`
- Finetuning: LoRA/QLoRA via Unsloth
- RL algorithm: GRPO (TRL)
### Run snapshot (from training logs)
- Hardware: 1x T4 (Colab run snapshot)
- Curriculum steps: easy 200, medium 220, hard 180
- Reward ceiling reference: 0.85
### Post-phase eval (10 episodes each)
- **Easy:** reward `0.289 ± 0.137`, detect `70.0%`, fp `0.0%`
- **Medium:** reward `0.067 ± 0.107`, detect `10.0%`, fp `0.0%`
- **Hard:** reward `0.172 ± 0.024`, detect `0.0%`, fp `0.0%`
Rounded curriculum means often shown in docs: **0.29 / 0.07 / 0.17**.
---
## Intended use
### Recommended
- MissionCtrl/OpenEnv-style oversight simulation
- Research on hallucination containment and supervisory policies
- Benchmarking action policies in multi-agent pipelines
### Not recommended
- General-purpose assistant/chatbot usage
- Factual QA without environment constraints
- Safety-critical deployment without independent validation
---
## Prompt / output format expectations
Best results are obtained when prompts follow MissionCtrl-style action grammar and environment context.
The policy is tuned to produce short structured actions such as:
- `APPROVE(task_id)`
- `REJECT(task_id, reason)`
- `FLAG(task_id, evidence)`
- `REDELEGATE(task_id, agent)`
- `ESCALATE(task_id)`
- `SYNTHESIZE_REPORT()`
- `NOOP`
---
## Limitations
- Performance drops on higher difficulty tiers (medium/hard) in current snapshot.
- Curriculum reward and full greedy rollout eval can diverge during early RL stages.
- Domain transfer outside MissionCtrl may be poor.
- Small model capacity (0.5B) limits robustness to long-horizon reasoning.
---
## Bias, risks, and safety
This model is specialized for synthetic oversight tasks and may:
- Overfit to simulator-specific patterns
- Underperform on unseen hallucination styles
- Produce confident but incorrect actions outside its trained setting
Use with human monitoring and external evaluation before broader use.
---
## Usage (PEFT)
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_id = "unsloth/qwen2.5-0.5b-instruct-unsloth-bnb-4bit"
adapter_id = "Jit-fnc/missionctrl_env" # replace if repo name differs
tokenizer = AutoTokenizer.from_pretrained(base_id)
base_model = AutoModelForCausalLM.from_pretrained(base_id)
model = PeftModel.from_pretrained(base_model, adapter_id)
prompt = "You are the MissionCtrl Overseer. Return one valid action."
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=64)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
---
## Hugging Face Inference Endpoints (`task=custom`)
This repo includes [`handler.py`](handler.py) and [`requirements.txt`](requirements.txt) so a dedicated endpoint configured with **custom** task can load the LoRA adapter.
- **Base weights at runtime:** `Qwen/Qwen2.5-0.5B-Instruct` (non-quantized). Override with env `BASE_MODEL` if needed.
- **Adapter + tokenizer:** loaded from the endpoint’s `/repository` (this model repo).
**Deploy:** commit and push `handler.py` and `requirements.txt` to the Hugging Face model repo backing the endpoint, then restart or redeploy the endpoint.
**Clients (e.g. MissionCtrl `inference.py`):**
- Set `MISSIONCTRL_HF_LLM_STRATEGY=native_only` so requests use the root URL with `{"inputs": "...", "parameters": {...}}` instead of relying on `/v1/chat/completions` (custom handlers do not expose OpenAI routes by default).
- The handler returns `[{"generated_text": "..."}]`, matching native HF generation parsing. |